Azure data engineer (Pittsburgh) at Cognizant

Job Title - DataBricks Data EngineerLocation - Pittsburg, PA**Must have 8+ years of real hands on experience**We are specifically seeking a Data Engineer-Lead with strong expertise in Databricks development. The role involves:Building and testing data pipelines using Python/Scala on DatabricksHands on experience to develop and lead the offshore team to perform development/testing work in Azure data bricks Architect data platforms using Azure services such as Azure Data Factory (ADF), Azure Databricks (ADB), Azure SQL Database, and PySpark.Collaborate with stakeholders to understand business needs and translate them into technical solutions.Provide technical leadership and guidance to the data engineering team and need to perform development.Familiar with Safe Agile concepts and good to have working experience in agile model.Develop and maintain data pipelines for efficient data movement and transformation.Onsite and offshore team communication and co-ordination.Create and update the documentation to facilitate cross-training and troubleshootingHands on experience in scheduling tools like BMC control-M and setup jobs and test the schedules.Understand the data models and schemas to support the development work and help in creation of tables in databricksProficiency in Azure Data Factory (ADF), Azure Databricks (ADB), SQL, NoSQL, PySpark, Power BI and other Azure data tools.Implementing automated data validation frameworks such as Great Expectations or DeequReconciling large-scale datasetsEnsuring data reliability across both batch and streaming processes The ideal candidate will have hands-on experience with:PySpark, Scala, Delta Lake, and Unity CatalogDevops CI/CD automationCloud-native data servicesAzure databricks/OracleBMC Control-MLocation: Pittsburgh, PA