Data Engineer at Conduent

DescriptionData EngineerAbout the RoleAs a Data Engineer youll play an integral role in building the roadmap of the companys technology strategy.ResponsibilitiesEngineer robust AZURE data factory pipelines using Linked Services/Datasets to ingest high volume diverse data from disparate source systems into Azure BLOB storageDesign and implementation of scalable data pipelines using Azure Databricks and Azure Data factory adhering to Medallion architecture principles to ensure data integrity consistency and performanceDevelop complex & advanced transformation logic using PySpark (Python)user-defined functions (UDFs) within Azure Databricks to design and implement data processing and ETL workflow enabling efficient transformation of heterogeneous source data into an integrated advanced Benefit and Eligibility systemCreating modular and reusable notebooks in Azure Databricks utilizing functions parameters widgets for dynamic data processing and various python libraries such Panda numpyPyspark SQL SQL alchemy etc.Developing highly optimized Databrick notebooks using PySpark (Python & SQL) for advanced data joining data filtering pre-aggregation and processing of large data sets.Working with SQL Server and Oracle writing optimized queries with CTEs ANSI SQL window functions Views materialized Views stored procedures and Oracle-specific syntax to perform data profiling validation balancing reconciliation to ensure data accuracy (data quality) and data completeness.Solving complex data engineering challenges including Delta merge for hundreds of millions of records deduplication performance tuning and improving ETL run time efficiency using partitioning strategies Z-Ordering etc.Closely working with Client and Business analysts to define and assess complex data conversion and data integration requirements translating business needs into technical solutions.RequirementsBachelors Degree in Computer Science or related fieldTotal experience 10 as data analyst/engineerFederal government or State level experience2 years experience in Health and Huma Services programs (SNAP and TANF)Flexible WorkingAt Conduent we want you to be yourself. We recognize that everyone is different and that how people want to work and deliver at their best is different for everyone too. In this role you can expect the following working conditions:Remote work: Enjoy the convenience of working from home and maximize your time by unplugging at the end of your workday.Working For YouPerks and rewards designed for you:Career Growth Opportunities: We help you thrive so together we can grow. We provide opportunities to advance your career with a vast portfolio of businesses and a global footprint.Great Work Environment: We are proud of our award-winning culture and the recognition weve received for our diversity efforts.Join UsAt Conduent we are one team one mission. We understand that our success is directly related to the success of our associates. We strive to create a culture where you can: Bring your authentic self to work Grow and thrive both personally and professionally Make a difference with our clients in our communities and with the millions of people we support When you join Conduent you are engaged in creating the future - both our companys and your own. With more than 60000 associates across 24 countries we will provide you the opportunity to grow with a team of people who will challenge and inspire you to be the best!Pay Transparency Laws in some locations require disclosure of compensation and/or benefits-related information. For this position actual salaries will vary and may be above or below the range based on various factors including but not limited to location experience and addition to base pay this position based on business need may be eligible for a bonus or addition Conduent provides a variety of benefits to employees including health insurance coverage voluntary dental and vision programs life and disability insurance a retirement savings plan paid holidays and paid time off (PTO) or vacation and/or sick time. The estimated salary range for this role is$96250 - $125000 Key Skills Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala Employment Type : Full Time Experience: years Vacancy: 1 Monthly Salary Salary: 96250 - 125000