Senior Data Engineer (With Python Experience) at Augment LLC

Job DescriptionWe are seeking a skilled Senior Data Engineer with a solid background in data engineering and software engineering and exposure to machine learning. The ideal candidate will work with Polars as the primary data frame library and contribute to building testing and optimizing data-driven solutions.Key ResponsibilitiesDevelop robust data transformations using Python and Polars for efficient processing and analytics tasks.Hands-on experience in data engineeringWrite clean maintainable and efficient code with a strong emphasis on software engineering principles.Gather data from diverse sources transforming it efficiently and delivering clean reliable datasets and reports that support critical business decision-making.Implement and maintain unit testing and end-to-end testing.Utilize Docker for testing and development environments.Manage data infrastructure on GCPApply a foundational understanding of machine learning concepts in project work.Contribute to projects requiring some Ruby coding support.Requirements6 years of experience in data engineeringStrong proficiency in Python.Hands-on experience in data engineering including designing building and optimizing data pipelines ETL processes and workflows.Solid software engineering skills (design architecture and best practices).Experience working with Polars (or similar data frame libraries).Knowledge of unit testing and end-to-end testing frameworks.Hands-on experience with Docker for testing purposes.Exposure to GCP services.Basic understanding of machine learning concepts.Good to have some experience with Ruby. Key Skills Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala Employment Type : Full Time Experience: years Vacancy: 1