Data Engineer at Perpay, Inc.

Perpay, Inc. seeks a Data Engineer in Philadelphia, PA. The Data Engineer will analyze, resolve, and optimize data-related challenges, implement solutions to unify application data from various sources, and facilitate automated data solutions across business domains for enhanced scalability. Additional specific job duties will include: Develop architecture and model important performance metrics and organize data models within a high-capacity analytical data lake house using SQL, Python, and Spark; Analyze the execution plan high latency and/or long run time data models and deploy optimized and efficient improvements by incorporating best practices such as CTEs, materialized views with incremental refresh rates, and data compression; Optimize analytical workflows by identifying opportunities and conducting automation; Construct entire data pipelines to support all business units by utilizing an Airflow orchestrator hosted in AWS EC2 and incorporate parallel processing methods to facilitate rapid reporting; Implement solutions to bring together application data generated by distributed first and third-party systems driving our centralized data lake; Maintain Perpay’s complex cloud-based data infrastructure using Terraform and build new infrastructure POCs to explore viable updates and upgrades in a staging environment; Contribute to a modern data architecture to ensure scalability, reliability, and data quality; Integrate with a variety of third part data sources via in house constructed Python API solutions as well as partnered ETL and Reverse-ETL solutions, such as Fivetran and Census; Utilize CI/CD tools, such as Circle CI, Travis, and Github Actions, to automate standard processes and test suites to reduce the labor time of manual procedures create a more robust data environment; Integrate with the data team’s cloud data cataloging and lineage service as well as Perpay’s BI tool, DOMO, to create and distribute the context and documentation for stakeholder use; Become the subject matter expert for any data focused question at Perpay and regularly update and present to the Perpay leadership and influence the data team’s technical journey; Work on projects of growing responsibility, both individually and as part of a team, to build experience and skills at a pace matched to shown ability; and cultivate a comprehensive understanding of both the industry and company operations, laying the groundwork essential for fostering long-term career advancement. The Data Engineer must have at a minimum a Master's degree in Data Science, Information Science, or Health Data Science. Email resumes to [email protected].