Data Engineer at Duckbill

About SkywayWe are developing a SaaS product that simplifies financial planning and analysis of cloud billing data for large enterprises with complex cloud spending requirements. We're looking for a data engineer to wrangle complex cloud billing data by designing the pipelines that power our product.We have fascinating technical challenges around data modeling and continuous quality control. We're analyzing massive amounts of semistructured data at scale: processing cloud bills with constantly evolving schemas—complexity that only increases as we expand functionality and provider support. On the frontend, customers use the data to drive large financial decisions, so full data product ownership and quality is key.What You'll DoBuild and maintain ETL pipelines processing hundreds of millions of rows of cloud billing dataWork with ClickHouse, Parquet files, and S3 to design efficient data storage and retrieval systemsDevelop data validation and quality control systems using Python and SQLDesign data models for complex, evolving cloud billing schemas (AWS CUR and beyond)Build and optimize Airflow workflows for reliable data processingCollaborate with the entire engineering team to investigate and resolve data quality issuesScale data infrastructure as we expand to new cloud providers and use casesOur Tech StackAWSPython + FlaskReact + TypeScriptPostgreSQLClickHouseAirflowAbout You3+ years experience with data products: warehouses/lakehouses/OLAPs, ETL pipelines, or job queues2+ years software engineering experience, with significant Python experienceStrong SQL skills including CTEs, window functions, and query optimizationExperience with data validation and quality control systemsComfortable with columnar databases, Parquet, and cloud storage (S3)Ability to deliver results in hours instead of daysSome experience in a startup environment, or ability to work well in a startup environmentFastidiousness about data quality and comfort when there's no answer keyNice-to-haveExperience with ClickHouse or other OLAP datastoresPast experience with cost management tools and/or cloud billing dataExperience with Airflow or similar workflow orchestration toolsBackend engineering experience beyond data pipelinesWhy This Role is ExcitingYou'll tackle genuinely complex technical challenges at scale while building expertise in cloud financial data—a rapidly growing and specialized field. You'll have end-to-end ownership of data products that directly impact customer success, working in a small team where your contributions immediately matter.About UsWe are a small and growing team (less than 10 people!), which means you get the opportunity to be on the ground floor of building the product and company. Our founders are the founders of The Duckbill Group, who bring their wealth of domain expertise and deep industry and customer connections in cloud cost management to the product. We're currently in a semi-stealth mode while we're focusing on building the initial product.This is an in-office roleWe work together in the office in San Francisco three days per week, so you must be located in the SF Bay Area and willing to work in the office on a regular basis.