Data Integration Engineer at RSM Solutions, Inc

Thank you for stopping by to take a look at the Data Integration Engineer role I posted here on LinkedIN, I appreciate it.If you have read my job descriptions in the past, you will recognize how I write job descriptions. If you are new, allow me to introduce myself. My name is Tom Welke. I am Partner & VP at RSM Solutions, Inc and I have been recruiting technical talent for more than 23 years and been in the tech space since the 1990s. Due to this, I actually write JD's myself...no AI, no 'bots', just a real live human. I realized a while back that looking for work is about as fun as a root canal with no anesthesia...especially now. So, rather than saying 'must work well with others' and 'team mindset', I do away with that kind of nonsense and just tell it like it is.So, as with every role I work on, social fit is almost as important as technical fit. For this one, technical fit is very very important. But, we also have some social fit characteristics that are important. This is the kind of place that requires people to dive in and learn. The hiring manager for this one is actually a very dear friend of mine. He said something interesting to me not all that long ago. He mentioned, if you aren't spending at least an hour a day learning something new, you really are doing yourself a disservice. This is that classic environment where no one says 'this is not my job'. So that ability to jump in and help is needed for success in this role.This role is being done onsite in Irvine, California. I prefer working with candidates that are already local to the area. If you need to relocate, that is fine, but there are no relocation dollars available.I can only work with US Citizens or Green Card Holders for this role. I cannot work with H1, OPT, EAD, F1, H4, or anyone that is not already a US Citizen or Green Card Holder for this role.For this role, you will be working with a team of about 6 other data centric individuals. That team is a mix of ML Cluster engineers, db engineers and a BA. You won't really be working that much on requirements gathering, as that is something that the BA on the team does. But, if you have worked on requirements gathering, documentation, process flow diagramming and so on, that would be great to see, as partnering with the BA would be a great thing to see in the candidate chosen for this role.You will design, build and operate batch & streaming pipelines that move data from SQL Server, MongoDB, legacy files, and third-party APIs into this client's Data Vault warehouse and machine-learning (ML) cluster, ensuring that data is accurate, timely, and analytics-ready. This role blends hands-on ETL/ELT development in SSIS, Spark, Runbooks, and Azure Data Factory with data-modeling expertise (hubs, links, satellites) to support scalable reporting, predictive models, and AI agents. Working closely with development team and cross-functional product teams.Here are some of those key responsibilities:Design, develop, and deploy incremental and full load pipelines using SSIS, Spark, Runbooks and Azure Data Factory to ingest data into landing, raw, and curated layers of the Data Vault.Build CDC (change data capture) solutions to minimize latency for downstream reporting and ML features.Automate schema evolution and metadata population for hubs, links, and satellites.Implement validation rules, unit tests, and data quality frameworks to enforce referential integrity and conformance to business rules.Maintain a requirements traceability matrix and publish data lineage documentation Metadata Management / SSAS models. This includes partnering with this team's BA to translate user stories into technical interfaces and mapping specs.Create CI/CD pipelines (Azure DevOps, Git) to version ETL code, infrastructure as code, and automated tests.Develop PowerShell/.NET utilities to orchestrate jobs, manage secrets, and push metrics to Grafana or Azure Monitor.Benchmark and tune Spark, SQL, and SSIS performance; recommend index strategies, partitioning, and cluster sizing strategies for cost/performance balance.Stay current with emerging integration patterns (e.g., event driven architectures, Delta Lake) and propose pilots for adoption.Here is what we are seeking in terms of requirements for this role:4+ years building data integration with MS SQL, SSIS, and Spark.At least 2 years of ML Cluster build experience.At least 2 years of experience with Data Vault.Strong T SQL, Python/Scala for Spark, PowerShell/.NET scripting; working knowledge of MongoDB aggregation, SSAS tabular models, and Git CI/CD.Data Vault 2.0 certification a plus.Excellent problem solving, communication, and stakeholder management abilities.