Azure Databricks Architect at Maiden Technologies Inc

Position - Azure Databricks Architect Location: Houston TX- Hybrid Level: 5 Exp: 10 Years Core Responsibilities Design and implement scalable data platforms and pipelines using Azure Databricks Apache Spark Pyspark Delta Lake and MLflow. Design and implement data pipelines using Databricks Spark and Delta Lake for batch and streaming data. Optimize Spark jobs for performance scalability and cost efficiency. Develop and maintain Lakehouse architecture leveraging Databricks and cloud storage solutions. Lead the migration from legacy platforms to Lakehouse architecture. Develop batch and streaming data pipelines for ingestion transformation and analytics. Establish standards for data governance quality and security. Collaborate with stakeholders to align architecture with business goals. Mentor data engineers and developers on Databricks best practices. Integrate Databricks with tools like Power BI Tableau Kafka Snowflake and Azure Data Factory. Required Skills & Experience 8 10 years in data engineering or architecture roles. 8 years of hands-on experience with Databricks and Azure. Strong command of SQL Python Scala and Spark. Experience with CI/CD pipelines DevOps and orchestration tools like Airflow or Data Factory. Familiarity with Azure cloud platforms. Deep understanding of distributed computing performance tuning and data security. Preferred Qualifications Bachelors or Masters in Computer Science Data Science or related field. Certifications such as Azure Solution Architect Azure Data Engineer or Databricks Certified Data Engineer or Solutions Architect. Experience with data mesh data fabric or enterprise data architectures. Domain knowledge in finance healthcare or public sector is a plus. Hadoop & Scala experience Additional Insights from Internal Communications Emphasis on DataOps data orchestration and data modeling. Strong ERP knowledge (e.g. SAP Salesforce) is valued. Candidates should be hands-on capable of self-exploration and able to drive analytics adoption across business teams. A structured questionnaire is often used to assess candidates on areas like Unity Catalog Spark optimization partitioning techniques and data security. Key Skills APIs,Pegasystems,Spring,SOAP,.NET,Hybris,Solution Architecture,Service-Oriented Architecture,Adobe Experience Manager,J2EE,Java,Oracle Employment Type : Full Time Experience: years Vacancy: 1