Overview

We are seeking an experienced Databricks Engineer with 12+ years of experience in designing, developing, and implementing enterprise-scale data engineering solutions. The ideal candidate must possess strong expertise in Databricks, Apache Spark, PySpark, Python, SQL, and cloud platforms (AWS/Azure/GCP), along with hands-on experience delivering multiple end-to-end Databricks implementations.

This role requires a customer-focused professional capable of leading technical discussions, architecting scalable data solutions, and delivering production-grade data platforms in cloud environments.

Mandatory Certification
Databricks Certified Data Engineer Professional (Mandatory)

Key Responsibilities

• Lead end-to-end Databricks implementation projects from solution design through deployment and production support.
• Design, develop, and optimize scalable data pipelines using Databricks, Apache Spark, and PySpark.
• Build cloud-native data engineering solutions leveraging AWS, Azure, or Google Cloud Platform.
• Design and implement modern Lakehouse architectures for enterprise data platforms.
• Translate complex business requirements into scalable technical solutions and reusable design patterns.
• Integrate Databricks solutions with enterprise applications, APIs, and third-party systems.
• Configure, monitor, troubleshoot, and optimize Databricks environments for performance, reliability, and scalability.
• Implement CI/CD pipelines and DevOps best practices for data engineering workflows.
• Collaborate with business stakeholders, architects, and cross-functional teams to deliver high-quality solutions.
• Provide technical leadership, mentor engineering teams, and drive best practices across projects.
• Participate in customer workshops, technical consulting sessions, and architecture discussions.
• Ensure solution quality, security, governance, and operational excellence throughout project delivery.

Required Skills

• Databricks
• Apache Spark
• PySpark
• Python
• SQL
• Data Engineering
• Lakehouse Architecture
• ETL/ELT Pipeline Development
• AWS / Microsoft Azure / Google Cloud Platform (GCP)
• Systems Integration
• Solution Architecture
• CI/CD Pipelines
• Performance Optimization
• MLOps
• Git, DevOps Practices
• Data Modeling & Data Warehousing

Required Experience
12+ years of overall IT experience.
Extensive hands-on experience with Databricks and modern data engineering platforms.
Experience delivering 10+ end-to-end Databricks implementation projects.
Strong expertise in distributed data processing using Apache Spark and PySpark.
Proven experience designing enterprise-scale cloud-based data platforms.
Strong understanding of data integration, governance, and Lakehouse architecture.
Experience implementing CI/CD pipelines and deployment automation.
Excellent troubleshooting and performance tuning skills.
Strong client-facing consulting and stakeholder management experience.

Preferred Skills

• Experience with Delta Lake, Unity Catalog, MLflow, and Databricks Workflows.
• Knowledge of modern data architecture patterns and cloud-native services.
• Experience working in Agile/Scrum environments.
• Additional cloud certifications (AWS, Azure, or GCP) are a plus.

Soft Skills

• Excellent communication and presentation skills.
• Strong analytical and problem-solving abilities.
• Customer-first mindset with excellent consulting skills.
• Ability to manage multiple priorities in a fast-paced environment.
• Leadership, ownership, and mentoring capabilities.
• Ability to communicate effectively with both technical and non-technical stakeholders.