Overview

Position: Solution Architect – Data Engineering, Governance & AI

Experience: 7+ Years

Employment Type: Contract

Work Location: Remote (India)

Working Hours: 12:00 PM – 12:00 Midnight IST (Any 8–9 Hours within the shift)

Job Summary: We are looking for an experienced Solution Architect – Data & AI to lead the design, governance, security, and implementation of modern enterprise Data Engineering and AI platforms. The ideal candidate should possess strong expertise in Databricks, AWS, Data Governance, dbt, PySpark, Unity Catalog, and Generative AI architectures, with hands-on experience building secure, scalable, and production-grade data platforms. The role involves architecting enterprise data solutions, implementing governance frameworks, enabling AI-powered analytics, and driving secure adoption of Generative AI and Agentic AI solutions across the organization.

Key Responsibilities

• Data Engineering & Governance
  • Design and implement enterprise-scale data engineering architecture using Databricks Lakehouse.
  • Define and enforce enterprise-wide data governance standards, policies, and best practices.
  • Establish metadata management, data lineage, lifecycle management, and data classification using Unity Catalog.
  • Implement robust change management and version control processes across data assets.
  • Drive governance adoption across enterprise and self-service analytics platforms.
• Data Security
  • Design secure data access frameworks using RBAC and ABAC.
  • Implement dynamic data masking and privacy controls.
  • Ensure compliance with enterprise security and regulatory requirements.
  • Build secure AI access boundaries for LLMs, AI agents, and enterprise applications.
  • Perform security audits and governance reviews across the data platform.
• Data Engineering & Modern Architecture
  • Architect scalable data pipelines using PySpark, Spark SQL, Delta Lake, and Delta Live Tables.
  • Develop enterprise transformation models using dbt (dbt Labs).
  • Design Medallion Architecture and Lakehouse implementations.
  • Optimize data pipelines for performance, scalability, and cost efficiency.
  • Build production-ready Databricks applications and data services.
• AI & Generative AI Solutions
  • Design and deploy enterprise Generative AI solutions.
  • Develop multi-agent architectures using Supervisor Agents and Databricks Genie.
  • Integrate Model Context Protocol (MCP) with enterprise AI applications.
  • Build AI-powered self-service analytics and natural language query solutions.
  • Integrate AWS Bedrock and modern AI services into enterprise workflows.
• AWS & Cloud Architecture
  • Design secure cloud-native data platforms on AWS.
  • Utilize AWS services including:
    • IAM
    • S3
    • Lambda
    • Glue
    • EMR
    • Redshift
    • ECS/EKS
    • CloudWatch
  • Architect scalable hybrid cloud solutions integrating Databricks and AWS.
• DevOps & Platform Engineering
  • Implement CI/CD pipelines and GitOps practices.
  • Automate deployment of data engineering and AI workloads.
  • Monitor infrastructure performance and optimize resource utilization.
  • Maintain production-grade reliability, scalability, and observability.

Required Skills

• Data Engineering
• Databricks Lakehouse Platform
• Delta Lake
• Delta Live Tables (DLT)
• Unity Catalog
• Medallion Architecture
• SQL Warehouse
• PySpark
• Spark SQL
• Python
• SQL
• dbt (dbt Labs)
• AI & Machine Learning
• Generative AI
• Agentic AI
• Databricks Genie
• Supervisor Agents
• Model Context Protocol (MCP)
• AWS Bedrock
• AI/BI Solutions
• Cloud
• AWS
• IAM
• S3
• Glue
• Lambda
• EMR
• Redshift
• ECS
• EKS
• CloudWatch
• Governance & Security
• Data Governance
• Metadata Management
• Data Lineage
• Data Classification
• RBAC
• ABAC
• Dynamic Data Masking
• Compliance Frameworks
• Data Privacy
• Security Auditing
• DevOps
• Git
• GitOps
• CI/CD Pipelines
• Infrastructure Automation

Preferred Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field.
• Databricks Certified Data Engineer Professional.
• Databricks Machine Learning Professional.
• AWS Certified Data Engineer.
• AWS Solutions Architect Certification.
• Data Security or Cloud Security certifications are an added advantage.

Preferred Experience

• 7+ years of overall IT experience.
• 5+ years in Data Engineering.
• 2+ years in Data Governance.
• 3+ years in Data Security and AI Architecture.
• Experience working in regulated industries such as Banking, Financial Services, Healthcare, Insurance, or FinTech.
• Strong experience in enterprise-scale cloud migration and modernization projects.

Soft Skills

• Excellent communication and stakeholder management skills.
• Strong solution architecture and technical documentation abilities.
• Ability to collaborate with cross-functional teams.
• Leadership experience in enterprise data transformation initiatives.
• Strong analytical and problem-solving skills.