Sr Data Engineer (Databricks)
At Unison Group · Bengaluru, Karnataka, India
- Data and ML
- Full-time
- Senior
Where you’ll work
Hybrid, based in Bengaluru
“Bengaluru, Karnataka, India”
From the location.
“Integrate with Azure ecosystem including ADLS Gen2, Azure SQL Database, Serverless/Dedicated SQL Pools; orchestrate via Databricks workflows and hybrid ADF triggers for high-volume ACDP workloads.”
From the job description.
Hybrid means some days in the office. The listing says how many, and where.
Report a mistake
- Posted
- 2 October 2026
- First seen by us
- 9 October 2026
- Last checked
- 11 hours ago
Job description
From Unison Group’s job post, as they wrote it.
Position Overview:
* Lead the design, architecture, and optimization of enterprise-scale data pipelines using Azure Databricks for ACDP projects, driving innovation in DATA & AI initiatives.
* Leverage 7-8 years of expertise to mentor teams, ensure scalability, and deliver mission-critical data solutions.
Key Responsibilities:
* Architect and implement end-to-end data pipelines with Azure Databricks (notebooks, jobs, clusters) for ingestion, ETL/ELT, and complex transformations, incorporating Medallion Architecture (Bronze, Silver, Gold) using Delta Lake for ACID compliance, versioning, schema evolution, and data quality.
Read the full description
* Provide hands-on leadership in Databricks Spark (PySpark, Scala, SQL), Delta Live Tables, Structured Streaming, and Unity Catalog for advanced governance, security, and multi-workspace management.
* Integrate with Azure ecosystem including ADLS Gen2, Azure SQL Database, Serverless/Dedicated SQL Pools; orchestrate via Databricks workflows and hybrid ADF triggers for high-volume ACDP workloads.
* Design and deploy CI/CD pipelines using Azure DevOps, Bicep for infrastructure-as code, and advanced automation for Databricks workspaces, emphasizing zero downtime deployments and cost optimization.
* Champion data quality, validation, cleansing with Databricks expectations/constraints, and compliance via RBAC, encryption, and Purview; mentor on best practices for Photon engine, auto-scaling, spot instances, and performance tuning
* Collaborate with senior analysts, data scientists, and executives to translate complex business requirements into reusable Databricks components, fostering AI/ML integration.
* Drive monitoring, troubleshooting, and optimization for petabyte-scale processing, including unstructured data handling, data modeling, and streaming from Kafka/Event Hubs
Requirements
Required Qualifications:
* 7-8+ years in data engineering, with deep Azure Databricks expertise for big data processing in high-demand environments
* Advanced proficiency in Python/PySpark, SQL, Scala; strong data architecture and distributed systems experience
* Proven track record in cloud-native solutions (AZ-900+ preferred), CI/CD, and mentoring junior engineers
* Excellent communication for stakeholder alignment and documentation
Preferred: Power BI, Power Automate, Azure ML, enterprise governance tools.
Get jobs like this by email
Once a week, we email the new remote jobs open where you live, each with the line that proves it. Every email has a one-click unsubscribe.
