Data Platform Engineer (102-08SENG-01)
At Cloudary SAS · -, Acre, Brazil
- Data and ML
- Contract
See how well you match Match me reads your CV and picks the jobs that suit you best.
Get jobs like this within hours Radar emails you new jobs open to you within hours of us finding them, before most people see them.
Where you can work from
Open to Brazil
“Open to candidates from all LATAM”
From the job description.
“-, Acre, Brazil”
From the location.
Report a mistake
- Posted
- 20 August 2026
- First seen by us
- 3 November 2026
- Last checked
- 15 hours ago
Job description
From Cloudary SAS’s job post, as they wrote it.
This role converts a large Azure Data Factory estate into Databricks workflows on AWS. The scope for year one: 2,094 ADF pipelines to migrate — built as reusable templates rather than one-by-one — 9,859 pipeline activities to translate (some map directly, others need rewriting as Lambda or Step Functions), 471 Spark dataflows to move onto Databricks on AWS, and 4 Databricks workspaces (Dev, QA, Pre-prod, Prod) to rehost, including notebook paths and Unity Catalog rewiring. This is a regulated environment, so reconciling migrated data against source systems is part of the definition of done, not an afterthought.
Read the full description
What you will do
* Convert Azure Data Factory pipelines into Databricks workflows on AWS, building reusable templates rather than migrating one at a time.
* Rehost Databricks workspaces onto AWS and migrate ADLS Gen2 storage to S3.
* Rewrite ADF Web Activities as Lambda functions or Step Functions tasks, and replace ADF-specific scaling with native Databricks mechanisms.
* Build and tune PySpark transformations for production data volumes.
* Replace Azure Synapse Serverless with Databricks SQL Warehouse.
* Reconcile migrated data against source systems as part of the definition of done.
Required
* Production experience with Databricks: workspaces, jobs and workflows. The central skill for this role.
* Strong Spark and PySpark experience for real data volumes, including tuning.
* Production-grade Python.
* Experience building or migrating Azure Data Factory pipelines, with a solid understanding of the ADF activity model.
* AWS data services: S3, Glue, Athena, Lambda and Step Functions.
* Advanced SQL, including reading and reasoning about stored procedures.
* Professional written and spoken English.
Nice to have
Delta Lake, Unity Catalog, Azure Synapse, Terraform, Airflow/MWAA, dbt, Kafka, Databricks certification, data modeling, Great Expectations, SAS/analytics platform integration, CRM data.
Engagement details
* Estimated start date: December 2026
* Duration of project: 9 months
* EST Time Zone
* Open to candidates from all LATAM
Data Platform Engineer (102-08SENG-01)Cloudary SAS
Apply on Cloudary SAS (opens in a new tab)Get jobs like this by email
Once a week, we email the new remote jobs open where you live, each with the line that proves it. Every email has a one-click unsubscribe.
