AWS Databricks Devops Engineer

Title: Devops Engineer Databricks+AWS

Duration: Long term

Location: Remote /Canada

Job Description:

About the role

Client is migrating 100+ Databricks pipelines (batch and streaming) from the AWS West environment into AWS East Databricks workspace. Four Data Engineers are handling pipeline migration. This role owns the infrastructure, CI/CD, and deployment layer that underpins all of them.

What you'll own

• Design and maintain Terraform modules using the Databricks Terraform provider to provision and manage jobs, clusters, cluster policies, permissions, secret scopes, and instance profiles must-have
• Build and maintain GitHub Actions CI/CD workflows for automated Databricks job deployment across dev, staging, and production environments must-have
• Configure and manage Databricks Asset Bundles (DABs) as an IaC deployment mechanism alongside Terraf...