Lead Azure Databricks Developer

Position Description:

We are seeking a skilled Azure Databricks Developer to design, develop, and optimize big data pipelines using Databricks on Azure. The ideal candidate will have strong expertise in PySpark, Azure Data Lake, and data engineering best practices in a cloud environment.

Your future duties and responsibilities:

> Review and analyze source system mapping documents, ETL workflows, and data flow diagrams.
> Understand business logic, transformation rules, and data dependencies in existing systems.
> Evaluate complexity and design patterns of legacy ETL processes (e.g., SSIS).
> Recommend and design equivalent or improved solutions in Azure Databricks using PySpark and Delta Lake.
> Provide data modeling recommendations for the target architecture, ensuring scalability and performance.
> Collaborate with data architects, business analysts, and migration teams to define migration strategy and executio...