Senior/Lead Data Engineer

This is a position we're hiring on behalf of our client in the digital banking industry.

Description

In this role, you can expect to:

• Design, develop, test, deploy and monitor data pipelines in Databricks on AWS from a wide variety of data sources.
• Design, develop, test, deploy and monitor scalable code with PySpark and SQL in Databricks.
• Identify opportunities to improve internal process through code optimisation and automation.
• Build data quality dashboards, lineage flows / and or monitoring tools to utilize the data pipeline, providing active monitoring and actionable insight into overall data quality and data governance.
• Assist in migrating data from legacy systems onto newly developed solutions.
• Follow and lead best practices on all data security, retention, and privacy policies.

Requirements

• Bachelor’s degree.
• 6+ ...