Senior Data Engineer

Overview

We are looking for a highly skilled Senior Data Engineer with a minimum of 12+ years of overall experience to join our dynamic team. The ideal candidate will have extensive expertise in building and managing scalable data pipelines, primarily using Scala or PySpark, along with strong experience in Apache Airflow. This is an excellent opportunity to contribute to advanced data platform initiatives within the banking domain, leveraging big data technologies and agile methodologies.

Key Responsibilities

• Architect, develop, and optimize scalable and efficient data pipelines using Scala and/or PySpark.
• Lead the design and management of workflows using Apache Airflow to ensure reliable and timely data processing.
• Work extensively with Hadoop/Cloudera platforms, managing Parquet file formats and big data infrastructure.
• Drive agile processes including sprint planning, story point estimation, and task prioritization...