Data Engineer

Location: PAN India Experience : 6 - 8 Years Must have Skills: Implementing data ingestion pipelines from different types of data sources i.e Databases, S3, Files etc Experience in building ETL/ Data Warehouse transformation process. Experience working with structured and unstructured data. Developing Big Data and non-Big Data cloud-based enterprise solutions in PySpark and SparkSQL and related frameworks/libraries, Developing scalable and re-usable, self-service frameworks for data ingestion and processing, Integrating end to end data pipelines to take data from data source to target data repositories ensuring the quality and consistency of data, Processing performance analysis and optimization, Bringing best practices in following areas: Design & Analysis, Automation (Pipelining, IaC), Testing, Monitoring, Documentation. Good to have (Knowledge): Experience in cloud-based solutions, Knowledge of data management principles.