Software Development Engineer, AI Upgrades & Transformation

Do you enjoy working on large-scale distributed systems and Agentic AI ?

The team manages the Apache Spark Upgrade Agent for Analytics services (Glue, EMR, Athena), the Remote MCP Service (SageMaker Unified Studio MCP), and the AWS Glue Python Service - Product and Runtime.

Apache Spark Upgrade Agent[1], is a new generative Agentic AI capability that enables customers to upgrade their Apache Spark applications to newer versions. Spark upgrades traditionally require 6-12 months of engineering effort for customers due to analyzing API changes, resolving dependency conflicts, and validating functional correctness across hundreds of enterprise applications. Apache Spark Upgrade Agent, reduces upgrade timelines from months to weeks through conversational interfaces, automated code transformation, and rigorous data quality validation, allowing engineers to use natural language for upgrades while maintaining approval control over changes and freeing engineering capacity for...