Senior LLM Systems Engineer

Responsibilities

• Own and improve LLM-driven components of the oracle automation stack
• Improve LLM accuracy through better prompts, model selection, and retrieval
• Optimize system performance regarding latency, token usage, and cost
• Design resilience mechanisms including validation, retries, and fallbacks
• Build evaluation datasets, regression tests, and observability dashboards
• Improve agent orchestration and tool use across internal services and APIs
• Debug live production issues and improve operational runbooks

Requirements

• 3+ years of professional software engineering experience in Python or TypeScript
• Hands‑on experience building production systems using LLMs, agents, or RAG
• Experience designing evaluations, test datasets, or quality metrics for AI systems
• Strong debugging skills across APIs, databases, queues, and logs
• Practical understanding of pr...