Software Engineer, Agent Optimization & Evaluation

The Role
As a Software Engineer, Agent Optimization & Evaluation, you will lead our agent optimization and evaluation platform. You'll build systems to measure and improve agents' impact on engineering velocity, including Reddit-specific evaluation environments, intelligent model orchestration, and feedback loops that learn from production usage. This hands-on role involves writing production code and building new systems with ownership and autonomy. Strong platform and infrastructure partners will support your work on agent optimization, evaluation, and orchestration.

What You’ll Do:
Design and build Reddit's agent optimization harness for evaluating, benchmarking, and improving AI agents across real software engineering workflows.
Develop Reddit-specific evaluation environments that accurately measure agent performance against meaningful engineering tasks.
Build systems for offline replay, automated evaluation, synthetic task generation, regression detection, and human-in-the-loop feedback.
Design model routing and orchestration strategies that optimize for quality, latency, and cost.
Define metrics and experimentation frameworks that measure developer productivity and engineering velocity.
Build tooling to analyze agent behavior, investigate failures, and accelerate iteration.
Partner closely with Platform Engineering, Infrastructure, and Developer Productivity teams to integrate agent capabilities into engineering workflows.
Help shape Reddit's long-term strategy for a flexible, model-agnostic agent platform that gives us control over our AI capabilities and economics

You Will Bring:
5-12+ years of engineering experience
Extensive experience building and shipping large-scale production software systems.
Experience designing evaluation, experimentation, benchmarking, or measurement systems for AI, search, developer platforms, or other complex distributed systems.
Strong understanding of modern LLMs, AI agents, and the engineering systems required to deploy them effectively.
Experience building systems that optimize across multiple dimensions, including quality, latency, reliability, and cost.
Excellent software engineering fundamentals and a passion for building production-quality systems.
Ability to operate independently in ambiguous environments and drive technical direction for highly strategic initiatives.
Strong cross-functional collaboration skills and experience working across engineering, infrastructure, and platform organizations.