Software Engineer, Applied AI- Evals at Mercor · LumiJobs

Responsibilities

The Applied AI org builds the systems that turn human expertise into training data for frontier models, task pipelines, expert workflows, evaluation infrastructure, and the services that tie them together. We have synthetic pipelines and modular quality control systems that run in unison to generate highest quality tasks and at scale. All of it runs on backend systems that have to stay correct, fast, and observable while the volume behind them grows every month.
As a Software Engineer on Backend Systems, you'll own services in that stack: designing the data models, building the APIs, services, data pipelines that move work through the platform. This is a build role. You'll take a problem that's roughly scoped, make the design calls, ship it to production, and own it afterward.
We're hiring for depth in backend fundamentals rather than any particular domain. If you've built and operated real services, handled the schema migration that couldn't take downtime, found the query that fell over at 10x traffic, designed the retry logic that made a flaky dependency invisible to users, or build a system that recover when things break that's the experience that matters here. You'll work with a talent dense group of engineers who will review your designs and push your thinking, and you'll be expected to grow into owning larger surfaces quickly.
Design, build, and operate backend services in the Applied AI stack including APIs, data models, background jobs, and the pipelines that connect them.
Own features end to end: scope the problem, write the design, ship the code, instrument it, and keep it healthy in production.
Build and tune high-throughput data and job pipelines: queuing, batching, idempotency, retries, and backpressure.
Make the system fast and reliable by adding failure recovery in pipelines, profile hot spots, Agent token and cost attribution, caching issues, and set latency, error, cost budgets you actually hold to.
Instrument what you ship: observability, metrics, logging, tracing, and alerts that proactively catch problems.
Work on modern cutting edge tools, libraries and frameworks.
Manage and launch 10s of 1000s of containers, sandbox environments, manage resource allocation and system health.
Manage infrastructure as code using Terraform.
Participate in on-call for the systems you own, debug production incidents, and write up what you learn in RCCA.

Software Engineer, Applied AI- Evals

Responsibilities

Qualification

Required

Preferred

Benefits

Company

Mercor

More roles at Mercor