Build and scale evaluation pipelines that compare model versions on large driving datasets: offline evals, slice-based metrics, regression detection, and model-vs-model reports.

Own embedding and vector-search pipelines over driving data, versioned per model, so engineers can search, cluster and compare how any model represents the world. This is where you’d start: owning embedding generation and search end to end.

Build model introspection infrastructure (attention, attribution, ablations) that turns “why did the model do that?” from a research project into a query.

Help define how frontier LLMs, VLMs and agents change ML analysis here: triaging, labeling, clustering and summarizing driving scenarios and eval results. We already have a prototype using LLMs for data curation, and you’d help decide where it goes next.

Work side by side with the ML engineers training these models to decide what to measure, then make those measurements trustworthy, observable and cheap enough to run on every candidate model.

Build the web app and CLI that engineers and AI agents use to find datasets, training runs, evals and their costs, designed with agents as first-class users.

Senior Software Engineer, ML Evaluation & Introspection Infrastructure

Responsibilities

Qualification

Required

Benefits

Company

Aurora Operations Inc

More roles at Aurora Operations Inc