Momentic AI Engineer at Momentic focused on fine-tuning models and building evaluations to improve AI agents behind the product. Own technical projects from problem definition through delivery working closely with the engineering team.
Responsibilities
Fine-tune models for long-running agents, including agents that investigate whether test failures come from infrastructure, recent code changes, or test authorship.
Build and curate training and evaluation datasets from production workflows, using failure patterns to identify opportunities for improvement.
Develop LLM evaluations and observability that connect offline results with production behavior and help the team verify improvements.
Tune prompts, context, and model behavior to improve task success, reliability, latency, token usage, and cost.
Design repeatable experiments and help publish benchmarks that demonstrate our core agents’ performance on testing and browser workflows.
Own technical projects from problem definition through delivery, working with the engineering team to make practical tradeoffs and ship improvements.
You’re perfect for this role if...
Qualification
You use AI agents to build softwareExperience with long-running agentsA team of 17 and growing
Required
You’ve worked with prompt and context tuning, evaluation datasets, and LLM observability to diagnose failures and measure improvements.
You use AI agents to build software, breaking down work, tracking progress, checking results, and redirecting agents when they get stuck.
You’re comfortable with autonomy and ambiguity. You can turn an unclear problem into a plan, manage your own work, and follow through to delivery.
We’re interested in the depth of your contributions and what you’ve shipped. Your experience may come from owning an AI capability within a larger product.
Preferred
Experience with reinforcement learning fine-tuning, RL environments, or LLM-as-judge evaluators.
Experience with long-running agents, recovery from failed steps, or coordinating multiple agents.
Experience publishing benchmarks or running rigorous experiments on AI systems.
Raised a $15M Series A led by Standard Capital, with backing from Y Combinator, FCVC, Transpose Platform, and Dropbox Ventures.
Our customer base has grown 4× year to date.
A team of 17 and growing.
Based in person in San Francisco at 650 5th Street.
