Founding Member of Technical Staff (Evals & Post-Training)

New Yorkfull timeOn Site

Apply on Employer Site

senior

$165k–$325k

H1B: no

Ambral Founding Member of Technical Staff at Ambral Labs working on building replayable environment engines and evaluation systems for enterprise AI workflows. Responsible for developing infrastructure and production systems to improve model performance through reinforcement learning and post-training techniques.

Responsibilities

At the center of Ambral Labs is a replayable environment engine for the enterpirse.
The system reconstructs a company’s world as it existed at a particular moment in the past then exposes that state through the same tools an agent would use in production. This lets us place new policies and agent configurations inside real historical environments, observe how they reason and act, and grade their performance against real outcomes.
You'll work across research, infrastructure, and production systems including:
Building an environment factory that converts recorded enterprise data and task definitions into runnable environments
Designing graders that turn ambiguous business objectives into verifiable rewards
Developing methods for mining useful tasks, trajectories, and evaluation cases from historical workflows
Creating eval sets that are representative, reproducible, and resistant to overfitting
Finding the right combinations of models, tools, context, and policies to maximize performance while reducing inference cost
Training and evaluating agents that operate over long horizons, incomplete information, and large tool spaces
Building replay and observability systems that make agent behavior explainable and measurable
Scaling from individual environments to thousands of concurrent training and evaluation runs
These problems are wide open. You’ll have significant ownership over both the research direction and the production systems that make it real.

Qualification

You understand how environment designYou care about reproducibility

Required

1-7 years of experience building production software or machine-learning systems

Preferred

You understand how environment design, reward design, context, tooling, and policy behavior interact
You’re comfortable turning fuzzy business objectives into tasks and signals that can be evaluated reliably
You can diagnose whether a model’s limitations come from the model itself, its context, its tools, its harness, or its training
You can move between research questions and production implementation without treating them as separate jobs
You write strong software and can build systems that process large, messy datasets at scale
You care about reproducibility, observability, and understanding why a model behaves the way it does
You’re looking to do the best work of your life and build something you’ll be proud of for decades
We care much more about what you’ve built and how you think than credentials or conventional career paths.

Benefits

Medical
PTO
Equity/RSU

Company

Ambral

View company page

Source: ashby

Active within the past hour

More roles at Ambral

Head of Research

New York

Member of Technical Staff (New Grad)

New York

Boost Your Interview Chances

Improve Resume Match Score

Free

Your Score

9.0

Strong Applicant

Must-Have Skills for This Role

llm

Optimize my Resume

Apply Faster with Autofill

Free

Save your profile once. LumiJobs pre-fills the employer's application form so you stop re-typing the same answers on every site.

Apply With Autofill

Or skip LumiJobs's tools and submit directly on the employer's career site.

Apply on Employer Site

Founding Member of Technical Staff (Evals & Post-Training) at Ambral · LumiJobs