Applied AI Engineer, Evals

San Franciscofull timeOn SiteH1B: no

Sable Applied AI Engineer at Sable responsible for designing evaluations and improving AI agents using voice, vision, and browser use data. The role involves building benchmarks and scoring systems to optimize agent performance.

Responsibilities

You translate what makes a good performance into clear, defensible signals, and use those signals to improve Aidan. Role involves carrying out evaluations of agents on a broad range of areas on the frontier including voice, vision, and browser use, and using the findings to optimize the agent, context, and models.
Design evaluations for voice, vision, and browser use to direct and accelerate improvements in the agent, context, and models
Build and maintain our internal benchmarks and simulations
Work on systems to score customer-specific scenarios and identify gaps in current capabilities.

Qualification

Comfortable in a production codebaseBonusPrior experience with evaluationsExperience with voice

Required

Engineer with statistics and/or applied machine learning experience
Comfortable in a production codebase
Bonus: prior experience with evaluations or with voice, realtime, or browser use/computer use agents.