Research Scientist – Frontier Evaluations

San Franciscofull time

Apply on Employer Site

On Site

$210k–$450k

H1B: no

Afterquery The Research Scientist at AfterQuery designs and publishes rigorous evaluations for frontier AI systems. The role includes leading benchmark development, collaborating with domain experts, and communicating research findings.

Responsibilities

Lead the end-to-end design, validation, launch, and continuous improvement of frontier AI benchmarks.
Partner with researchers and domain experts to develop evaluations around meaningful model failures, gaps in existing coverage, and high-priority domains.
Analyze model capabilities and failure modes using rigorous experimental design and statistical methods.
Build reproducible evaluation systems, including harnesses, graders, and benchmark infrastructure.
Collaborate with researchers to post-train models and measure the resulting performance gains.
Communicate results through benchmark reports, technical articles, and research papers.

Qualification

Commitment to experimental rigorDepth in agenticPhD in a related technical fieldHealth Insurance401(k) with Employer MatchDaily MealsMonthly Wellness Stipend

Required

Strong record of publishing benchmarks or research papers.
Clear technical communication and strong scientific writing skills.
Commitment to experimental rigor, including baselines, ablations, statistical validity, and contamination controls.
Ability to take an ambiguous evaluation question from initial scoping through a reproducible public release.
Depth in agentic, coding, and safety evaluations or applied machine learning in an expert domain.

Preferred

PhD in a related technical field.
Research publications at leading conferences or peer-reviewed journals.
Interest in multidisciplinary research and the creativity to combine methods and insights from AI, engineering, science, and other expert domains.
Company Benefits (For Eligible Employees):
Health Insurance: Medical, Vision, Dental
401(k) with Employer Match
Daily Meals: Daily UberEats Stipend
Monthly Wellness Stipend

Benefits

Medical
Dental
Vision
401(k)
Equity/RSU
Wellness

Company

Afterquery

View company page

Source: ashby

Active within the past hour

More roles at Afterquery

Senior Software Engineer - Security/Infrastructure

San Francisco

Head of Enterprise GTM

San Francisco

Strategic Projects Lead - Engineering (Life Sciences & Chemical)

San Francisco

Strategic Projects Lead - STEM

San Francisco