Frontier models still fail on real clinical work, and most of those failures never show up in public benchmarks. You'll build the benchmarks and evaluation systems that show where models break on real-world healthcare data, then turn those findings into the data that closes the gap. We call the loop measure, build, re-test.

Design and maintain clinical benchmarks for reasoning, longitudinal patient histories, multimodal records and tool use.

Build evaluation pipelines end to end: runs, scoring, dashboards and reports that compare models and training runs at scale.

Create rubrics, automated graders and LLM-as-a-judge systems, and calibrate them against clinician review.

Run systematic failure analysis: categorize failure modes, measure how often they occur and feed the results into dataset design and post-training.

Work with clinical experts and partner AI teams to turn benchmark results into research papers and products.

Member of Technical Staff, Evaluation

Responsibilities

Qualification

Required

Preferred

Benefits

Company

Genmd

More roles at Genmd