As a Research Scientist at idler, you'll own measuring and improving how models learn from our tasks. The job is to maximize learning signal we produce per unit time. You'll draw on your own experience and collaborate with researchers at the frontier to validate our data quality, identify where improvements are needed, and create new datasets. To succeed, you'll need to have extensive experience doing this work in production at a frontier lab.

Examples of what you’ll do

Work with our customers — researchers at frontier labs — to design novel post-training recipes and data quality measurement techniques

Design and run our in-house post-training stack to measure model lift on our tasks

Develop new data products based on datasets and experts available to us

Identify opportunities to take advantage of self-reinforcing exponential feedback loops

Create agents to analyze thousands of environments and millions of trajectories

Help curate and specify task distributions for new corpora

Work with procurement to ensure external data we acquire is suitable for refinement

Create scaleable systems for ingesting & evaluating data we are considering buying

Develop new techniques for mining data for signal

Research Scientist

Responsibilities

Qualification

Required

Benefits

Company

Idler

More roles at Idler