Cursor Software Engineer, RL Data at SpaceXAI focuses on shaping training data for reinforcement learning agents to improve model performance. The role involves building environments, designing graders, and scaling data production systems.
Responsibilities
The data behind a model shapes it more than almost anything else. On the RL Data team at SpaceXAI, you decide what that data is for our agents: which problems they practice, what world they practice in, and how they're judged. You'll take a weakness, like an agent that gives up on hard bugs or misuses a calendar API, and turn it into training data that fixes it, then show the next model is measurably better.
You form hypotheses about why data is helping or misleading the model, build the systems to test them, run ablations, and read the rollouts to find the bug, shortcut or mismatch behind a result. Every flaw you catch, or miss, shows up in what the next Grok models learn.
Turning a capability gap into thousands of verifiable tasks — for example, planting subtle bugs deep in a real project's dependencies for the agent to root-cause.
Building environments that behave like production: real codebases with their real dependencies, and simulations of apps like email, calendar, and docs that match the real ones down to search, errors, and pagination.
Designing graders for open-ended work that separate good solutions from bad without penalizing valid alternatives or rewarding shortcuts.
Building automated checks that catch reward hacking, leaked answers, broken environments, and paths the real product wouldn't allow, before data reaches training.
Scaling from a thousand hand-checked tasks to millions, with filters and judges you can trust in place of a person reading each one
You may be a fit if
Qualification
You enjoy turning messyIf there appears to be a fitReinforcement learning experience
Required
You are careful and systematic — understanding both aggregate metrics and close reading of raw outputs.
You enjoy turning messy, open-ended problems into repeatable, high-quality data production systems.
RL or ML experience is a plus but not required. We welcome strong engineers who want to do research on what makes models better.
If there appears to be a fit, we'll reach out to schedule 2-3 short technicals. After, we'll schedule an onsite in our office, where you'll work on a small project, discuss ideas, and meet the team.
