Staff Software Engineer, RL Environments

San Francisco, CAfull timeRemote

Apply on Employer Site

senior

$252k–$315k

H1B: yes

Scale AI Staff Software Engineer, RL Environments at Scale AI owns the technical foundation for building, running, verifying, and delivering reinforcement learning environments at scale. The role involves designing platforms for sandboxed execution, environment packaging, rollout orchestration, and building graders that withstand adversarial optimization.

Responsibilities

As a Staff Software Engineer, RL Environments, you'll own the technical foundation for how Scale builds, runs, verifies, and delivers RL environments at scale.
An RL environment is a real piece of software: a containerized world with real dependencies, real state, real tools, and a grader that has to be correct even when the agent is creative about breaking it. Building one is a full-stack engineering problem. Building thousands of them reproducibly, cheaply, with trustworthy reward signals and throughput measured in millions of rollouts is a systems problem that very few people have solved.
You'll work on both. You'll design the platform: sandboxed execution, environment packaging and versioning, rollout orchestration, trajectory capture, verifier frameworks, and the authoring surfaces that let engineers and domain experts produce environments without reinventing infrastructure each time. And you'll go deep on the environments themselves by instrumenting real applications, designing task suites that expose specific capability gaps, and building graders that hold up under adversarial optimization.
This is a hands-on engineering role. You'll set technical direction across multiple teams, and you'll still be the person who writes the hard part.

Qualification

Demonstrated ability to own ambiguousRL & Post-TrainingFamiliarity with post-training methodsSystems & Infrastructure

Required

8+ years of software engineering experience with strong fundamentals in distributed systems, system design, data structures, and algorithms.
Strong Python skills and a track record of shipping production software; comfort in at least one other part of the stack (TypeScript/React, Go, Rust, or similar).
Deep experience with containerization and sandboxed execution, including Docker, VMs, gVisor/Firecracker, Kubernetes, or equivalent.
Experience building or operating high-throughput backend systems: orchestration, job scheduling, queuing, and large-scale data pipelines.
Hands-on experience building with LLMs including agent loops, tool calling, MCP, or eval harnesses, and enough intuition about model behavior to reason about what a training signal actually teaches.
Demonstrated ability to own ambiguous, undefined problems end to end and drive them to a shipped system.
Excellent written and verbal communication; ability to align engineers, researchers, and non-engineering partners on a technical direction.

Preferred

RL & Post-Training
Direct experience building RL environments, agentic benchmarks, or eval harnesses (SWE-bench-style task suites, terminal or browser environments, tool-use benchmarks, or in-house equivalents).
Familiarity with post-training methods: RLHF, RLAIF, RLVR, GRPO/PPO-family algorithms, rejection sampling, reward modeling, and the practical failure modes of each.
Experience designing verifiable reward signals, and firsthand experience with reward hacking and how to defend against it.
Experience with RL training or serving stacks (verl, TRL, Ray, vLLM, SGLang, or similar).
Systems & Infrastructure
Experience with high-scale sandbox or code-execution infrastructure, remote development environments, or CI systems.
Experience with cloud-native infrastructure across AWS/GCP/Azure, Infrastructure as Code, and CI/CD.

Benefits

Dental
Vision
PTO
Equity/RSU

Company

Scale AI

View company page

Source: greenhouse

Active within the past hour

Remote-friendly

More roles at Scale AI

Senior Data Analyst, Public Sector

Washington, DC

Tech Lead Manager- MLRE, ML Systems

San Francisco, CA

SWE Fellow - Human Frontier Collective (Canada)

Remote (Growth only)

Subject Matter Expert

Remote (Growth only)

Boost Your Interview Chances

Improve Resume Match Score

Free

Your Score

9.0

Strong Applicant

Must-Have Skills for This Role

PythonTypeScriptReactGoRustDocker

Optimize my Resume

Apply Faster with Autofill

Free

Save your profile once. LumiJobs pre-fills the employer's application form so you stop re-typing the same answers on every site.

Apply With Autofill

Or skip LumiJobs's tools and submit directly on the employer's career site.

Apply on Employer Site