Sf Tensor Member of Technical Staff for Sandbox Infrastructure to build a serverless GPU container service across multiple vendors. Responsible for running untrusted kernels safely and efficiently on real silicon at scale.
Responsibilities
We build the fastest GPU compiler in the world. Most compilers have to preserve correctness at every transform, constraining how far they can search, while we prove correctness at the end instead, allowing us to search a far wider space, with agents, with RL, with anything that works and still guarantee the result. It's why we hold #1 on NVIDIA's own kernel benchmark across hundreds of production kernels.
For our search to work, we need to run an enormous amount of untrusted, freshly generated kernels on real silicon, quickly and safely, which is why we're hiring a Member of Technical Staff for Sandbox Infrastructure to build the layer that makes it possible: a serverless GPU container service across NVIDIA, AMD, TPU and Trainium, at a scale and fidelity nobody sells off the shelf. It needs to run three things: (1) our compiler measures every candidate program inside it, so a noisy or unfair sandbox directly corrupts the reward signal the search learns from, (2) our own post-training runs inside it (e.g., when we RL a model on AMD kernel engineering, every rollout is in a sandbox) and (3) customer workloads that require isolation, such as their RL rollouts, run on it too.
We already do this on NVIDIA and AMD, including having rolled AMD GPU support in gVisor from scratch. The work now is depth and breadth, adding support for more vendors, higher fidelity instrumentation, faster cold starts and more features. A lot of our sandboxes need to run on spot pools without losing works, they can span multiple GPUs and sometimes multiple nodes, they need to survive failure and preemption as well as live-migration (which our stack does while keeping sockets intact so that instances can relocate mid-flight while still streaming data in or out without a hiccup).
You'll extend our sandboxing stack to new vendors and accelerators
You'll build and maintain GPU virtualization below the runtime, including gVisor work at the driver and ioctl level
You'll make sandboxes first-class citizens on spot capacity, which means preemption-aware scheduling, checkpointing and rescheduling
You'll support multi-GPU and multi-node sandboxes, including the interconnect (NVLink, NVSwitch) and RDMA paths (InfiniBand, RoCE) paths those require
You'll own live migration end-to-end, including our socket-preserving migration
You'll guarantee measurement and profiling fidelity as well as their isolation
You'll work directly with the compiler, post-training and kernel teams to ensure their throughput is not capped by sandboxes
Qualification
Someone proficient in GoSomeone who's worked directly on CRIUSomeone familiar with NCCL/RCCL
Required
Someone with strong low-level systems engineering background: Linux kernel internals, containers, namespaces, cgroups, syscall interception or hypervisors
Someone with experience in GPU systems engineering: drivers, runtimes or scheduling on accelerator fleets
Someone comfortable with distributed systems failure modes: preemption, partial failure, checkpoint/restore and dealing with states you can't afford to loose
Someone proficient in Go, C/C++ or Rust
Someone with a strong bias toward building the thing yourself when no vendor supports what you need
Preferred
Someone who's worked directly with gVisor, Firecracker, Kata, QEMU/KVM or similar
Someone who's worked directly on CRIU, -live migration or connection-preserving failover work
Someone familiar with NCCL/RCCL, RDMA, InfiniBand or vendor interconnects
Someone who's run large fleets on spot or other preemptible capacity
Someone familiar with bare-metal provisioning, hypervisors or fleet management at scale
Someone with a security background in isolation boundaries and untrusted code execution
