Software Engineer, Inference

Redwood City, CA · Remotefull timeRemote

Apply on Employer Site

$195k–$345k

H1B: yes

Luma AI Software Engineer, Inference at Luma AI responsible for integrating new model architectures into the inference engine and scaling deployments across thousands of machines. Focus on scheduling, fleet management, deployment pipelines, and reliability across clusters and hardware providers.

Responsibilities

Ship new model architectures by integrating them into the inference engine
Collaborate across research, engineering, and infrastructure to optimize model efficiency and deployments
Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows
Automate, test, and maintain inference services for maximum uptime and reliability
Manage and optimize inference workloads across clusters and hardware providers, and scale deployments across thousands of machines
Build scheduling systems that use expensive GPU resources optimally while meeting SLOs
Maintain CI/CD for model checkpoints and SDKs

Qualification

Experience with queuesExperience with LinuxCUDAAbout Luma

Required

Strong Python and system-architecture skills
Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang, TensorRT-LLM, or similar
Experience with queues, scheduling, traffic control, and fleet management at scale
Experience with Linux, Docker, and Kubernetes, and with orchestration, deployment, and scheduling
Familiarity with Redis and S3-compatible storage

Preferred

Modern networking stacks including RDMA (RoCE, InfiniBand, NVLink).
High-performance large-scale ML systems (100+ GPUs).
CUDA, and FFmpeg or multimedia processing.
About Luma: Luma's mission is to build unified general intelligence that can generate, understand, and operate in the physical world. We believe multimodality is critical for intelligence — the next step beyond language models comes from vision. Luma is an equal opportunity employer.

Benefits

Vision

Company

Luma AI

View company page

Source: ashby

Active within the past hour

Remote-friendly

More roles at Luma AI

Editor + Motion Graphics

Los Angeles, CA · Remote

Research Scientist / Engineer – Performance Optimization

Redwood City, CA · Remote

Research Scientist / Engineer – Training Infrastructure

Redwood City, CA · Remote