Member of Technical Staff | Inference Platform

São Paulo · Remotefull timeRemote

Apply on Employer Site

senior

H1B: no

Avra Member of Technical Staff on the Inference Platform team owns model execution infrastructure including online and batch inference on Kubernetes. This role optimizes performance, cost, and reliability of model serving for enterprise and mid-market customers.

Responsibilities

At Avra, every technical IC is a Member of Technical Staff (MTS). The title doesn't put anyone in a silo: you own systems and outcomes, not steps in a function, and you keep building depth in your area.
Evolve Sophos, our online and batch inference runtime, built on Kubernetes.
Run large batch inference on ephemeral jobs, with multi-dimensional admission control (CPU, memory, GPU).
Build and extend the controller and its Kubernetes custom resources.
Optimize each model's inference engine and feature processing.
Serve graphs and data efficiently.
Own execution of training, post-training, and fine-tuning jobs, in our cloud and in customer dataplanes / on-premisse cloud.
Drive autoscaling, GPU serving, performance, and cost optimization, with telemetry for every model we run.
How we measure success
99.9% serving availability.
p95/p99 latency for online inference and throughput for batch.
Cost per prediction and per training job.

Qualification

A clear sense of costProduction-quality code and reviewsRayAdmission-control systemsArrowGCP/AWS and GKE/EKS

Required

Experience running model serving or large-scale batch compute on Kubernetes.
Experience building Kubernetes controllers or operators.
Skill at profiling and optimizing data-heavy Python pipelines.
A clear sense of cost: you treat compute efficiency as a product feature.
Production-quality code and reviews, and a willingness to operate what you build.

Preferred

Ray, Ray Serve, or KubeRay in production.
Admission-control systems.
GPU serving and performance optimization.
Arrow, Parquet, Lance, or other columnar formats.
Shipping software to customer-hosted Kubernetes.
GCP/AWS and GKE/EKS, and financial services or regulated environments.

Company

Avra

View company page

Source: ashby

Active within the past hour

Remote-friendly

More roles at Avra

Member of Technical Staff | Deployment Platform

Head of Research (AI)

São Paulo · Remote

Boost Your Interview Chances

Improve Resume Match Score

Free

Your Score

9.0

Strong Applicant

Must-Have Skills for This Role

KubernetesPythonRayRay ServeKubeRayGPU serving

Optimize my Resume

Apply Faster with Autofill

Free

Save your profile once. LumiJobs pre-fills the employer's application form so you stop re-typing the same answers on every site.

Apply With Autofill

Or skip LumiJobs's tools and submit directly on the employer's career site.

Apply on Employer Site

Member of Technical Staff | Inference Platform at Avra · LumiJobs