Our customers run large AI training and inference workloads on GPU clusters we source from providers worldwide. When a node goes dark or a job dies eight hours into a run, the Customer Reliability Engineer is who they hear from, and who gets it sorted.

You need to be comfortable in a Linux shell and know how Kubernetes works. You don't need GPU or HPC experience. Most people pick that up here.

Triage and fix customer issues

Own issues start to finish: reproduce, diagnose, fix or escalate, close the loop

Debug at the Linux layer: processes, networking, storage, kernel logs, resource contention, systemd, journald

Dig into Kubernetes problems like pods stuck pending or crash-looping, node conditions, scheduling failures, resource limits

Work GPU failures: driver and device-plugin issues, XID errors, thermal throttling, nodes that need cordoning or draining, jobs failing across multiple nodes

Escalate when you're past your depth, with the evidence already gathered

Handle incidents

Take part in a 24/7 on-call rotation

First response on alerts and customer-reported outages: assess impact, set severity, pull in the right people

Keep customers updated during incidents. Clear status, honest unknowns, no silence

Customer Reliability Engineer

Responsibilities

Qualification

Required

Benefits

Company

Andromeda

More roles at Andromeda