Crusoe The Senior Software Engineer at Crusoe builds the Managed Data Transfer (MDT) control plane to enable petabyte-scale data transfers into Crusoe storage. This role involves API design, scheduling, and ensuring correctness at scale.
Responsibilities
Crusoe's Managed Platform Services (MAPS) organization builds the managed services that make Crusoe Cloud easy to adopt and hard to leave, including Crusoe Container Registry, Cloud Monitoring and Managed Logs. Managed Data Transfer (MDT) is its newest service. MDT moves petabyte-scale training data from AWS S3, and later other clouds, into Crusoe storage with a single API call: the service shards the work across a fleet of transfer workers, verifies every object, and asks the customer to run nothing. For AI labs and Physical AI companies whose data lives with a hyperscaler, it removes one of the biggest obstacles to training on Crusoe GPUs.
We are hiring Senior Software Engineers to build the MDT control plane: the job API, scheduler and job state that turn a customer request into a coordinated, resumable transfer. You will join a new team at the beginning, take the service from first commit to its preview launch with design-partner customers, and shape the API that new clouds and Crusoe-internal transfers will plug into later. You will work closely with the Storage, Network, IAM and Cloud Console teams, and report to the Engineering Manager of Managed Data Transfer.
What You'll Be Working On:
Job API and Lifecycle: Design and build the data-transfer API (create, list, get, cancel) with a provider-neutral model for sources and destinations, so new clouds and storage types can be added without breaking customers.
Scheduling and Job State: Build the scheduler and state machine that split a bucket listing into units of work, place them on regional worker pools, and track progress, retries and resume across failures.
Correctness at Scale: Make petabyte-scale jobs safe to retry and cancel, with end-to-end checksum verification and clear errors when a customer's permissions or destination are wrong.
Cross-Cloud Identity: Implement secure access to customer buckets with short-lived, least-privilege credentials (AWS role assumption first, other clouds next), integrated with Crusoe IAM and audit logging.
Customer Surfaces: Ship the CLI and, with partner teams, the Terraform and Console experiences for creating and monitoring transfers.
Operational Excellence: Instrument the service with metrics, logs and alerts, write the runbooks, and take part in the team's on-call rotation.
Cross-Team Collaboration: Agree the interfaces MDT depends on with Storage, Network, IAM and Console, and work with Product and Solutions Architects on early customer onboarding.
What You'll Bring to the Team:
Backend Engineering Craft: Experience designing and shipping production backend services and APIs, ideally in Go, with gRPC or REST, relational databases and Kubernetes.
Qualification
Preferred
Experience with large-scale data movement, such as S3 multipart uploads, rclone, s5cmd, AWS DataSync or Google Storage Transfer Service.
Hands-on experience with cross-cloud identity: AWS IAM and STS, GCP service-account impersonation, or Azure workload identity.
Experience with storage systems (object storage, NFS, parallel file systems) or high-throughput networking.
Experience with job orchestration systems such as Temporal, Argo, or Kubernetes operators.
