Sunset Backend Engineer at Sunset responsible for designing and shipping backend systems for multi-stage, high-volume data processing pipelines. Ensures correctness, replayability, and safety of data de-identification pipelines.
Responsibilities
We're hiring a backend engineer to make the pipeline that de-identifies sensitive enterprise data correct, replayable, operable, and safe to change.
This is not a conventional data-platform role where a successful job is enough. A pipeline can finish while silently dropping records, duplicating output, applying stale policy, losing lineage, or producing evidence that cannot establish whether a dataset is safe to release. You will own the backend systems and contracts that make those failure modes visible, preventable, and recoverable.
Design and ship backend systems for multi-stage, high-volume data processing
Define authoritative, versioned contracts for manifests, artifacts, lineage, and state transitions
Make retries, checkpoints, partial failures, replay, backfills, migrations, and rollbacks safe and understandable
Build independent reconciliation and verification instead of treating job success as proof of correct output
Turn escaped and recurring failures into fixtures, regression coverage, release gates, and durable recovery paths
Expose trustworthy run state and safe controls to the products people use to investigate and release data
Diagnose production behavior across code, queues, stores, artifacts, data formats, and deployed versions
Improve correctness, throughput, and operating leverage without weakening privacy, security, or release confidence
Use AI deeply in development and in bounded verification systems, with explicit evaluation and independent checks
Partner with machine learning, applied science, full-stack product, platform, security, and data engineering teammates
Qualification
You communicate clearly across productThis Role May Not Be for You IfYou treat pipeline success
Required
You are strong in asynchronous or distributed systems and can reason precisely about queues, concurrency, state, storage, idempotency, partial failure, and recovery
You define invariants and use reconciliation, control totals, diffs, replay, goldens, shadow paths, or independent sources to verify correctness
You can design versioned data and artifact contracts and migrate them safely in a live system
You debug from evidence across system boundaries and turn incidents into durable system improvements
You choose technical work based on operator and customer consequences, not architecture in isolation
You use modern AI engineering tools fluently, verify their output, and know when model-backed checks need deterministic guardrails and human review
You communicate clearly across product, ML, data, platform, security, and customer-facing teams
This Role May Not Be for You If
You want to focus primarily on frontend product development or visual craft
You treat pipeline success, uptime, latency, or a green dashboard as sufficient evidence that the output is correct
You prefer isolated infrastructure work without responsibility for data and delivery consequences
You solve partial failure primarily with retries and manual runbooks
