Responsibilities

As a Senior Data Platform Engineer, you’ll own the durable view of every resolved entity on the platform: the storage layer that holds what we know, where each fact came from, how confident we are in it, and how that picture has changed over time. When two sources disagree, both are kept with their evidence. When someone asks what we believed on a given date, and why, the platform can answer.
You’ll join the analytics and AI engineering team behind a system that ingests records from dozens of disparate sources, resolves them to the right entity, highlights what analysts should review first, and provides transparent, explainable recommendations that users can trust. Operating within a secure government cloud environment, the platform depends on a storage layer that never quietly erases a disagreement between sources.
Upstream matching is probabilistic. A match arrives with a confidence score rather than a yes, and the storage layer carries that uncertainty forward rather than collapsing it into one clean record. Sources change shape without warning and there is no shared key to join on, so this is a current data-engineering problem rather than a classic warehousing one.
The seniority this role calls for is about judgment, not tooling breadth: knowing why you never overwrite, and being able to reconstruct a past decision with its evidence a year later. If your instinct when records disagree is to preserve both and let a human decide, you’ll be at home here.
This is a fully remote role with occasional travel to DEFCON AI headquarters, customer sites, and partner facilities as needed.
Design and build the storage layer that preserves source disagreement and history rather than resolving them away at write time
Build the first release’s result and evidence store: saved per-person results linked to the originating record, prior results and human feedback retained separately, and configuration versions on every result
Implement provenance and lineage across every node and edge against the platform’s evidence-record contract
Store the versioned rule library and configuration, and record on every result the exact rule versions and ordered context supplied to any model-assisted step
Ensure every write is traceable to its origin and every decision is reproducible
Carry match confidence and other uncertainty forward through the platform rather than collapsing it into a single value
Support point-in-time questions: what did we believe about this entity on this date, and on what evidence

Senior Data Platform Engineer at Defcon · LumiJobs

Senior Data Platform Engineer

Responsibilities

Qualification

Required

Preferred

Benefits

Company

Defcon

More roles at Defcon