Responsibilities

The Pentest Data Platform (PDP) team owns the shared data platform that ingests NodeZero output from ephemeral operations and delivers it into durable storage, increasingly, in near real time. The team's core mandate is to build a scalable, event-driven architecture that makes operational insights available in near real-time, not just after it finishes.
PDP is a foundational, platform-oriented team whose outputs power core user experiences within the NodeZero security platform. PDP owns the data contracts and internal data services that everything else is built on top of.
NodeZero generates large volumes of structured pentest telemetry: hosts, findings, attack paths, credentials, weaknesses, and relationship graphs. PDP is responsible for turning that raw telemetry into coherent, durable, and queryable records that downstream teams can build on reliably at enterprise scale.
The team is driving the largest architectural change in Horizon3's history: reworking pentest data processing from a batch-centric monolith into a real-time, event-driven microservice architecture. There is existing code to understand, live customers to protect, and hard distributed-systems problems to solve, all at the same time.
Own event-driven pipelines end to end — design, build, and operate streaming services that ingest, distribute, and process pentest operation results, including streaming and batch ETL, aggregation services, schema migrations, and replayable processing.
Solve real-time asset identity problems — work on asset matching, asset counting, and cross-operation reconciliation, including reducing false splits and overcounting, improving matching explainability, and making asset identity available while an operation is running rather than only after completion.
Evolve the data model — manage durable SQL-backed storage (PostgreSQL), drive schema evolution with zero-regression fidelity, and ensure data correctness, idempotency, and reliable reprocessing and failure handling.
Drive enterprise scalability — own database performance, connection pooling, resource management, and platform reliability for a system that must handle hundreds of thousands of hosts per customer.
Translate batch logic into streaming designs — convert batch business logic (finding enrichment, attack path analysis, host and identity correlation and other core requirements) into streaming designs without regressing fidelity for existing customers.
Build and maintain shared platform infrastructure — contribute to common processing services that underpin features that enable customers to discover, prioritize, and report on weaknesses; support integration-test infrastructure including delta tracking and batch-transformation frameworks; build auto-discovery and code-generation tooling to accelerate new streaming service development.
Own operational excellence — maintain ETL dashboards and data-platform monitors, participate in on-call rotations and incident response for production processing failures, manage stuck-operation workflows, define SLOs, and set up instrumentation and alerting for systems that are central to every pentest Horizon3 runs.
Raise the technical bar — contribute through design reviews, technical writing, merge request reviews, clear API contracts with neighboring platform and product teams, and durable decision records.

Senior Backend Engineer, Data & App Platform at horizon3ai · LumiJobs

Senior Backend Engineer, Data & App Platform

Responsibilities

Benefits

Company

horizon3ai

More roles at horizon3ai