What You Get to Do

Contribute to the architecture and improvement of our data acquisition and processing platform, increasing reliability, throughput, and observability

Use and develop web crawling technologies to capture and catalog data on the internet

Build, operate, and evolve large-scale distributed systems that collect, process, and deliver data from across the web

Design and develop backend services that manage distributed job orchestration, data pipelines, and large-scale asynchronous workloads

Structure and model captured data, ensuring high quality and consistency across datasets

Continuously improve the speed, scalability, and fault-tolerance of our ingestion systems

Partner with data product and engineering teams to design and implement new data products powered by the data you help collect, and enhance and improve upon existing products

Learn and apply domain-specific knowledge in web crawling and data acquisition, with mentorship from experienced teammates and access to existing systems

The Technical Chops You’ll Need

7+ years of professional experience building or operating backend or infrastructure systems at scale

Solid programming experience in Python, Go, Rust, or similar, including experience with async / await, coroutines, or concurrency frameworks

Senior Software Engineer, Data Acquisition

Responsibilities

Qualification

Required

Benefits

Company

People Data Labs

More roles at People Data Labs