Data Engineer

Remotefull timeRemote

Apply on Employer Site

mid

H1B: no

People Data Labs Data Engineer at People Data Labs builds infrastructure for data ingestion, transformation, and loading using Spark, SQL, AWS, and Databricks. The role involves developing scalable data processing systems and CI/CD pipelines to improve data quality.

Responsibilities

Build infrastructure for ingestion, transformation, and loading an exponentially increasing volume of data from a variety of sources using Spark, SQL, AWS, and Databricks
Build an organic entity resolution framework capable of correctly merging hundreds of billions of individual entities into a number of clean, consumable datasets
Develop CI/CD pipelines and anomaly detection systems capable of continuously improving the quality of data pushed into production
Dream up solutions to largely undefined data engineering and data science problems

Qualification

Experience with PythonExpertise with Apache Spark (JavaExperience with SQLExperience with data warehousing (e

Required

3-5+ years of industry experience with clear examples of strategic technical problem-solving and implementation
Strong software development fundamentals
Experience with Python
Expertise with Apache Spark (Java, Scala, and/or Python-based)
Experience with SQL
Experience building scalable data processing systems from the ground up
Experience using developer-oriented data pipeline and workflow orchestration (e.g., Airflow, dbt, dagster or similar)
Knowledge of modern data design and storage patterns
Experience working in Databricks (including delta live tables, data lakehouse patterns, etc.)
Experience with cloud computing services (AWS preferred, GCP, Azure or similar)
Experience with data warehousing (e.g., Databricks, Snowflake, Redshift, BigQuery, or similar)

Benefits

Dental
Vision
PTO

Company

People Data Labs

View company page

Source: ashby

Active within the past hour

Remote-friendly

More roles at People Data Labs

Senior Data Engineer

Remote

Manager, IT & Security

Remote

Senior Software Engineer, Data Acquisition

Remote

Senior Software Engineer, Full Stack

Remote

Boost Your Interview Chances

Improve Resume Match Score

Free

Your Score

9.0

Strong Applicant

Must-Have Skills for This Role

PythonApache SparkSQLAirflowdbtdagster

Optimize my Resume

Apply Faster with Autofill

Free

Save your profile once. LumiJobs pre-fills the employer's application form so you stop re-typing the same answers on every site.

Apply With Autofill

Or skip LumiJobs's tools and submit directly on the employer's career site.

Apply on Employer Site