Data Warehouse Software Engineer III at Babelstreet · LumiJobs

Responsibilities

This is an experienced engineering role requiring strong Python skills, deep expertise across the entire GCP analytics stack, and hands-on experience managing and upgrading production Airflow environments.
Data Warehouse & Pipeline Engineering: Architect, build, and maintain production-grade ELT/ETL data pipelines that stream and batch high-volume, complex datasets into Google BigQuery.
Airflow Management & Platform Upgrades: Manage, monitor, and upgrade our Apache Airflow / Cloud Composer platform. Write clean, modular, and maintainable DAGs in Python, implement custom operators/hooks, and optimize cluster performance.
GCP Architecture & Optimization: Maintain and optimize core GCP data services (BigQuery, Cloud Storage, Cloud Functions/Run, Pub/Sub, IAM). Implement advanced BigQuery performance tuning (partitioning, clustering, slot management, and query cost control).
Python Engineering & Integrations: Develop high-performance Python scripts, API connectors, and automation tools for data extraction, schema validation, and third-party data integrations.
Data Quality & Integrity: Enforce strict data quality frameworks, monitoring, and automated alerting across all ingestion pipelines to ensure zero data loss and minimal latency.
Technical Leadership & CI/CD: Maintain infrastructure-as-code and CI/CD deployment pipelines for data workloads using Git. Assist in architectural reviews and mentor junior engineers on data warehousing best practices.
Job Requirements:
U.S. Citizenship: Must be a U.S. Citizen due to federal government contract requirements.
Software Engineering Experience: 4–7+ years of professional software development experience, with at least 3+ years focused heavily on data warehousing, cloud data pipelines, and data platform operations.
Advanced Python Mastery: Production-grade Python development skills for data processing, custom pipeline tooling, and API integrations.
Airflow Platform Operations: Hands-on experience with Apache Airflow (Cloud Composer preferred), including DAG architecture, troubleshooting, environment configuration, and platform version upgrades.

Data Warehouse Software Engineer III

Responsibilities

Qualification

Required

Preferred

Benefits

Company

Babelstreet

More roles at Babelstreet