Scoutmotors Senior DataOps Engineer at Scout Motors to design, implement, and support enterprise-scale cloud data platforms using Databricks and AWS. Lead development and optimization of ETL/ELT pipelines and data infrastructure.
Responsibilities
Become part of an iconic brand that is set to revolutionize the electric pick-up truck & rugged SUV marketplace by achieving the following:
Contribute to the design, implementation, automation, maintenance, and operational support of enterprise-scale cloud data platforms leveraging Databricks, AWS cloud services, and modern Infrastructure as Code (IaC) practices.
Lead the design, development, optimization, and operational management of enterprise-scale ETL/ELT pipelines within Databricks.
Build and maintain scalable batch and streaming data pipelines using PySpark, Spark SQL, Delta Lake, and Databricks Workflows.
Configure, optimize, and manage Databricks clusters for performance, scalability, reliability, and cost efficiency.
Work closely with software development and systems teams to build Data Integration solutions.
Design and build Data models using tools such as Lucid, Talend, Erwin, MySQL workbench.
Define and enhance enterprise data model to reflect relationships and dependencies.
Review application data systems to ensure adherence to data governance policies.
Design and build ETL (Python), ELT(Python) infrastructure, automation, and solutions to transform data as required.
Design and Implement BI dashboards to visualize Trends and Forecasts.
Design and implement data infrastructure components, ensuring high availability, reliability, scalability, and performance.