Brillio 2 Data Architect at Brillio 2 designs scalable data solutions using Databricks and modern cloud platforms. The role includes developing ETL pipelines, implementing data models, and ensuring data quality and security.
Responsibilities
Design and architect scalable, high-performance data solutions utilizing Databricks and leading cloud data platforms
Develop, optimize, and maintain ETL pipelines with SQL, Python, and Databricks to ensure efficient data ingestion, transformation, and loading
Implement robust data models and data warehousing solutions to drive analytics, reporting, and business intelligence outcomes
Automate and streamline data workflows to enhance system efficiency and minimize processing times
Monitor, troubleshoot, and resolve performance bottlenecks in data pipelines and platform components
Ensure data quality, consistency, and security across all data assets and platforms
Collaborate with cross-functional teams to translate business requirements into technical data architecture solutions
Document architecture designs, data flows, and best practices to support operational excellence and knowledge sharing
Qualification
Expert-level SQLExperience with Azure Data LakeExposure to data governanceBachelor's degree in Computer Science
Required
Advanced proficiency with Databricks platform and ecosystem
Expertise in designing and developing ETL pipelines using SQL and Python
Expert-level SQL, including advanced query optimization
Proficiency in Python for data engineering and automation
Comprehensive knowledge of data warehousing concepts and design patterns
Experience with data modeling techniques and best practices
Hands-on experience with PLSQL and T-SQL
Ability to develop and maintain stored procedures for data transformation
Experience with performance tuning and troubleshooting in distributed data environments
Preferred
Experience with Azure Data Lake, AWS S3, or Google BigQuery
Familiarity with Delta Lake and lakehouse architecture concepts
Knowledge of CI/CD practices for data engineering pipelines
Working experience with Scala or Java in data engineering contexts
Exposure to data governance, cataloging, and lineage tools
Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a closely related discipline
Databricks Certified Data Engineer Associate or Professional certification
Certification in cloud data platforms such as Microsoft Azure Data Engineer Associate, AWS Certified Data Analytics, or Google Professional Data Engineer