SunnyData Data Solutions Architect at SunnyData responsible for designing and deploying data solutions on Databricks and supporting presales activities. Manages multiple customer accounts and mentors internal teams.
Responsibilities
Serve as a trusted advisor to customers on data engineering and architecture strategy
Design, build, and deploy comprehensive data solutions for AI, ML, and BI initiatives
Partner with sales teams to present technical solutions to prospective clients
Manage multiple customer accounts ensuring timely delivery and clear reporting
Architect data solutions incorporating governance, security, and data quality best practices
Evaluate data sources and recommend inclusion strategies
Guide internal teams and provide mentorship
Perform system analysis, diagnose data and system defects, and apply fixes
Test data movement, transformation code, and data components for accuracy and reliability
Qualification
Proficiency in JavaExperience with AWSAble to writeStrong written and verbal communicationExperience in client facing rolesDistributed StorageData IntegrationSoftware Development LifecycleAutomated PipelinesWorkflow ManagementDomain KnowledgeRegulatory Compliance
Required
10+ years as a hands on Solutions Architect and/or Senior Data Engineer
Recent experience implementing data solutions on Databricks
Expertise in data engineering technologies (Spark, Hadoop, Kafka)
Expertise in Databricks platform and data science/ML tooling (pandas, scikitlearn)
Deep understanding of end to end data analytics workflow
Distributed Storage: Familiarity with cloud and distributed data storage systems such as S3, ADLS, HDFS, GCS, Kudu, ElasticSearch/Solr, Cassandra, or other NoSQL storage systems.
Data Integration: Experience with data integration technologies like Spark, Kafka, StreamSets, Matillion, Fivetran, NiFi, AWS Data Migration Services, Azure Data Factory, and Informatica Intelligent Cloud Services (IICS).
Software Development Lifecycle: Comprehensive experience with the complete SDLC, including design, documentation, implementation, testing, and deployment.
Automated Pipelines: Expertise in automated data transformation and curation using tools like dbt, Spark, Spark Streaming, and automated pipelines.
Workflow Management: Experience with workflow management and orchestration tools like Airflow, AWS Managed Airflow, Luigi, and NiFi.
Domain Knowledge: Background working in the Financial Services industry, preferably in a banking environment.
Regulatory Compliance: Understanding of industry compliance requirements and standards, specifically in banking IT landscapes.
Certifications (at least 2 of the following): Associate Developer for Apache Spark, Data Engineer Associate, Professional Data Engineer, Machine Learning Associate, Professional ML Engineer.
