Responsibilities

Own the architecture and multi-year technical strategy for the data platform, including the lake/lakehouse, ClickHouse, Elasticsearch, and governance, and keep them coherent as one platform rather than separate systems. Define where each engine fits (for example, ClickHouse versus open table formats on object storage) and the migration path between them.
Make and document the major design decisions (ADRs, RFCs) on storage and open table formats, catalog strategy, compute engines, tenancy, data placement, and build-versus-adopt tradeoffs, with cost and performance at petabyte scale as explicit design constraints.
Set the reference architectures for how our core storage and query engines (ClickHouse, Elasticsearch, and lake engines) are deployed, tuned, scaled and operated as shared platforms, including capacity, cost, tenant isolation, and upgrade strategy.
Define the reliability model for the platform: availability targets, failure domains, disaster recovery, observability, and incident response patterns that teams can adopt.
Build privacy and compliance into the architecture, not onto it. Design deletion and right-to-erasure, retention, data residency, and PII classification so they work across the platform’s storage layers, backups, and derived datasets, including data that arrives from upstream streaming systems.
Drive data governance: classification, catalog and metadata, lineage, access control integrated with our authentication and authorization systems, data contracts, quality and integrity standards, and a catalog teams actually use.
Review designs and technical direction across the org, and act as the escalation point on hard distributed-systems and performance problems.
Mentor principal and senior engineers, and grow architectural judgment across the org, not only within one squad.
Partner with product, security, infrastructure and consuming teams to turn business needs into platform roadmaps, and say no or re-sequence when the tradeoffs require it.
Track emerging data technologies (such as streaming-first architectures, federated query, and AI-ready data layers) and bring forward the ones worth adopting, with evidence from prototypes or benchmarks.

Qualification

Proficiency in one or more of Python

Required

Twilio values diverse experiences from all kinds of industries, and we encourage everyone who meets the required qualifications to apply. If your career is just starting or hasn't followed a traditional path, don't let that stop you from considering Twilio. We are always looking for people who will bring something new to the table!
Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
15+ years of experience in software development or a related field, with at least 8 years in an architecture or senior technical leadership role.
Proven track record of architecting and delivering large-scale data platforms and setting their technical strategy, on cloud infrastructure such as AWS, deployed with Kubernetes, Terraform and Helm.
Deep expertise in storage and compute for analytical data: columnar and open table formats, lake and lakehouse architecture, partitioning, indexing, query planning, and how the choices affect cost and performance at petabyte scale.
Breadth across data storage and retrieval technologies, with hands-on depth in at least two of: analytical/OLAP engines (e.g., ClickHouse), search engines (e.g., ElasticSearch/OpenSearch), and lake or distributed processing engines (e.g., Spark, Trino, Flink). You can reason about when to use each, how they interoperate, and how to migrate between them.
Working knowledge of streaming systems (e.g., Kafka) and how they interact with downstream storage.
Practical experience designing data governance at scale: data classification, catalog and metadata management, lineage, and access control integrated with authentication and authorization systems.
Practical experience implementing privacy and compliance requirements (deletion, retention, residency) across a large, heterogeneous data estate.
Proficiency in one or more of Python, Java, and Go.
Track record of writing and driving architecture decisions across multiple teams, and of changing the direction of an organization’s technical roadmap through influence rather than authority.
Experience elevating engineering standards and mentoring senior engineers across teams.

Architect at Twilio · LumiJobs

Architect

Responsibilities

Qualification

Required

Benefits

Company

Twilio

More roles at Twilio