Netskope Machine Learning Engineer at Netskope designs and develops scalable cloud-based security systems for LLM APIs and Generative AI applications. Leads architectural design and strategic technical directions for the Cloud API Gateway platform.
Responsibilities
Netskope Cloud Data Plane engineers architect and design one of the most scalable, high-performance cloud API routing and orchestration platforms in the world, processing 10+ Gbps of traffic, performing intelligent routing and policy application across LLM providers and AI services using advanced techniques to detect threats and apply dynamic policies, all at line rate.
What's in it for you
What you will be doing
Design, architect and develop high performance, scalable cloud-based security systems to detect and mitigate threats on LLM APIs and Generative AI applications
Setting and driving the overall architectural design and strategic technical directions for the Cloud API Gateway platform
Define and drive best practices around scalable microservices, distributed systems, APIs and secure data flows across multiple LLM providers (Layer 3 to Layer 7)
Lead and inspire a world-class engineering organization, fostering a culture of innovation, collaboration and continuous improvement
Partner with product management to understand customer requirements, lead the team and collaborate with other engineering teams to design and implement high-quality solutions
Build scalable and performant services that route, transform and secure various API protocols (LLM APIs, REST, gRPC, etc.), using advanced data structures and algorithms in GoLang. Perform deep inspection of LLM API traffic in real-time to detect and mitigate attacks on AI services
Design and implement dynamic routing algorithms for multi-provider LLM orchestration, intelligent request transformation, load balancing, and failover logic
Integrate with security features including DLP, Threat protection, and UEBA for LLM-specific attack pattern recognition, content inspection and policy application
Implement sophisticated credential and token management algorithms for secure multi-provider orchestration
Qualification
Proven experience designing large-scaleConfidence under pressure
Required
10 to 12+ years of experience in software engineering, with 5+ years in architectural or technical leadership role
Deep expertise in backend engineering using Golang, or similar languages. Ability to review code in GoLang and similar languages, such as C and C++.
Proven experience designing large-scale, secure and cloud-native systems
Expertise in building high throughput and low latency data systems, such as networking from Layer 3 to Layer 7, web proxy, load balancing, request routing, API gateways, security, monitoring and observability, distributed systems, data processing, etc.
Expertise in dealing with complex architectural problems across multiple products and at the same time handling design issues at module level
Strong understanding of computer architecture – multi-threading, CPU scheduling, memory management with practical knowledge of Linux at system/kernel level
Hands-on experience with Docker and Kubernetes, specifically debugging workloads, networking, and storage in on-prem or hybrid environments
Hands-on experience handling complex and ambiguous customer issues. Persistence in accomplishing objectives despite uncertain situations, obstacles, and setbacks
Excellent communication skills and the ability to position views and arguments appropriately to win support. The ability to negotiate skillfully in tough situations to win concessions without damaging relationships
Confidence under pressure, a mature EQ, and the ability to show a high degree of self control while maintaining a positive attitude despite adversity
Willingness to work with a globally distributed team in different time zones, including US, Taiwan and India.
The ability to represent the company in appropriate external forums and industry events promoting technical thought leadership
