Samsung Semiconductor Intern for LLM Inference Acceleration Architecture at Samsung Semiconductor focusing on distributed memory-centric accelerators. Works on token-selection patterns, hardware cost estimation, and numerical accuracy validation.
Responsibilities
The AGI team at Samsung Semiconductor explores memory-centric system architectures for next-generation AI inference, where computation happens close to where data lives. We co-design hardware and software to make large-scale LLM inference faster and more efficient, and our work informs Samsung’s future memory and accelerator products. As an intern, you will own a research-driven architecture study end to end — from workload modeling to design-space exploration and an architecture proposal — working closely with architects and performance-modeling engineers, and you will present your findings to the broader organization at the end of the summer.
Location: Daily onsite presence at our San Jose headquarters in alignment with our Flexible Work polic
Reports to: Director
Extend the team’s in-house performance model and simulator to capture modern attention workloads — GQA, MLA, and sparse top-k attention — during LLM decode.
Analyze how each attention variant changes the balance between local computation and cross-device merge/reduction traffic, and pinpoint when merge becomes the bottleneck.
Explore the merge/reduction design space, including where reduction happens (centralized, in-network, or hierarchical), interconnect topology, link bandwidth and latency, and partial-result precision.
Characterize token-selection patterns of sparse attention models and design distributed top-k selection and data-placement strategies that minimize communication and load imbalance.
Estimate first-order area and power costs of proposed reduction hardware and weigh them against performance gains.
Validate the numerical accuracy of reduced-precision merging against full-precision reference implementations.
Collaborate weekly with your mentor and architecture engineers, and share your results through team meetings, a final presentation, and a technical report that inform future accelerator architecture and potential publications or patents.
Complete other responsibilities as assigned.
Pursuing a Master’s or PhD in Computer Engineering, Electrical Engineering, Computer Science, or a related field preferred.
Qualification
Proficiency in Python and/or C++Familiarity with in-network reductionPublications in ISCA
Required
Pursuing a Master’s or PhD in Computer Engineering, Electrical Engineering, Computer Science, or related field preferred
At least 1 academic quarter/semester remaining
Strong background in computer architecture including memory systems and interconnects
Solid understanding of Transformer inference and attention variants
Proficiency in Python and/or C++
Experience using or building architecture simulators or analytical performance models
Preferred
Familiarity with sparse attention models like DeepSeek Sparse Attention, NSA, Quest
Familiarity with online-softmax merging as in FlashAttention/FlashDecoding
Familiarity with in-network reduction
Publications in ISCA, MICRO, HPCA, ASPLOS, or MLSys