Principal engineer, AI Serving Framework Architect (Software)
The Architecture Research Lab (ARL) at Samsung Semiconductor is seeking a Principal AI System Architect to address system-level challenges in modern AI, focusing on memory capacity, bandwidth, and system-scale communication. This role involves leveraging Samsung's advanced memory technologies to develop next-generation AI system architectures that enhance performance, efficiency, and scalability.
Key responsibilities include leading research teams in Korea, researching dynamic scheduling methodologies to maximize AI inference performance in multi-rack scale memory-centric systems, investigating methods to accelerate search operations in RAG's vector database and AI Agent's knowledge graph using compute-capable memory, studying strategies for optimal placement of KVCache and vector databases in hierarchical memory to minimize SSD accesses and reduce I/O stalls, and proposing software designs to implement optimization algorithms on open-source platforms such as vLLM.
The ideal candidate will possess a PhD in Computer Science or a related field with over 10 years of experience in AI Serving Frameworks for large-scale computing, particularly focusing on AI workloads. They should have led projects to build and optimize Large Language Model (LLM) Inference Software Stacks on multi-rack scale systems serving over 100,000 users, have extensive experience in designing AI Inference Software Stacks for heterogeneous devices, and possess in-depth understanding of inference engines like vLLM. Proficiency in AI Inference System Profiling and optimization, knowledge of future AI workloads including reasoning models, multi-modal solutions, AI agents, and world models, and strong understanding of compute, memory, and networking bottlenecks in AI systems are also required. Essential skills include proficiency in PyTorch, Python, and C++, along with excellent verbal, presentation, and written communication skills. A collaborative mindset, curiosity, and resilience in solving complex challenges are highly valued.
The position offers a competitive salary range of $219,000 to $351,000 per year, along with a comprehensive benefits package that includes medical, dental, vision, and 401(k) plans. Additional perks encompass over four weeks of paid time off annually, holidays, sick leave, a stipend for fertility care or adoption, medical travel support, virtual veterinary care, on-demand apps, confidential therapy sessions, an onsite café and gym, virtual classes, and a flexible work environment.