Research Engineer, Infrastructure, Inference
Thinking Machines Lab is seeking a Research Engineer to enhance and scale the infrastructure supporting large AI models. This role focuses on optimizing inference systems to improve performance, cost-effectiveness, reliability, and reproducibility, enabling teams to concentrate on advancing model capabilities without infrastructure constraints.
Key responsibilities include collaborating with researchers and engineers to transition cutting-edge AI models into production, designing and implementing techniques and architectures that enhance performance metrics such as latency and throughput, and optimizing codebases and compute resources like GPUs to maximize hardware utilization. The role also involves extending orchestration frameworks for distributed inference and establishing standards for reliability and observability across the inference stack.
Candidates should possess a bachelor's degree or equivalent experience in computer science, engineering, or a related field, along with a solid understanding of deep learning frameworks like PyTorch or JAX and their system architectures. Experience with inference serving systems optimized for throughput and latency, such as SGLang or vLLM, is essential. The role requires strong engineering skills, the ability to write maintainable code, and proficiency in debugging complex codebases.
The position is based in San Francisco, California, with an expected annual salary range of $350,000 to $475,000 USD, depending on background, skills, and experience. Thinking Machines Lab offers generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.
Thinking Machines Lab fosters a collaborative environment where scientists, engineers, and builders work together to create widely used AI products and open-source projects. The company values innovation and provides opportunities for professional growth, making it an ideal workplace for those passionate about advancing AI infrastructure.