Software Engineer, GPU Performance
HeyGen is seeking a Software Engineer specializing in GPU performance to enhance the efficiency of our AI applications, including Avatar IV, Photo Avatar, Interactive Avatar, and Video Translation. This role involves optimizing model execution and inference infrastructure to improve latency, throughput, and GPU cost. Ideal candidates will have a strong understanding of GPU hardware and a passion for performance optimization.
In this position, you will utilize tools such as NVIDIA Nsight Systems, Nsight Compute, and PyTorch Profiler to analyze GPU utilization, kernel execution, memory bandwidth, and CPU–GPU data movement. Your responsibilities will include identifying bottlenecks across model execution, preprocessing, and inference serving, and implementing optimizations through batching, scheduling, memory management, and enhanced GPU utilization. Additionally, you will develop or integrate high-performance GPU kernels and collaborate with AI researchers and infrastructure engineers to deploy optimizations into production.
Candidates should have experience optimizing GPU-based AI workloads or high-performance computing systems, proficiency in Python, and familiarity with machine learning frameworks like PyTorch. A strong curiosity about GPU hardware, including memory bandwidth, cache behavior, tensor cores, and data movement between CPU and GPU, is essential. Experience with profiling tools to connect hardware behavior to application-level bottlenecks and validate improvements is also required.
Preferred qualifications include experience with CUDA, Triton, or C++ GPU programming, optimizing video, image, audio, diffusion, or Transformer models, and familiarity with multi-GPU inference, GPU interconnects, quantization, or large-scale model serving. Experience building performance benchmarks or regression testing infrastructure and prior experience in a fast-paced technology environment are also advantageous.
HeyGen offers a competitive salary and benefits package, a dynamic and inclusive work environment, and opportunities for professional growth and advancement. We value innovation and creativity and provide access to the latest technologies and tools. Join us in making AI video faster and more accessible.