Senior Software Engineer, LLMs
Upstart is seeking a Senior Software Engineer to join our Core GenAI Platform team, which is dedicated to building foundational infrastructure that democratizes access to generative AI and large language models (LLMs) across the company. This team operates with broad scope and high visibility, partnering across Machine Learning, Product, and Compliance to ensure every model integration is performant, secure, and compliant.
In this role, you will design and build a unified interface for securely interacting with LLMs, abstracting away the complexity of model selection, prompt orchestration, evaluation, and scaling. You will develop a centralized GenAI layer that enables engineers to use best-in-class LLMs through clean APIs, configurable pipelines, and intuitive tooling. Additionally, you will drive cross-functional adoption of AI tooling by delivering reusable components and intuitive interfaces, increasing developer productivity through automation, metrics dashboards, and streamlined GenAI integrations.
The ideal candidate will have at least 6 years of experience in object-oriented software engineering, with a strong systems engineering background. Proficiency in backend development with Python (leveraging frameworks like FastAPI or Flask), microservices architecture, and infrastructure tools like Kubernetes, Docker, and Terraform is required. Hands-on experience building ML platforms or infrastructure supporting LLMs or inference systems is essential. Strong written and verbal communication skills, along with a collaborative approach to cross-functional work, are also necessary.
Upstart offers competitive compensation, including base salary, bonus opportunities, and equity grants. Comprehensive medical, dental, and vision coverage is provided, along with Health Savings Account contributions. The company also offers a 401(k) plan with company match, Employee Stock Purchase Plan (ESPP), and generous paid time off, including holidays, vacation, sick, and safety leave. Supportive parental, family care, and military leave programs are also available.
As a digital-first company, Upstart provides employees with the flexibility to work from anywhere in the United States, with offices in San Mateo, California; Columbus, Ohio; and Austin, Texas. The company fosters a collaborative and inclusive culture, offering opportunities for professional growth and development. Regular on-site collaboration sessions occur three days per quarter at an Upstart office, with travel expenses covered by the company.