Staff Software Engineer- AI Agent Evaluations

ID.me is seeking a Staff Software Engineer to lead the evaluation, testing, and monitoring of AI agents within their engineering team. This role focuses on developing scalable evaluation infrastructure and developer tools to ensure the reliability of AI systems in production environments. The position is based in Mountain View, California, and requires a strong background in backend engineering and AI agent evaluation.

In this role, you will define standards for evaluating and monitoring AI agents, design and maintain evaluation pipelines for large language model (LLM) outputs and agent behaviors, and implement systems to detect issues such as behavioral drift and tool misuse. You will also lead testing strategies for non-deterministic AI systems, build internal tooling to improve the AI development process, and collaborate with cross-functional teams to integrate quality gates into agent workflows.

Candidates should have a Bachelor's degree in Computer Science, Engineering, or a related field, with over 8 years of experience in production software systems. Proficiency in backend programming languages like Python, Java, or Go is essential, along with experience in evaluating or testing LLM-powered features or autonomous agents. Familiarity with AI-assisted development tools and agentic frameworks is also required.

The compensation package includes an annual base salary ranging from $217,565 to $271,000, along with comprehensive benefits such as medical, dental, and vision coverage, health savings and flexible spending accounts, commuter benefits, life and disability insurance, a 401(k) plan with company match, parental leave, paid time off, and additional wellbeing and learning benefits.

ID.me fosters a collaborative and innovative work environment, emphasizing the responsible development and deployment of AI technologies. This role offers opportunities for professional growth and the chance to make a significant impact on the company's AI initiatives.