Applied Scientist (Evaluation & Safety)

Job DescriptionJob DescriptionAbout Amigo

Amigo builds trust and safety infrastructure for AI in mission-critical environments.

We partner with organizations in healthcare and other regulated sectors to deploy AI systems that operate reliably when the stakes are highest. Our infrastructure enables verification, monitoring, and real-time oversight—ensuring AI serves people safely at scale.

We've raised $6.5M from General Catalyst and GSV Ventures. Our team combines expertise in distributed systems, quantitative research, clinical operations, and regulatory environments to build AI that organizations can trust.

About this role

As an Applied Scientist - Evaluation & Safety at Amigo, you'll build the evaluation frameworks that ensure our systems are safe and effective in healthcare environments. You'll design adversarial testing systems, create safety evaluation protocols, and build frameworks that catch potential failures before they reach patients. Your work is critical to maintaining the reliability and safety standards that healthcare organizations require.

What you'll do

• Design evaluation frameworks for healthcare systems

• Build adversarial testing systems that identify potential failure modes before deployment

• Create safety evaluation protocols that ensure systems meet healthcare standards

• Design test suites that cover edge cases, rare diseases, and complex medical scenarios

• Build automated evaluation pipelines that continuously assess system performance

• Collaborate with healthcare professionals to develop clinically relevant evaluation metrics

What we're looking for

• Strong background in system safety and adversarial testing

• Experience with bias detection and fairness testing

• Knowledge of healthcare safety standards

• Understanding of statistical evaluation methods

• Track record of building evaluation systems for production software

Nice to have

• Healthcare or regulated industry experience

• Background in medical device testing or clinical trial design

• Experience with formal verification or safety-critical systems

• Knowledge of healthcare compliance and audit requirements

Benefits

Health & Wellness

• Comprehensive health, dental, and vision insurance

• Mental health support and wellness coaching

• Flexible wellness stipend for fitness, therapy, or personal growth

• Daily catered lunch and dinner

Growth & Development

• Annual learning budget for courses, books, or conferences

• Conference attendance budget for professional development

• Development setup of your choice

• Academic collaboration opportunities

Compensation Range: $150K - $300K