Full Description

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Primary Responsibilities

Design, lead, and execute comprehensive evaluation plans for GenAI systems, including accuracy, hallucination rate, semantic correctness, and safety compliance Perform advanced prompt testing, API testing, and AI agent validation using Python and pytest, ensuring robust coverage Drive manual and automated testing for React/Next.js applications and chatbots using Cypress, Playwright, and other modern tools Validate and enhance guardrails, safety mechanisms, and ethical compliance for AI systems Collaborate closely with developers and product teams to reproduce complex issues, ensure regression coverage, and improve quality processes Maintain and optimise dashboards for quality metrics, integrating evaluation gates and automated checks into CI/CD pipelines Apply Node.js, yarn, npm for scripting and automation workflows, and contribute to framework improvements Ensure adherence to organisational standards and best practices for GenAI quality assurance and risk mitigation Document test cases, maintain regression suites, and enforce traceability for compliance audits Lead and support human-in-the-loop evaluation processes and continuous improvement initiatives Mentor junior QA engineers and contribute to architecture and quality strategy discussions Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications

Bachelor's or Master's in Computer Science, Engineering, or related field 6+ years of experience in software quality assurance with increasing responsibility Hands-on experience with Cypress, Playwright, Jest for UI and end-to-end testing Experience integrating QA processes into CI/CD pipelines and DevOps workflows Expert proficiency in Python and test automation frameworks (pytest) Solid familiarity with AI/ML evaluation strategies, prompt testing, and safety validation Working knowledge of Node.js and scripting for automation Deep understanding of security, compliance, and ethical considerations in AI systems Proven solid analytical, problem-solving, and debugging skills with attention to detail Proven ability to lead collaborative efforts with development teams and influence quality strategy Proven excellent documentation, communication, and leadership skills