
AI/ML Software QA Lead
Nesco Resource โข Austin, Texas, United States
Role & seniority: Quality Assurance Manager (senior) overseeing end-to-end QA for enterprise apps, data-driven and AI-enabled solutions; leads analysts and engineers.
Stack/tools: Azure DevOps/TFS/MTM/HPQC; Selenium, UiPath; MLflow, TFX, PyCaret, pytest; Azure AI services, Power Automate; SharePoint; MS Office.
Top 3 responsibilities
-
Lead, mentor, and scale a QA team focused on AI/ML testing; build and execute test plans for models, data pipelines, and features.
-
Oversee manual and automated testing across projects; define acceptance criteria and test strategies with engineering, product, and data science.
-
Manage test environments/data, track QA metrics, mitigate quality and AI-specific risks, and drive continuous QA improvement across automation and tooling.
Must-have skills
-
5+ years QA leadership in software, data, or AI/ML domains.
-
Strong AI/ML testing knowledge (model evaluation, fairness, bias; supervised/unsupervised learning); automation scripting; advanced SQL.
-
Proficiency with QA methodologies, SDLC, and MLOps; experience coordinating offshore/nearshore teams; strong communication and stakeholder management.
Nice-to-haves
- Experience with data privacy, ethical AI standards, and regulatory compliance; collaboration with automation teams; production validation support.
Location & work type: Location not specified; full-time role.
Full Description
Summary: The Quality Assurance Manager oversees full end-to-end testing for enterprise applications, data-driven systems, and AI-enabled solutions. This position leads a team of analysts and engineers, ensuring detailed test planning, execution, and defect resolution throughout the software lifecycle. The role requires deep expertise in QA methodologies, AI/ML testing practices, and cross-functional collaboration across engineering, product, and data teams.
KEY RESPONSIBILITIES
Lead, mentor, and grow a QA team with a focus on AI/ML testing. Build and execute test plans for AI models, data pipelines, and application features. Oversee manual and automated testing across multiple projects and integrated systems. Partner with engineering, product, and data science to define acceptance criteria and test strategies. Support deployments across Dev, QA, and Production environments. Maintain test environments, test data, and required infrastructure. Identify and mitigate quality risks, including AI-specific risks such as bias, data leakage, and adversarial threats. Track QA metrics (coverage, defect leakage, RAG status) and report to stakeholders. Drive continuous improvement in QA processes, automation, and tooling. Ensure compliance with ethical AI standards, data privacy requirements, and regulatory expectations. Collaborate with automation teams to enhance automated test suites. Support production validation and provide user support when needed.
SKILLS & EXPERIENCE
5+ years of QA leadership experience within software, data, or AI/ML environments. Strong understanding of AI/ML testing concepts (model evaluation, fairness/bias, supervised/unsupervised learning). Hands-on experience with automation frameworks and scripting. Advanced SQL skills and comfort validating data workflows. Proficiency with QA methodologies, SDLC processes, and MLOps practices. Experience coordinating offshore/nearshore development or testing teams. Excellent communication, documentation, and stakeholder engagement skills. Highly organized, detail-oriented, and effective in fast-paced environments.
TOOLS & TECHNOLOGIES
ALM/QA Tools: Azure DevOps, TFS, MTM, HPQC or similar
Automation: Selenium, UiPath, comparable frameworks
ML/AI Testing: MLflow, TFX, PyCaret, pytest
Cloud/Automation: Azure AI services, Power Automate
Collaboration: SharePoint
General: MS Office Suite
Nesco Resource offers a comprehensive benefits package for our associates, which includes a MEC (Minimum Essential Coverage) plan that encompasses Medical, Vision, Dental, 401K, and EAP (Employee Assistance Program) services.
Nesco Resource provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws.