Full Description

Job Title: Gen AI QA Engineer

Location: Downtown, Toronto, ON (Hybrid)

Duration: Long-term contract.

Key Skills: GenAI experience, AI/ML testing, model validation, performance & reliability testing.

Job Description role and Responsibilities

• Define quality standards for generative AI solutions, ensuring alignment with business and compliance requirements.
• Design and execute test strategies that validate AI outputs for accuracy, consistency, and usability.
• Collaborate with delivery teams to embed QA practices into the development lifecycle from the start.
• Automate testing frameworks to handle large-scale validation of AI models and workflows.
• Monitor performance and drift in AI models, ensuring solutions remain reliable over time.
• Identify risks and anomalies in AI behavior and work with teams to resolve them proactively.
• Document test results and insights to provide transparency and confidence to stakeholders.
• Champion responsible AI practices by validating fairness, bias mitigation, and ethical use in solutions.

Mandatory Skills

• GenAI experience
• AI/ML Testing Frameworks
• Automation for AI QA
• Model Validation & Benchmarking
• Data Quality Assurance
• Performance & Reliability Testing
• Stakeholder Communication
• Documentation & Reporting
• Agile/DevOps Practices
• Risk Management
• Ethical & Responsible AI Awareness

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience in QA roles with a focus on AI solutions.
• Strong analytical and problem-solving skills.