Mphasis logo

Quality Assurance Engineer

Mphasis • Mumbai City, Maharashtra, India

onsitefull-time
Posted Nov 17, 2026Apply by Dec 17, 2026
  • Role & seniority

    • AI/ML Test Engineer (QA/SDET) — 7+ years total experience; 1–2 years in AI/ML domain (also described as 6+ years QA/SDET with AI/ML focus)
  • Stack/tools

    • Testing/automation: Playwright, Selenium (listed), PyTest, API testing via Postman, human-led verification

    • Programming: Python

    • Back-end/data: MongoDB, PostgreSQL, key-value caching, vector DBs Chroma DB / PGVector

    • AI/agent tech: LLMs, prompts/tokens/context/hallucinations, agent workflows, tool/function calling, RAG, memory, multi-agent

    • Cloud/infra: Azure/AWS/GCP (AI services), Git, CI/CD, Docker

    • Agent orchestration: Agents SDK / orchestration frameworks (e.g., tool/function calling reliability)

  • Top 3 responsibilities

    • Evaluate agent decision-making & safety: verify scope decisions, planning steps, tool-call sequences; test guardrails/autonomy levels

    • Audit and supervise agentic testing systems: review AI-generated tests for correctness and coverage; tune what the system chooses to test

    • Verify orchestration reliability & end-to-end workflows: validate agent/LLM pipelines (single/multi-agent, RAG, memory) and troubleshoot via execution logs

  • Must-have skills

    • Strong QA/SDET fundamentals; manual + API testing experience

    • Strong Playwright proficiency

    • Clear understanding of Agentic AI: planning/execution, tool/function calling, memory, non-deterministic outputs, orchestratio

Full Description

AI/ML Test Engineer – Job Description

Minimum 7 years of experience Job Description –

Evaluate agent decision‑making: Assess whether an agent’s scope decisions, planning steps, and tool‑calling sequences are correct and safe Strong Playwright experience and understanding Agentic AI experience and understanding API

Audit AI‑generated tests: Review assertions and test cases produced by agentic testing systems to ensure they verify the right behaviours.

Supervise agentic testing systems: Tune and correct what the AI testing system chooses to cover, rather than writing every test manually

Verify orchestration reliability: Validate the orchestration framework (e.g., Agents SDK) and ensure tool/function‑calling reliability.

Check guardrails and autonomy levels: Test how much autonomy agents have in production and whether guardrails prevent unsafe actions

Build independent verification layers: Maintain human‑owned checks that validate AI‑generated outputs. 6+ Years of Total IT experience in QA/SDET and Testing, with 1to 2 years in the field of AI/ML domain. Test Agentic AI applications and AI-Driven Workflows Validate agent workflows complete flow from start to end Test LLM-based applications for accuracy, relevance, consistency, and hallucinations. Test multi-agent workflows and communication between agents. Test RAG (Retrieval-Augmented Generation) workflows and knowledge retrieval. Validate agent memory and context handling. Perform API testing using Postman or similar tools. Perform database testing and backend data validation using MongoDB/PostgreSQL. Knowledge on Key Value Caching, Vector Databases like Chroma DB/PGVector Perform basic prompt injection and AI security/guardrail testing. Analyse agent execution logs and identify the root cause of failures. Create and maintain test cases, test scenarios, bug reports, and QA documentation. Work closely with AI/ML and development teams to improve overall agent reliability. Required Skills Strong knowledge of Software Testing / QA fundamentals. Experience with Manual Testing and API Testing. Understanding of Generative AI, LLMs, prompts, tokens, context, and hallucinations. Understanding of Agentic AI concepts and agent workflows. Knowledge of AI agents, tool/function calling, planning, execution, and memory. Basic understanding of RAG and vector databases. Ability to design test cases for non-deterministic AI outputs. Strong analytical, debugging, and problem-solving skills. Perform database testing and backend data validation using MongoDB/PostgreSQL. Knowledge on Key Value Caching, Vector Databases like Chroma DB/PGVector. Python knowledge and test automation experience. Selenium, Playwright, PyTest, or similar automation tools. Knowledge of AI evaluation techniques such as groundedness, relevance, faithfulness, and response quality. Experience with Azure/AWS/GCP and AI services. Git, CI/CD, and Docker knowledge. Experience testing multi-agent or AI orchestration systems. Strong QA/SDET engineering background Understanding of agentic orchestration frameworks Ability to evaluate AI generated code, tests, and decisions Risk based thinking and judgment-heavy evaluation

Agentic AIAI Security TestingAPI TestingCI/CDGenerative AILarge Language ModelsManual TestingMongoDBPlaywrightPostgreSQLPythonQuality AssuranceRetrieval-Augmented GenerationSoftware TestingTest AutomationVector Databases

Cookies & analytics consent

We serve candidates globally, so we only activate Google Tag Manager and other analytics after you opt in. This keeps us aligned with GDPR/UK DPA, ePrivacy, LGPD, and similar rules. Essential features still run without analytics cookies.

Read how we use data in our Privacy Policy and Terms of Service.