**Role & seniority: ** Senior QA Engineer (5+ years)

**Stack/tools: **

Languages: Python, SQL
Testing: REST/GraphQL API testing; PyTest; Postman; dbt tests (desirable)
QA/automation: automation scripting, real-time/streaming validation, API monitoring
GenAI testing: prompt/grounding validation, golden sets, non-deterministic assertions
AI tools: LLMs, chatbot frameworks; Copilot/LLM assistants (e.g., GitHub Copilot, ChatGPT) (nice-to-have)
Quality/safety checks: hallucination, bias, toxicity, PII leakage
Top 3 responsibilities:
- Define and execute QA for data pipelines, APIs, and conversational/AI agents
- Design generative AI test scenarios (chatbot interactions, intent recognition, tone, edge cases; handle non-determinism)
- Perform safety/quality validation (hallucinations, bias/toxicity, PII leakage) and validate structured/unstructured outputs
Must-have skills:
- Strong QA background with data systems or AI applications
- Analytical rigor and attention to detail for detecting subtle inconsistencies
- Python + SQL proficiency for validation and automation
- API testing experience (REST/GraphQL)
- Familiarity with conversational/LLM testing approaches
Nice-to-haves:
- Use of LLM-based assistants (Copilot/ChatGPT) in QA workflow
- Experience with QA automation tools (PyTest, Postman, dbt tests)
Location & work type: UAE (Dubai or Abu Dhabi); work type not speci

Full Description

Senior QA Engineer

Summary We are seeking a detail-oriented Senior QA Engineer to keep the quality assurance efforts across our complex data flows and AI-driven products, specifically focusing on conversational agents. This role requires a blend of automation expertise and strong analytical skills to define quality standards, validate complex data outputs, and ensure the reliable and responsible behavior of our generative AI systems.

Key Responsibilities

Expertise in quality assurance efforts for data pipelines, APIs, and conversational agents. Design and execute generative AI testing scenarios, including exploratory testing for chatbot interactions, intent recognition, tone, and edge-case behavior. Experience in LLM test designs, including prompt and grounding validation, golden sets, and non-deterministic assertions. Perform crucial safety and quality checks, such as testing for hallucination, bias, toxicity, and PII leakage. Validate structured and unstructured data outputs, ensuring consistency, accuracy, and compliance. Establish and drive real-time testing approaches, including streaming data validation and API monitoring. Collaborate with ML and NLP teams to define comprehensive evaluation metrics and criteria for agent performance. Integrate AI-driven tools (like Copilot) into the QA lifecycle to accelerate test design, documentation, and defect analysis.

Requirements

5+ years of experience in QA engineering, with proven experience in testing data systems or AI applications. Strong analytical skills and attention to detail, capable of spotting subtle inconsistencies in data and agent behavior. Proficiency in Python and SQL for data validation, automation scripting, and test preparation. Experience with API testing (REST/GraphQL automation and validation). Familiarity with chatbot frameworks, LLMs, or conversational testing. (Desirable) Experience or strong interest in using LLM-based assistants (e.g., GitHub Copilot, ChatGPT) in test and execution. (Desirable) Exposure to QA automation tools (e.g., PyTest, Postman, dbt tests).

Location: Candidates based in UAE (Dubai or Abu Dhabi)

Come and join our #ParserCommunity. Follow us on Linkedin

Senior QA Engineer (AI-based)

Full Description