QA Tester - AI
Dicetek LLC • Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates
**Role & seniority: ** Senior Quality Automation Engineer (UI & API & AI Validation)
**Stack/tools: **
-
Test automation: Playwright (TypeScript/JavaScript or Python)
-
Backend/event-driven: Kafka (or similar), contract testing, schema validation
-
AI/LLM validation: Ragas, LangSmith, TruLens; dataset/prompt regression for RAG/LLM features
-
DevOps (plus): CI/CD, Docker, Kubernetes
-
Languages: TypeScript/JavaScript or Python
-
Top 3 responsibilities:
-
Build/maintain scalable UI + API automation frameworks (Playwright) and integrate into CI/CD; reduce flaky tests and improve speed.
-
Validate asynchronous microservices and distributed workflows (Kafka), including contract testing and end-to-end event/data integrity.
-
Test/evaluate GenAI/LLM features (accuracy, relevance, hallucination handling) using dedicated evaluation frameworks and regression datasets.
-
-
Must-have skills:
-
5+ years in QA automation/software testing
-
Strong hands-on Playwright experience
-
API + microservices + event-driven testing experience (Kafka)
-
Contract testing and schema validation knowledge
-
Basic exposure to AI/LLM testing and RAG applications
-
-
Nice-to-haves:
-
CI/CD pipeline expertise
-
Docker and Kubernetes experience
-
-
Location & work type: Not specified in provided text.
Full Description
Senior Quality Automation Engineer
(UI & API & AI Validation)
About The Role
We are looking for a Senior Quality Automation Engineer with strong experience in modern test automation, backend validation, and AI-driven application testing.
You will work across UI/API automation, event-driven microservices, and GenAI evaluation workflows to ensure high-quality releases across distributed systems.
Key Responsibilities
Automation Engineering
Build and maintain scalable UI and API automation frameworks using Playwright (TypeScript/JavaScript or Python). Integrate automated tests into CI/CD pipelines. Improve execution speed, reliability, and reduce flaky tests.
Backend & Event-Driven Testing
Validate asynchronous workflows and distributed systems using Kafka or similar messaging platforms. Implement contract testing and schema validation to ensure reliable communication between microservices. Verify event publishing, consumption, and end-to-end data integrity.
AI & LLM Validation
Test and evaluate GenAI/LLM-based features for accuracy, relevance, and hallucination handling. Work with evaluation frameworks such as Ragas, LangSmith, or TruLens. Build and maintain datasets for regression and prompt validation.
Collaboration
Work closely with developers, AI engineers, and product teams to define quality strategies. Participate in architecture, API, and prompt design discussions.
Required Skills
5+ years of experience in QA Automation or Software Testing. Strong hands-on experience with Playwright. Experience testing APIs, microservices, and event-driven systems. Good understanding of Kafka, contract testing, and schema validation. Exposure to AI/LLM testing and RAG-based applications. Proficiency in TypeScript, JavaScript, or Python. Experience with CI/CD pipelines, Docker, and Kubernetes is a plus.
