
QA Engineer (Remote)
Joblet-AI • United Arab Emirates
**Role & seniority: ** QA Engineer (Contractor) — no formal seniority level stated; target specialists with strong professional QA background
**Location & work type: ** Remote
**Stack/tools (examples): ** Selenium, Playwright, Cypress, Appium, Postman; Jira, TestRail, Zephyr, BrowserStack (or similar)
**Top 3 responsibilities: **
-
Evaluate and rate AI-generated technical outputs using defined quality criteria/rubrics (flag plausible-but-wrong/incomplete answers)
-
Design/assess functional, regression, and edge-case test cases (including negative/boundary tests)
-
Review bug reports/test documentation; ensure reproducibility, correct severity; document defects with reproduction steps and tracking
-
Must-have skills:
-
Strong QA fundamentals: test case design, regression testing, bug tracking, manual + automated testing
-
Meticulous attention to detail and analytical/problem-solving skills
-
Clear written English (B2+) with actionable feedback/annotations
-
Ability to begin promptly; reliable internet connection
-
-
Nice-to-haves:
-
Paid experience with human data annotation, RLHF, AI response/model output evaluation, or rubric-based grading for AI training
-
Hands-on automation frameworks and test management tool experience (from the listed examples)
-
-
Additional responsibility (also listed): Collaborate to refine evaluation guidelines and continuously improve testing standards/methodologies
Full Description
Role Title: QA Engineer
Role Type: Contractor
Location: Remote
QA Engineers to contribute expertise in software quality evaluation for a customer's high-volume project. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters. This opportunity is open to specialists with strong professional QA backgrounds. Prior paid experience with human data annotation, RLHF, model output evaluation, or rubric-based grading for AI training is a plus, but not required.
Scope of Work
-
Evaluate and rate AI-generated technical outputs against defined quality criteria and rubrics, identifying answers that may appear correct but are actually inaccurate or incomplete.
-
Design and assess comprehensive test cases for functional, regression, and edge-case scenarios, including negative and boundary tests.
-
Review bug reports and test documentation to ensure issues are reproducible, documentation is complete, and severity is properly assessed.
-
Identify, isolate, and document defects with precise reproduction steps and utilize structured tracking mechanisms.
-
Provide detailed, actionable written feedback and annotations to support efficient resolution by developers without the need for clarification.
-
Collaborate with project teams to refine evaluation guidelines and ensure continuous improvement of testing standards and methodologies.
Preferred Qualifications
-
Proven professional experience as a QA Engineer, SDET, Test Engineer, QA Analyst, or similar role in software quality assurance.
-
Strong command of QA fundamentals, including test case design, bug tracking, regression testing, and both manual and automated testing strategies.
-
Hands-on experience with automation frameworks and test management tools such as Selenium, Playwright, Cypress, Appium, Postman, Jira, TestRail, Zephyr, or BrowserStack (or similar tools).
-
Prior paid human data experience supporting AI training — annotation, labeling, RLHF, AI response evaluation, model evaluation, or rubric-based grading — is a plus.
-
Excellent analytical and problem-solving skills with meticulous attention to detail.
-
Ability to communicate complex findings clearly in written English, providing specific, actionable feedback (B2 level or above).
-
No formal degree required—practical, demonstrable testing experience takes precedence.
-
Reliable internet connection and readiness to begin promptly.
What You’ll Do
-
Evaluate and rate AI-generated technical outputs against defined quality criteria and rubrics, identifying answers that may appear correct but are actually inaccurate or incomplete.
-
Design and assess comprehensive test cases for functional, regression, and edge-case scenarios, including negative and boundary tests.
-
Review bug reports and test documentation to ensure issues are reproducible, documentation is complete, and severity is properly assessed.
-
Identify, isolate, and document defects with precise reproduction steps and utilize structured tracking mechanisms.
-
Provide detailed, actionable written feedback and annotations to support efficient resolution by developers without the need for clarification.
-
Collaborate with project teams to refine evaluation guidelines and ensure continuous improvement of testing standards and methodologies.
Apply Now!