Role & seniority
- AI Tester / AI Quality Engineer
- Senior: 5+ years QA/Automation testing; 2+ years AI/GenAI testing
Stack/tools
- AI eval & testing: RAGAS, DeepEval, LangSmith, custom eval frameworks
- Automation/testing: Python, test automation
- API & QA automation: Postman, Playwright
- Voice testing: ASR/TTS
- CI/CD integration for automated AI tests
Top 3 responsibilities
- Design and execute AI testing strategies/frameworks (functional + non-functional + model evals)
- Validate LLM outputs: grounding, hallucination control, citation accuracy; evaluate latency, relevance, faithfulness, toxicity, safety, drift
- Perform RAG + chatbot/voice testing (retrieval accuracy, context validation, multilingual conversation-flow validation)
Must-have skills
- AI/GenAI testing expertise (LLM/RAG/agentic/chatbot/voice)
- Hands-on with RAGAS/DeepEval/LangSmith and custom evals
- Automation skills in Python, plus API testing (Postman) and test automation (Playwright)
- Ability to test STT/TTS and run performance/security evaluations
- Prompt validation and eval result measurement/analysis
Nice-to-haves
- Extensive multilingual testing experience
- Deeper security testing experience specific to AI systems (beyond general security/performance)
Location & work type
- Not specified in the provided text.

Full Description

AI Tester (AI Quality Engineer) Role Summary Responsible for validating AI, GenAI, Agentic AI, Chatbot, and Voice AI solutions through functional, non-functional, and model evaluation testing. Key Responsibilities Design and execute AI testing strategies and frameworks. Validate LLM responses, grounding, hallucination control, and citation accuracy. Perform RAG testing, retrieval accuracy testing, and context validation. Conduct Evals using RAGAS, DeepEval, LangSmith, and custom evaluation frameworks. Perform chatbot testing, voice testing, STT/TTS testing, multilingual testing, and conversation-flow validation. Automate AI testing and integrate into CI/CD pipelines. Measure latency, relevance, faithfulness, toxicity, safety, and model drift. Required Skills AI/GenAI Testing RAGAS, DeepEval, LangSmith Functional & Automation Testing API Testing (Postman, Playwright) Voice Testing (ASR/TTS) Prompt Validation Performance & Security Testing Python, Test Automation Experience 5+ years QA/Automation Testing 2+ years AI/GenAI Testing