🤖 15+ AI Agents working for you. Find jobs, score and update resumes, cover letter, interview questions, missing keywords, and lots more.
Cookies & analytics consent
We serve candidates globally, so we only activate Google Tag Manager and other analytics after you opt in. This keeps us aligned with GDPR/UK DPA, ePrivacy, LGPD, and similar rules. Essential features still run without analytics cookies.
Read how we use data in our Privacy Policy and Terms of Service.
🤖 15+ AI Agents working for you. Find jobs, score and update resumes, cover letter, interview questions, missing keywords, and lots more.
Three Across • India
Role & seniority: QA-AI/GenAI Testing Specialist (5–8 years experience), full-time
Location & work type: Pune, India; on-site or hybrid options implied by role; Vollzeit/Full-time
Languages & frameworks: Python for test automation; AI/GenAI testing approaches
GenAI stack: LLMs, LangChain, LangGraph; RAG pipelines; agentic AI systems
ML eval: LLM evaluation frameworks (RAGAS, DeepEval); vector databases
Cloud / infra: AWS (Bedrock, Lambda, CloudWatch)
QA/DevOps: CI/CD, shift-left, Agile, DevOps practices
Test tooling: Jira, Xray, ALM, TestRail, Git; basic Docker/Kubernetes (nice-to-have)
Others: performance/load testing, governance, ethics/bias detection
Design and execute comprehensive testing strategies for AI/GenAI applications; validate ML models, LLM outputs, NLP/NLU, and RAG workflows
Build and maintain scalable Python-based test automation frameworks; perform root cause analysis and drive defect resolution with Dev/PM teams
Establish quality gates and governance for AI systems; integrate testing into CI/CD and promote shift-left quality practices
Strong understanding of LLMs, LangChain/LangGraph, GenAI architectures
Hands-on testing of GenAI and agentic AI in production; Python-based framework design
Experience with LLM evaluation (RAGAS, DeepEval); vector databases and RAG testing
AI ethics, bias detection, responsible AI testing; AWS Bedrock/ Lam
Role: QA-AI/ GenAI Testing Specialist
Experience: 5-8 Years
Location: Pune
Overview
We are looking for an experienced QA professional with deep exposure to AI and Generative AI systems to design and execute advanced testing strategies across intelligent, cloud-native platforms.
This role goes beyond traditional QA. You will work at the intersection of AI quality, automation, cloud, and DevOps, helping ensure that GenAI-driven applications are reliable, ethical, scalable, and production-ready.
Key Responsibilities
Design and execute comprehensive testing strategies for AI and GenAI-based applications Build and maintain scalable Python-based test automation frameworks Validate machine learning models, LLM outputs, and GenAI workflows across functional, performance, and quality dimensions Define and execute testing approaches for NLP, NLU, RAG pipelines, and agentic AI systems Perform root cause analysis and collaborate closely with developers and product teams on defect resolution Establish quality gates, governance frameworks, and best practices for AI systems Integrate testing into CI/CD pipelines and promote shift-left quality practices Contribute to technical communities and drive continuous improvement in QA methodologies
Core Skills & Experience
Strong understanding of LLMs, LangChain, LangGraph, and GenAI architectures Hands-on experience testing Generative AI and Agentic AI systems in production Expertise in Python for test automation and framework design Experience with LLM evaluation frameworks (RAGAS, DeepEval, etc.) Knowledge of vector databases and RAG testing strategies Performance and load testing experience for AI-driven applications Strong understanding of AI ethics, bias detection, and responsible AI testing Solid AWS experience, especially Bedrock, Lambda, CloudWatch Deep knowledge of Agile, CI/CD, DevOps, and modern SDLC practices Proficiency with Jira, Xray, ALM, TestRail, Git, and branching strategies
Highly Valued
Experience with Docker and Kubernetes for test environments Background in data engineering or data quality testing Exposure to UI, API, and mobile testing methodologies Contributions to open-source projects, publications, or tech communities This role offers the opportunity to define how quality is measured in the age of AI, influence enterprise-wide testing standards, and work on some of the most advanced technology stacks in the industry.
In an AI-driven world, quality isn’t just about finding defects; it’s about earning trust in machines that think.
Karrierestufe Management Beschäftigungsverhältnis Vollzeit Tätigkeitsbereich Qualitätssicherung Branchen Investment Banking