Venxr logo

Senior AI Product Tester

Venxr • Pune, Maharashtra, India

onsitefull-time
Posted Oct 29, 2026Apply by Apr 27, 2027

**Role & seniority: ** Senior AI Product Tester (QA)

**Stack/tools: ** Claude Code; UI testing: Playwright/Cypress/Selenium; API testing: Postman/pytest/REST Assured; Languages: Python and/or JS/TS; LLM testing: LLM-as-judge with rubrics, structural/intent assertions; SQL; CI: GitHub Actions (implied); containers/tools: Docker (nice-to-have); Load: k6/Locust/JMeter (nice-to-haves).

**Top 3 responsibilities: **

  1. Own daily QA for AdsGPT and WhatsApp bot: write/validate test cases from acceptance criteria, run 30-min morning sanity checks, ensure issues are flagged pre-standup.

  2. Build and maintain CI-integrated QA automation with Claude Code; keep regression suite healthy (fix flakies, remove dead tests, close coverage gaps).

  3. Test AI/agent and data-action correctness: evaluation sets (golden prompts/edge cases), handle non-deterministic outputs, validate rupee-accurate numbers against Google Ads/Meta, and ensure bots/rules/bulk uploads behave safely with clear bug reports.

  • Must-have skills:

    • 3–5 years software QA; 2+ years building CI-integrated automation.

    • Hands-on experience using Claude Code (or similar coding agent) to build/maintain an automated test suite.

    • Proficiency in Playwright/Cypress/Selenium + Postman/pytest/REST Assured.

    • Ability to test non-deterministic LLM outputs (intent/structure assertions, LLM-as-judge, know when to use human validation).

    • Strong ability to write actionable test cases and bug reports (repro steps, e

Full Description

About The Opportunity

Venxr is hiring a Senior AI Product Tester to ensure zero broken AI products reach clients, especially those managing real ad budgets for major Indian brands like Haier, Orient Electric, and Suzuki. The role centers on QA automation powered by Claude, testing products that act on data — launching campaigns, moving budgets, and conversing with customers via WhatsApp.

Role & Responsibilities

Own quality across AdsGPT, and the WhatsApp bot — not as a gate at sprint end, but as a daily habit starting at 10am and embedded in every story shipped. Write test cases for every user story from acceptance criteria, and review them with the PM before development finishes. Run a 30-minute morning sanity check daily across all products — logins, key flows, ad-platform syncs, bot replies — and flag red issues before stand-up. Build and own QA automation using Claude Code for UI, API, and conversational flows, wired into CI to run on every deploy.

Log 5–10 actionable bugs daily: clear repro steps, expected vs actual, severity, and evidence — so developers can fix without follow-up. Build evaluation sets for AI features — golden prompts, edge cases, adversarial inputs — to measure if new prompts or models improve outcomes. Validate numbers against source systems — dashboards must match Google Ads and Meta down to the rupee.

Maintain the regression suite: fix flaky tests, delete useless ones, and identify coverage gaps. Test non-deterministic AI outputs using structural/intent assertions, LLM-as-judge with rubrics, and human validation when needed. Ensure bulk uploads (e.g., Excel-based ad campaigns) fail safely and report precise errors for malformed inputs. Verify automated rules (e.g., spend alerts) fire correctly and stay silent when they shouldn’t. Test WhatsApp bot conversations — Hinglish, one-word replies, angry customers, off-topic questions, prompt injection. Validate live, self-updating client reports to ensure data accuracy against ad platforms.

Skills & Qualifications

Must-Have

3–5 years in software QA, with at least 2 years building CI-integrated automation. Hands-on experience using Claude Code (or similar coding agent) to write, debug, and maintain test suites — must show a working suite built this way. Proficient with Playwright, Cypress, or Selenium for UI testing, and Postman, pytest, or REST Assured for API testing. Comfortable reading and testing code in Python or JavaScript/TypeScript. Skilled at testing non-deterministic outputs — assertions on structure/intent, LLM-as-judge with rubrics, and knowing when human judgment is needed. Ability to write clear, concise, and actionable bug reports and test cases — no padding, just specifics. Working knowledge of SQL to validate data directly in the database, not just UI.

Preferred

Experience testing chatbots or WhatsApp Business API flows. Exposure to Google Ads, Meta Ads, or ad-tech platforms. Familiarity with MCP servers (e.g., Playwright MCP), GitHub Actions, or Docker. Basic load testing experience with k6, Locust, or JMeter.

Benefits & Culture Highlights

Small, fast team — bugs you find get fixed in days, not quarters. Build QA automation from the ground up — you choose tools, structure, and what to automate first. Flat hierarchy — no layers between you and release decisions. Direct client exposure — see how brands use your work and learn the ad industry. Work at the edge of AI testing — help write the playbook as you go. Offsites, learning budget, and an open culture for asking questions.

Skills: typescript,rest assured,k6,claude code,python,selenium,sql,github actions,pytest,javascript,cypress,playwright

Claude CodeCypressDockerGitHub ActionsJavaScriptJMeterk6LocustPlaywrightPostmanPytestPythonREST AssuredSeleniumSQLTypeScript

Cookies & analytics consent

We serve candidates globally, so we only activate Google Tag Manager and other analytics after you opt in. This keeps us aligned with GDPR/UK DPA, ePrivacy, LGPD, and similar rules. Essential features still run without analytics cookies.

Read how we use data in our Privacy Policy and Terms of Service.