Role & seniority
- Senior Quality Engineering leader (end-to-end quality ownership; release sign-off authority)
- Not a passive QA/testing executor; accountable for production quality
Stack/tools
- Automation (Python); hybrid automation frameworks
- UI automation: Playwright / Selenium / Cypress
- API testing: pytest, requests, REST Assured; Postman/REST clients
- DB validation: SQL
- CI/CD: GitHub Actions, GitLab, Jenkins
- AI/LLM/agent testing: eval frameworks, agent critics, benchmark datasets/scoring rubrics
- Agent tooling: MCP servers, tool calling, orchestration; RAG pipelines; guardrails
- Cloud exposure: GCP / AWS / Azure (QA/automation context)
Top 3 responsibilities
- Own end-to-end quality across delivery lifecycle and act as release authority for go/no-go
- Define and execute automation-first quality strategy (UI/API/DB/agentic) integrated into CI/CD
- Lead AI agent & LLM quality engineering: validate workflows, tool calls, grounding (RAG), safety/guardrails, and run evals
Must-have skills
- 5+ years hands-on manual + exploratory testing; strong SDLC/STLC knowledge
- Proven end-to-end test planning, traceability, defect lifecycle management
- Strong API testing experience
- Experience building UI/API/DB automation frameworks and CI/CD test pipelines
- Hands-on experience testing LLMs/AI agents including eval frameworks and

Full Description

We are hiring a senior Quality Engineering leader who owns end-to-end quality, release sign-off, and automation strategy across our platform — including AI agents, web applications, APIs, and databases.

This role blends

Quality Ownership
Automation & Agentic Testing Leadership
Release & Go-Live Authority
Product-aligned execution (PO-style ownership)
You will start by establishing strong manual and exploratory testing foundations, then systematically drive automation-first quality, including AI agent validation, critic agents, eval frameworks, and safety guardrails.
This is not a passive QA role.
You are accountable for production quality.
Key Responsibilities
End-to-End Quality Ownership & Release Sign-Off
Own quality outcomes across the full delivery lifecycle
Act as release authority, driving go/no-go decisions
Define and enforce quality gates across features, agents, and releases
Partner with Product and Engineering to balance speed and quality
Reduce production defects through proactive automation and validation
Automation-First Quality Strategy
Define and own the automation roadmap (UI, API, DB, agentic systems)
Build and maintain hybrid automation frameworks using Python
Drive CI/CD-integrated automated testing pipelines
Ensure automation coverage continuously replaces repetitive manual testing
Guide aligned Quality Engineers in automation implementation
Agentic AI & LLM Quality Engineering
Own quality for AI agents, tool calls, and multi-step workflows

Validate

Agent behavior & reasoning paths
Tool invocation correctness
Hallucinations & unsafe outputs
RAG accuracy and grounding
Define and execute agent evaluation (evals) frameworks

Collaborate with Critic Engineers to design

Critic agents
Automated agent validation
Benchmark datasets & scoring rubrics
Lead innovation in agent testing methodologies
Manual, Exploratory & Scenario-Based Testing
Drive high-quality manual and exploratory testing where automation is not yet viable
Design test scenarios covering edge cases, failure modes, and real-world flows
Ensure strong test data strategies and defect lifecycle management
Use manual testing strategically — not as a long-term crutch
Product-Aligned Ownership
Work as a quality-focused Product Owner partner
Align closely with Product Managers on acceptance criteria and readiness
Participate in sprint planning, refinement, and release discussions
Ensure quality is embedded into feature design, not added later
Cross-Team Collaboration

Work closely with

Product
Engineering
AgenticOps
RAG & AI teams
Act as the single quality voice across delivery discussions
Mentor and guide aligned Quality Engineers and contributors
Required Skills & Experience
Quality & Manual Testing
5+ years of strong hands-on experience in manual and exploratory testing
Deep understanding of SDLC/STLC, test planning, traceability, and defect management
Strong API testing experience (Postman, REST clients)
Automation (UI, API, Database)

Strong experience in

UI automation (Playwright / Selenium / Cypress)
API automation (pytest, requests, REST Assured)
Database validation using SQL
Experience designing end-to-end automation frameworks
CI/CD integration (GitHub Actions, GitLab, Jenkins)
Agentic AI & LLM Testing
Hands-on experience testing LLMs and AI agents

Understanding of

Agent workflows
Tool calling
RAG pipelines
Guardrails & safety validation
Experience designing or executing eval frameworks
Python, MCP & Cloud
Strong Python skills for automation, utilities, and validation
Understanding of MCP servers, agent tools, and orchestration
Cloud exposure (GCP / AWS / Azure) in QA or automation contexts
What Success Looks Like
Releases go out predictably and confidently
Automation coverage grows sprint after sprint
Agent failures are caught before production
Quality becomes a shared responsibility, led by you
Manual testing effort reduces as automation and agent critics mature
Why This Role Is Special
Owns release sign-off
Drives automation-first quality
Shapes how agentic AI is tested
Works closely with Product & CTO
Leads innovation in AI testing
Not a test executor role
Not a checkbox QA position

Product-Aligned Agentic Quality Engineering Lead (Automation & Agentic AI)