🤖 15+ AI Agents working for you. Find jobs, score and update resumes, cover letter, interview questions, missing keywords, and lots more.
Cookies & analytics consent
We serve candidates globally, so we only activate Google Tag Manager and other analytics after you opt in. This keeps us aligned with GDPR/UK DPA, ePrivacy, LGPD, and similar rules. Essential features still run without analytics cookies.
Read how we use data in our Privacy Policy and Terms of Service.
🤖 15+ AI Agents working for you. Find jobs, score and update resumes, cover letter, interview questions, missing keywords, and lots more.
AgileEngine • Vitoria-Gasteiz, Autonomous Community of the Basque Country, Spain
Role & seniority: Manual QA Engineer (mid-level), 2+ years manual testing experience.
Stack/tools: TestRail (test management), Postman (API testing), browser dev tools; Jenkins CI/CD; JIRA; Agile ceremonies; familiarity with AI/ML concepts and LLM evaluation; OpenAI/Azure OpenAI/LangChain knowledge.
Manually test web applications and AI-driven features; design and execute detailed manual test cases; perform cross-browser UI testing.
Validate AI-generated outputs (test cases, summaries, recommendations) and verify AI behavior and acceptance metrics (precision, recall, hallucination rate).
Conduct manual API testing, report defects (JIRA), maintain test documentation, and contribute to CI/CD quality gates.
2+ years manual testing experience; strong test design and defect reporting.
Knowledge of SDLC/STLC, QA processes, and testing methodologies; API testing with Postman; UI testing.
Familiarity with AI/ML concepts, LLM evaluation (prompt engineering, output validation); OpenAI/Azure OpenAI basics; data protection/compliance basics (GDPR, PII).
Proficiency with TestRail, basic CI/CD understanding; upper-intermediate English.
Location & work type: Flexible schedule with remote and office options; USD-based compensation; Mobility across Fortune 500 and startup projects.
AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
WHY JOIN US If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!
ABOUT THE ROLE We are looking for a Manual QA Engineer to ensure the quality and reliability of web applications and AI-driven features in an Agile environment. This role is critical in validating user experiences, APIs, and LLM-powered functionality, safeguarding accuracy, compliance, and performance across releases. You’ll collaborate closely with engineering to strengthen testing practices, contribute to CI/CD quality gates, and elevate standards in both traditional and AI-focused QA processes.
WHAT YOU WILL DO
Perform manual testing of web applications to identify bugs, glitches, and inconsistencies;
Develop, maintain, and execute detailed manual test cases based on product requirements and specifications;
Perform thorough cross-browser UI testing to ensure consistent behavior across Chrome, Firefox, Safari, Edge, etc.;
Validate AI-generated test cases, summaries, and recommendations produced by integrated LLMs to ensure accuracy, completeness, and alignment with project requirements;
Design and execute test scenarios to verify AI behavior, including prompt–response consistency, contextual relevance, and deterministic vs. non-deterministic outputs;
Evaluate AI model quality using defined acceptance metrics (precision, recall, hallucination rate, duplication rate);
Conduct manual API testing to validate request/response accuracy, error handling, and integration;
Use TestRail to create, manage, and track test plans, test cases, and test execution results;
Use Jenkins CI/CD pipelines;
Report and document defects clearly using the bug tracking system (JIRA);
Participate in daily stand-ups, sprint planning, and other Agile team meetings;
Review user stories and acceptance criteria to understand the scope of testing;
Estimate own tasks, report on risks to the team lead;
Collaborate with developers and other QA team members to reproduce issues and verify fixes;
Maintain and update test documentation as needed.
MUST HAVES
2+ years of experience with manual testing;
Basic understanding of methodologies and SDLC and STLC (Software Development and Testing Life Cycle) models;
Knowledge of Testing and Quality Assurance Processes;
Strong experience with Test Design activities to cover a set of product requirements;
Knowledge of Test Design techniques;
Knowledge of testing methodologies;
Defect analysis skills and the ability to write detailed and useful defect reports with expected defect-fixing outcomes;
Experience with Postman (API testing) and Browser developer tools;
Experience with UI testing;
Understanding of LLM evaluation principles, including prompt engineering, output validation, and model fine-tuning cycles;
Familiarity with AI/ML concepts, APIs, and frameworks (e.g., OpenAI API, Azure OpenAI, LangChain);
Knowledge of data protection and compliance standards for AI systems (GDPR, zero-data retention, PII anonymization);
Experience with TestRail as a Test Management system (as a user, test execution is required);
Basic understanding of CI/CD process;
Upper-intermediate English level.
NICE TO HAVES
Experience with SSO testing;
Experience in Performance or Load.
PERKS AND BENEFITS
Professional growth: Mentorship, TechTalks, and personalized growth roadmaps.
Competitive compensation: USD-based pay with education, fitness, and team activity budgets.
Exciting projects: Modern solutions with Fortune 500 and top product companies.
Flextime: Flexible schedule with remote and office options.
Meet Our Recruitment Process