Tror - AI for everyone logo

Test Automation Specialist

Tror - AI for everyone โ€ข United States

remotecontract
Posted Oct 23, 2026

**Role & seniority: ** Test Automation Specialist (senior; 9+ yrs testing, 6+ yrs automation framework design)

**Location & work type: ** Remote

**Stack/tools (primary): **

  • Playwright (TypeScript preferred)

  • TypeScript, Python, Java

  • AI/LLM & agents: AI agents, orchestration, prompt engineering, context management, tool calling, retrieval/grounding, LLM failure analysis, AI/LLM evaluation frameworks

  • CI/CD: Jenkins, Azure DevOps

  • Version control: Git, GitHub, Bitbucket

  • GitHub Copilot custom agents/instructions

  • Enterprise testing: SDLC/STLC, Agile

  • Experience with AI governance/security/compliance

**Top 3 responsibilities: **

  • Design/maintain Playwright automation frameworks and coding standards

  • Build UI/API/DB/E2E automated tests and integrate them into CI/CD

  • Develop AI agent/orchestration workflows and LLM evaluation/quality metrics

**Must-have skills: **

  • 9+ yrs software testing; 6+ yrs framework design

  • Playwright expertise (TS preferred)

  • AI Agents/LLM-based tooling development + strong evaluation/metrics

  • CI/CD integration (Jenkins/Azure DevOps)

  • Git + framework architecture/shared tool development

  • Strong technical communication/leadership/mentoring

  • Prompt engineering + context management + retrieval/grounding + tool calling + LLM failure analysis

**Nice-to-haves: **

  • Acentra experience

  • EvoBrix experience

  • Preference

Full Description

Test Automation Specialist

Remote

Job Summary

We are seeking a Test Automation Specialist to design, build, and support AI-driven test automation solutions. This role combines deep Playwright automation expertise with AI Agent development, LLM evaluation, and automation framework architecture to support enterprise-wide QA modernization initiatives.

Key Responsibilities

Design and maintain Playwright automation frameworks. Create UI, API, Database, and End-to-End automated tests. Build AI agents and orchestration workflows for automation assets. Define coding standards and framework best practices. Integrate automation into CI/CD pipelines. Establish AI evaluation frameworks and quality metrics. Package and distribute reusable automation solutions. Provide technical leadership and mentoring. Ensure AI governance, security, and compliance standards.

Mandatory Skills

9+ years of Software Testing experience 6+ years of Test Automation Framework Design AI Agents / LLM-based Tools Development GitHub Copilot Custom Agents & Instructions Playwright (TypeScript Preferred) TypeScript, Python, and Java Prompt Engineering & LLM Concepts Context Management Tool Calling Retrieval/Grounding LLM Failure Analysis AI/LLM Evaluation Frameworks CI/CD (Jenkins, Azure DevOps) Git, GitHub, Bitbucket Framework Architecture & Shared Tool Development SDLC, STLC & Agile Methodologies Strong Communication & Technical Leadership Provider Testing Background

Acentra experience

EvoBrix experience

Resources that have all 3 are preferred, but 2 out of 3 could work as well.

AI Agent DevelopmentCI/CDContext ManagementFramework ArchitectureGitHub Copilot Custom AgentsJavaLLM EvaluationPlaywrightPrompt EngineeringPythonRetrieval and GroundingSoftware TestingTechnical LeadershipTest Automation Framework DesignTool CallingTypeScript

Cookies & analytics consent

We serve candidates globally, so we only activate Google Tag Manager and other analytics after you opt in. This keeps us aligned with GDPR/UK DPA, ePrivacy, LGPD, and similar rules. Essential features still run without analytics cookies.

Read how we use data in our Privacy Policy and Terms of Service.