Bidgely, Inc logo

SDET 3

Bidgely, Inc โ€ข India

remotefull-time
Posted Oct 23, 2026
  • Role & seniority

    • Senior SDET (hands-on)

    • Own quality for customer-facing products and GenAI/AI agent features

  • Stack/tools

    • Languages: Python, Java

    • Testing/automation: Selenium, Playwright, REST API testing, Postman, TestNG, PyTest

    • AI agent testing/evals: LLM-as-judge, rubrics, LLM evaluation, RAG testing, AI agent testing

    • AI frameworks/observability: LangChain, LangGraph, LangSmith (or similar)

    • CI/CD & tooling: GitHub Actions, Jenkins, AWS, end-to-end regression, CI-integrated eval harnesses

  • Top 3 responsibilities

    • Define test strategy and set release quality criteria for dashboards/APIs/notifications and GenAI products

    • Build/maintain UI, API, and end-to-end regression automation integrated with CI/CD

    • Design and implement AI agent evaluation (eval datasets, automated eval suites, safety/accuracy/tool-use testing, LLM observability)

  • Must-have skills

    • 6โ€“9 years testing with hands-on SDET experience

    • Python + Java proficiency

    • UI automation with Selenium or Playwright

    • API automation + end-to-end framework experience with CI/CD

    • Ability to analyze requirements for test coverage gaps

    • Experience designing AI agent evals (rubrics, LLM-as-judge)

    • GenAI/agent testing experience (chatbots/IVR/agents)

  • Nice-to-haves

    • Prior experience building eval harnesses/tooling for AI agents

    • Experience with LangChain/LangGraph/LangSmith (or similar) beyond baseline

Full Description

About Bidgely Bidgely is a Silicon Valley AI company transforming how utilities and their customers understand and manage energy. The UtilityAI platform turns smart-meter data into appliance-level insights, and Agentic CX brings GenAI assistants to utility chat, voice IVR and customer service. Leading utilities across North America, Europe and Asia rely on Bidgely, and the Bengaluru engineering center is a certified Great Place to Work. The Role

We are hiring a senior SDET to own quality across customer-facing products: web dashboards, APIs, notifications, and GenAI products such as chatbots, voice IVR and AI agents. You will define how AI agents are tested, build automated evals, and set the quality criteria for release. You will also own test strategy for your area, and build and improve UI, API and end-to-end automation and regression coverage. This is a hands-on engineering role working closely with GenAI engineering, CX engineering and product teams. Key Responsibilities Own test strategy and release quality criteria for your product area. Build and maintain UI automation (Selenium or Playwright), API automation and end-to-end regression suites. Design evaluation strategy for AI agents using LLM-as-judge, rubrics and other eval techniques. Build eval datasets and automated evaluation suites in Python, integrated with CI/CD. Test AI agents for accuracy, consistency, tool usage, safety and data handling. Debug agent failures using LangSmith and similar AI observability tools. Analyse release-over-release quality and identify regressions. Review product requirements and identify gaps in requirements and test coverage. Review test code, set testing standards and mentor SDETs. Communicate findings and recommendations to engineering, product and leadership. Requirements 6โ€“9 years in software testing, including hands-on SDET experience. Good working knowledge of both Python and Java. Experience owning test strategy and improving automation and release quality. Experience with UI automation using Selenium or Playwright. Experience building API automation and end-to-end frameworks with CI/CD. Ability to analyse product requirements and identify testing gaps. Prior experience testing chatbots, IVRs and GenAI agents. Experience designing evals, rubrics and LLM-as-judge evaluations. Working knowledge of LangChain, LangGraph and LangSmith or similar tools. Experience building eval harnesses or tooling for AI agents. Strong analytical and communication skills. Tech You Will Work With Python, Java, Selenium, Playwright, REST API testing, Postman, TestNG, PyTest, LangChain, LangGraph, LangSmith, LLM evaluation (evals), LLM-as-judge, RAG testing, AI agent testing, end-to-end testing, regression testing, CI/CD, GitHub Actions, Jenkins, AWS.

API TestingAWSCI/CDEnd-to-end TestingGenAIJavaLangChainLangGraphLangSmithLLM EvaluationPlaywrightPostmanPythonRegression TestingSeleniumTest Automation

Cookies & analytics consent

We serve candidates globally, so we only activate Google Tag Manager and other analytics after you opt in. This keeps us aligned with GDPR/UK DPA, ePrivacy, LGPD, and similar rules. Essential features still run without analytics cookies.

Read how we use data in our Privacy Policy and Terms of Service.