P

QA Engineer � AI Analytics / Databricks Genie

PRI INDIA IT SERVICES PRIVATE LIMITED • Bengaluru, Karnataka, India

onsitefull-time
Posted Nov 11, 2026Apply by Dec 11, 2026

**Role & Seniority: ** QA Engineer (AI/data analytics QA), ~3–6 years experience

**Stack/Tools: ** SQL; API testing (Postman); Automation with Python + PyTest (or similar); Databricks / Databricks Genie (preferred); LLM/GenAI concepts; GenAI/AI testing focus (accuracy/groundedness/hallucinations/relevance)

**Top 3 Responsibilities: **

  • Test AI agents/analytical solutions: validate natural-language questions and Genie-generated answers vs source data and SQL results.

  • Cover KPI/aggregation/trend/comparison/filter/drill-down scenarios, including ambiguous queries, edge cases, and multi-turn conversations.

  • Perform functional/regression/API/integration/data validation; build reusable test cases and automated regression suites; collaborate on acceptance criteria and defect resolution.

  • Must-have Skills:

    • Strong SQL and data validation for analytical correctness.

    • Experience testing analytical/data-driven applications and automation using Python/PyTest (or equivalent).

    • API testing experience (e.g., Postman).

    • Understanding of analytical datasets, KPIs, aggregations, and data pipelines.

    • Familiarity with AI testing concepts: accuracy, groundedness, hallucination, relevance, response consistency.

  • Nice-to-haves:

    • Azure Databricks / PySpark knowledge.

    • Testing conversational BI or natural-language-to-SQL systems.

    • Automation to compare Agent responses against expected SQL/data outputs.

  • Location & Work Type: Not specified in the

Full Description

Role Overview

We are looking for a QA Engineer to validate AI-powered analytical solutions where Agents use Databricks Genie to interpret natural-language questions and generate data-driven insights. The role will focus on validating data accuracy, response quality, business logic, and end-to-end reliability.

Key Responsibilities

Test AI Agents and analytical solutions powered by Databricks Genie. Validate natural-language questions and Genie-generated analytical responses against source data and SQL results. Test analytical scenarios including KPIs, aggregations, trends, comparisons, filters, and drill-downs. Validate responses across different question phrasings, ambiguous queries, edge cases, and multi-turn conversations. Identify incorrect calculations, inconsistent responses, hallucinations, and incorrect interpretation of business terminology. Perform functional, regression, API, integration, and data validation testing. Build reusable test cases and automated regression suites for analytical queries. Collaborate with Data Engineers, AI Engineers, business SMEs, and Product Owners on acceptance criteria and defect resolution.

Required Skills

3-6 years of QA/testing experience, preferably in data, analytics, BI, or AI solutions. Strong SQL and data validation skills. Experience testing analytical or data-driven applications. Experience with API testing using tools such as Postman. Automation experience using Python/PyTest or similar frameworks. Good understanding of data pipelines, KPIs, aggregations, and analytical datasets. Exposure to Databricks, Databricks Genie, LLMs, or GenAI testing is highly preferred. Understanding of AI testing concepts such as accuracy, groundedness, hallucination, relevance, and response consistency.

Good to Have

Azure Databricks / PySpark knowledge. Experience testing conversational BI or natural-language-to-SQL solutions. Experience automating comparison of Agent responses against expected SQL/data outputs.

AI TestingAnalytical TestingAPI TestingData PipelinesData ValidationDatabricksDatabricks GenieKPI ValidationNatural-Language-to-SQL TestingPostmanPySparkPyTestPythonQuality AssuranceRegression TestingSQL

Cookies & analytics consent

We serve candidates globally, so we only activate Google Tag Manager and other analytics after you opt in. This keeps us aligned with GDPR/UK DPA, ePrivacy, LGPD, and similar rules. Essential features still run without analytics cookies.

Read how we use data in our Privacy Policy and Terms of Service.