Role & seniority: Data Quality Engineer (Data QE), 6+ years experience, full-time

Stack / tools: Python, Pytest, API testing, data ingestion testing, SQL; Databricks (notebooks, jobs, clusters); CI/CD integration; Databricks Genie (nice-to-have)

Top 3 responsibilities

Validate end-to-end data quality for ETL/ELT pipelines (transformation logic, schema validation, data reconciliation)
Conduct data ingestion testing across APIs, databases, files, and streaming sources; validate API responses and downstream integration
Design/develop automation test suites for data validations and regression; integrate frameworks into CI/CD; build reusable DQ scripts

Must-have skills

6+ years in Data Quality Engineering or Data Testing
Strong data ingestion testing, API testing, and DQ validation experience
Hands-on Python, Pytest, and automation framework proficiency; SQL for complex validation
Familiarity with Databricks environment (notebooks, jobs, execution)

Nice-to-haves

Experience with Databricks Genie or AI-driven productivity tools for code enhancement
Location & work type: All Persistent locations; hybrid work allowed; full-time employment

Full Description

About Position

We are looking for a highly skilled Data Quality Engineer (Data QE) with strong experience in validating data pipelines, automation testing, and ensuring end‑to‑end data reliability across ETL/ELT processes. The ideal candidate will have hands-on experience in Python-based test automation, data ingestion testing, and exposure to cloud data platforms such as Databricks.

Role: Data Quality Engineer

Location: All Persistent Location

Experience: 6+ Years

Job Type: Full Time Employment

What You'll Do

Perform comprehensive Data Quality (DQ) validations on ETL/ELT pipelines, including transformation logic, schema validation, and data reconciliation. Execute data ingestion testing across multiple sources such as APIs, databases, files, and streaming systems. Validate data flows on Databricks-based ETL frameworks leveraging notebooks, jobs, and cluster executions. Design and develop automation test suites using Python and Pytest for data validations, DQ rule enforcement, and regression testing. Integrate automation frameworks into CI/CD pipelines. Build reusable DQ scripts and utilities to ensure data consistency and stability throughout the pipeline. Perform API testing for ingestion and consumption layers using data-focused validation frameworks. Validate API responses, data structures, and end-to-end integration with downstream systems. Work with Databricks notebooks, data workflows, and jobs for validating ETL pipelines. Familiarity with Databricks Genie or similar AI-driven productivity tools for code enhancement and debugging. Collaborate with data engineering teams to troubleshoot and optimize data workflows.

Expertise You'll Bring

6+ years of experience in Data Quality Engineering or Data Testing roles. Strong experience in data ingestion testing, API testing, and DQ validation. Hands-on expertise in Python, Pytest, and automation frameworks. Working knowledge of Databricks environment, notebooks, and job execution. Proficiency in SQL for complex data validation and reconciliation.

Benefits

Competitive salary and benefits package Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications Opportunity to work with cutting-edge technologies Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards Annual health check-ups

Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.