crewlia logo

Software Tester (Remote)

crewlia • New Zealand

remotecontract
Posted Oct 26, 2026

**Role & seniority: ** Software Tester (contract), likely non-manager/individual contributor

**Location & work type: ** Remote; candidates must currently reside in Australia, Canada, Ireland, New Zealand, or the United States (note: text also states US/Canada-only—US/Canada applicants likely required). Contract.

**Stack/tools: ** Not explicitly specified. Work involves multimodal data review/annotation across text, images, spreadsheets, and structured documents.

**Top 3 responsibilities: **

  • Annotate/categorize/validate large-scale multimodal datasets per strict project guidelines.

  • Audit AI outputs for hallucinations, formatting errors, and prompt deviations.

  • Identify edge cases (ambiguities/inconsistencies) to improve labeling taxonomies while meeting daily volume targets.

**Must-have skills: **

  • Strong reading comprehension, logical reasoning, and attention to subtle errors.

  • Native-level English and excellent written communication.

  • Ability to follow strict rubrics/guidelines and maintain QA quality at speed.

  • Reliable high-speed broadband and suitable quiet remote workspace.

**Nice-to-haves: **

  • Experience evaluating creative or technical work against structured rubrics.

  • Familiarity with AI dataset labeling and quality assurance workflows (implied by responsibilities).

Full Description

Role: Software Tester (Remote)

Location: Remote (Australia, Canada, Ireland, New Zealand, United States)

Employment Type: Contract

Location: Applicants must currently reside in Australia, Canada, Ireland, New Zealand or the United States.

Overview

  • We are looking for a detail-oriented Software Tester to review, annotate, and structure multimodal datasets for advanced artificial intelligence models. You will play a crucial role in improving data quality across text, images, spreadsheets, and complex structured documents.

Key Responsibilities

  • Annotate, categorize, and validate large-scale multimodal datasets according to strict project guidelines.
  • Audit AI system outputs to identify hallucinations, formatting errors, or deviations from user prompts.
  • Identify edge cases, labeling ambiguities, and data inconsistencies to help improve labeling taxonomies.
  • Consistently meet daily volume targets while upholding rigorous quality assurance standards.

Requirements

  • Superb reading comprehension, logical reasoning, and a sharp eye for spotting subtle errors.
  • Experience evaluating creative or technical work against strict rubrics or guidelines is strongly preferred.
  • Native-level fluency in English with exceptional written communication skills.
  • Consistent, high-speed broadband connection and a quiet remote working environment.
  • Geographically located within the US or Canada (applications from other regions will be rejected).

Equal Opportunity Employer

  • We are an equal opportunity employer. We are committed to building an inclusive workplace and welcome applications from all qualified individuals regardless of race, color, ethnicity, nationality, gender, gender identity or expression, sexual orientation, age, religion, disability, marital status, or any other characteristic protected by applicable law. All hiring decisions are made solely on the basis of qualifications, skills, and demonstrated ability.

Apply Now!

AI Output EvaluationAttention to DetailData CategorizationData ValidationDataset AnnotationEnglish FluencyError DetectionLogical ReasoningMultimodal Data ReviewQuality AssuranceReading ComprehensionRubric EvaluationTaxonomy ImprovementWritten Communication

Cookies & analytics consent

We serve candidates globally, so we only activate Google Tag Manager and other analytics after you opt in. This keeps us aligned with GDPR/UK DPA, ePrivacy, LGPD, and similar rules. Essential features still run without analytics cookies.

Read how we use data in our Privacy Policy and Terms of Service.