Full Description

Position: Bilingual Japanese Generalist Evaluator Expert

Type: Hourly contract

Compensation: $25-$30 per hour

Location: Remote

Commitment: At least 20 hours per week

Role Responsibilities

Create detailed prompts in Japanese and/or English with multiple constraints and instructions, ensuring natural phrasing and real-world relevance for Japanese-speaking users Establish expectations for correct responses in Japanese consumer contexts and develop comprehensive rubrics accounting for linguistic nuance, tone, and cultural conventions Run prompts through models and assess outputs for accuracy, fluency, and cultural fit in Japanese, comparing results against English where needed Collaborate in QA review processes to ensure prompt tasks and rubrics meet rigor and maintain consistency across Japanese-language benchmarks Author Japanese and English prompt–golden answer pairs to train and evaluate advanced language models

Requirements

Native-level fluency in Japanese (written) with strong reading and writing ability in English BS or BA from a reputable institution, completed or in progress Strong writing and critical thinking skills Ability to work independently and meet deadlines Familiarity with ChatGPT or similar tools Based in Japan or able to produce culturally accurate Japanese Experience in teaching, research, editing, or academic writing preferred Experience creating evaluation criteria, rubrics, or grading guidelines preferred Familiarity with LLMs, prompting, or model evaluation preferred

Application Process (Takes 20 Mins)

Upload resume Interview (15 min) Submit form