🤖 15+ AI Agents working for you. Find jobs, score and update resumes, cover letter, interview questions, missing keywords, and lots more.
Cookies & analytics consent
We serve candidates globally, so we only activate Google Tag Manager and other analytics after you opt in. This keeps us aligned with GDPR/UK DPA, ePrivacy, LGPD, and similar rules. Essential features still run without analytics cookies.
Read how we use data in our Privacy Policy and Terms of Service.
🤖 15+ AI Agents working for you. Find jobs, score and update resumes, cover letter, interview questions, missing keywords, and lots more.
Seargin • Poland
Role & seniority
Stack / tools
Generative AI, RAG architecture (Vector Search + LLM)
Azure Cloud: Azure AI Studio / Azure OpenAI Service, Azure DevOps
Python (advanced: Pandas, PyTest)
QA & automation: LLM evaluation metrics, Prompt Flow
DataOps / CI/CD: pipelines triggering AI evals on PRs
Optional: LangChain, Semantic Kernel; LangGraph/AutoGen; Azure ML (bonus)
Top 3 responsibilities
Design, deploy, and validate generative AI solutions in Azure (including RAG + LLM integration)
Build and maintain automated evaluation pipelines for LLMs, including “LLM-as-a-Judge” vs deterministic assertions
Develop and manage CI/CD workflows for AI model/test orchestration, generate/maintain Golden Datasets for regression testing
Must-have skills
1–2 years Generative AI experience; hands-on RAG architecture
5+ years Azure experience; proven deployment/test in Azure AI Studio or Azure OpenAI
7+ years Python; expert scripting (Pandas, PyTest), not basic automation
8+ years QA/automation with LLM evaluation metrics
5+ years DataOps / CI/CD; Azure DevOps pipelines triggering AI evaluations
Nice-to-haves
Agentic AI / multi-agent systems (LangGraph, AutoGen)
Azure AI Search (Cognitive Search): indexing, vector profiles, hybrid search
LangChain or Semantic Kernel for test harnesses
Prompt Flow for evaluation pipelines
Golden Datasets for regression testing
Master’s in Dat
Requirements (Must Have) Generative AI
1–2 years of Generative AI experience.
Practical hands-on experience with RAG architecture (Vector Search + LLM).
Azure Cloud
5+ years of Azure experience.
Experience with Azure AI Studio (or Azure OpenAI Service) — must have deployed/tested models in Azure, not just locally.
Coding (Python)
7+ years of Python experience.
Python Specialist: Able to write complex scripts (Pandas, PyTest), not basic Selenium-type scripting.
QA & Automation
8+ years of QA experience.
Deep understanding of LLM evaluation metrics, including “LLM-as-a-Judge” vs deterministic assertions.
DataOps / CI/CD
5+ years of DataOps/CI/CD.
Azure DevOps pipelines that automatically trigger AI evaluations on pull requests.
Education
Technical degree (CS, Engineering, Math) or equivalent heavy experience.
Requirements (Should Have) Generative AI
Experience with Agentic AI (LangGraph, AutoGen) or multi-agent systems.
Azure Cloud
Experience with Azure AI Search (Cognitive Search), including deep knowledge of indexing, vector profiles, and hybrid search.
Coding (Python)
Experience with LangChain or Semantic Kernel for building test harnesses.
QA & Automation
Experience using Prompt Flow to build evaluation pipelines.
DataOps / CI/CD
Experience creating Golden Datasets (ground truth) for regression testing.
Education
Master’s degree in Data Science or an AI-related field.
Nice to Have (Bonus) Generative AI
Experience fine-tuning Small Language Models (SLMs) like Phi-3 or Llama.
Azure Cloud
Experience with Azure ML (AML), Workspaces, and Compute Instance management.
Coding (Python)
Background in C# / .NET to integrate with the client’s backend.
QA & Automation
Experience with Red Teaming (security, jailbreak, adversarial testing).
DataOps / CI/CD
Experience with MLflow for experiment tracking.
Education
Certification: Azure AI Engineer Associate (AI-102).