GenAI Testing Consultant/Architect
Infosys • Bengaluru, Karnataka, India
-
Role & seniority
- Cloud/ML/GenAI Solution Architect (hands-on architecture and delivery; senior level assumed from ownership/governance/roadmapping)
-
Stack / tools
-
Cloud: AWS, Azure, or GCP (scalable Cloud AI architectures)
-
LLM/GenAI patterns: RAG, tool/function calling, prompt engineering, automated evaluation
-
LLMOps/MLOps: experiment tracking, model registry, CI/CD, observability, drift detection, incident response
-
Data engineering: feature stores, batch/stream processing
-
Agentic/MCP: MCP and agentic solutions; workflow orchestration, multi-step reasoning
-
Languages: Python (or other AI/DS languages)
-
(Also referenced) Infosys and “industry-standard” tools/frameworks for evaluation/deployment
-
-
Top 3 responsibilities
-
Architect end-to-end Generative AI systems: pipelines, model selection, orchestration, evaluation, deployment
-
Implement LLMOps/MLOps capabilities: CI/CD, observability, drift detection, incident response, registry/experiments
-
Lead architecture governance and discovery workshops: reference architectures, success metrics, roadmaps for GenAI/ML use cases
-
-
Must-have skills
-
Hands-on LLM application development (RAG, tool calling, prompt engineering) and evaluation frameworks
-
Cloud AI design with security patterns and cost/performance optimization
-
Strong software engineering practices for ML/AI systems (maintainability, reliability, perf
-
Full Description
Experience architecting Cloud AI solutions, including scalable inference, data security patterns, and cost/performance optimization. Hands-on experience with LLM application patterns such as RAG, tool/function calling, prompt engineering, and automated evaluation frameworks.
Strong MLOps/LLMOps expertise: experiment tracking, model registry, CI/CD, observability, drift detection, and incident response for AI services. Experience with data engineering concepts (feature stores, batch/stream processing) to support ML and GenAI workloads. Proven ability to lead architecture governance, create reference architectures, and drive adoption across teams and portfolios. Lead discovery workshops to identify high-value GenAI and ML use cases, define success metrics (standard and custom), and create solution roadmaps. Architect end-to-end Generative AI systems, including data pipelines, model selection, orchestration, evaluation, and deployment patterns using Infosys and industry-standard tools and platforms. Design and implement ML/AI solutions using Python (or other relevant Data Science/AI languages), applying strong software engineering practices for maintainability, performance, and reliability. Build MCP and agentic-based solutions using appropriate frameworks, enabling tool/function calling, workflow orchestration, and multi-step reasoning patterns aligned to business needs. Test and validate GenAI applications using relevant tools and frameworks; capture, analyze, and report standard and custom quality metrics (e.g., relevance, groundedness, toxicity, latency, cost) and drive continuous improvement. Collaborate with cloud and platform teams to design scalable Cloud AI architectures and cost-efficient deployments across AWS, Azure, or GCP. Explore and evaluate new technologies, tools, and testing methodologies to improve development processes and solution quality; stay up-to-date with advancements in Generative AI and evaluation practices.
Good to have skills: GenAI, Python, Agile methodologies, CI/CD pipelines, API testing, SQL/Database basics Location: BANGALORE, PUNE, HYDERABAD, CHENNAI, TVM, INDORE, NAGPUR, MLR, MYS, NOIDA, BBSR, KOLKATA, COIMBATORE, HUBLI