Cloud Engineer - QA ( Agentic Ai )

Role Summary
Ensure reliability, accuracy, and safety of AI/Agentic systems through structured evaluation and testing.

Key Responsibilities

• Design test cases for agent behavior, workflows, and edge cases
• Validate outputs for correctness, hallucination, bias, and compliance
• Build automated and manual evaluation pipelines
• Define quality metrics (accuracy, grounding, latency, safety)
• Perform regression testing for prompts, models, and workflows
• Collaborate with engineers to improve prompts and system responses

Must-Have Skills

• 3–6 years in QA/testing with exposure to AI systems
• Experience validating LLM outputs and prompt behavior
• Strong analytical and debugging skills
• Familiarity with evaluation techniques (human + automated)

Good to Have

• Health insurance domain (policy/claims validation scenarios)
• Experience with AI safety and governance