Back to Jobs

Senior Quality Assurance Analyst - Black Box Testing

Actively Reviewing the Applications

Searchnscore

India, Tamil Nadu, Coimbatore Contract On-site

Posted 3 weeks ago • Apply by May 18, 2026

Job Description

Joining: Immediate Joining Required.

Location: Coimbatore.
Contract Period: 1 Year.

Seeking a Senior Quality Assurance Analyst with strong technical acumen to test our conversational AI customer service agents. This role requires expertise in both black box testing (customer interaction simulation) and gray box testing (technical validation against APIs and code logic), combined with TypeScript proficiency for troubleshooting quality issues at the code level.

Youll work with a sophisticated AI agent platform, testing complex conversation flows, subscription management operations, and API integrations that power real-time customer service interactions.

Primary Responsibilities

Black Box Testing (Customer-Facing Interaction Simulation & Customer Persona Testing:
Simulate diverse customer scenarios through natural language interactions without visibility into internal system mechanics.
Create and execute test personas representing different customer types, emotional states, and communication styles (frustrated customers, price-sensitive customers, confused customers, satisfied customers considering cancellation, etc.).
Test conversation flows across multiple interaction patterns: information inquiries, account management requests, billing questions, product concerns, subscription modifications, and cancellation scenarios.
Validate that AI agent responses feel natural, helpful, and appropriate for the customer context.
Assess empathy, tone, and brand voice consistency across various conversation types.
Test edge cases including ambiguous requests, off-topic questions, emotional escalations, and multi-intent conversations.
Verify escalation triggers work appropriately (when agents should hand off to human User Journey Testing:
Execute complete customer service flows from initial greeting through issue resolution.
Test multi-turn conversations with context retention across messages.
Validate that agents properly gather required information before taking action.
Assess user experience quality including clarity of responses, appropriate question sequencing, and resolution effectiveness.
Test account verification flows with various customer identification methods.
Verify proper handling of authentication failures and error Testing:
Compare agent behavior against documented customer service standards.
Validate compliance with brand guidelines and communication policies.
Test for consistency across similar scenarios (same issue, different phrasing).
Identify gaps between expected customer experience and actual agent performance.
Gray Box Testing (Technical Validation & API Response Validation:
Validate agent outputs against actual API responses from backend systems.
Compare agent statements about customer data (order history, subscription details, account status) against actual API data.
Test error handling when APIs return failures, timeouts, or unexpected data Testing:
Test complete workflows from conversation action trigger through API call execution and response handling.
Test error propagation from API layer through to customer-facing Strategy Development & Execution:
Design test scenarios covering standard operations, edge cases, error conditions, and stress scenarios.
Maintain test documentation including test cases, execution results, defect reports, and coverage metrics.
Reproduce reported defects through systematic testing.
Analyze conversation logs and system traces to identify root causes.
Identify opportunities for test automation (where appropriate).
Write automated tests for API integration functions and data transformation utilities.
Strong experience in both black box (customer-facing) testing and gray box testing involving APIs and backend logic validation.
Hands-on experience testing conversational AI systems, including multi-turn conversations, context retention, tone, empathy, and escalation behavior.
Ability to simulate diverse customer personas and evaluate response quality, brand voice consistency, and overall user experience.
Experience validating AI responses against backend APIs, testing integrations, and handling error, timeout, and failure scenarios.

(ref:hirist.tech)