Software Engineer in Test

About Interactly.ai

Interactly.ai is transforming healthcare operations with AI-powered automation across scheduling, lab follow-ups, eligibility checks, prior authorization, and insurance verification. Our multimodal AI agents work across voice, email, chat, fax, and EHR integrations improving patient access, reducing staff burden, and accelerating care delivery.

We serve 2,000+ physicians and healthcare providers, delivering measurable outcomes, 20–30% fewer no-shows and thousands of staff hours saved. Now, we're scaling our AI platform and we want you to help lead that charge.

Role Overview

Architect scalable test automation and quality infrastructure for Interactly's real-time AI agents, healthcare workflows, and voice systems. Ideal for engineers who leverage AI tools daily and believe in AI-assisted software testing and quality engineering, with the experience to set technical direction for the team.

Key Responsibilities

• Framework Architecture: Design and build scalable, custom Python-based automation frameworks from scratch to handle end-to-end voice assistant workflows.
• AI/ML Validation Strategy: Establish methodologies for testing AI/LLM behaviors, including evaluating Word Error Rate (WER), semantic accuracy, hallucinations, and multi-turn conversation context retention.
• Performance & Scale: Lead the design of infrastructure to simulate thousands of concurrent voice sessions to test system latency, audio degradation, and infrastructure bottlenecks.
• Tooling & Innovation: Create internal tools or mocks to simulate microphone inputs, network jitter, and telephony interfaces (e.g., VoIP/SIP/WebRTC).
• Technical Leadership: Set the technical direction for the SDET team, establish code review standards, and mentor SDET I/II engineers.

Required Qualifications

• Experience: 6+ years of experience in software test architecture, development, or automation leadership.
• Expert Python: Expert-level Python skills with deep knowledge of concurrency, async programming (asyncio), and framework design.
• Voice/Telecom Domain: Deep understanding of audio streaming protocols, codecs, and WebSockets/gRPC.
• Cloud & Infrastructure: Strong experience with cloud platforms and container orchestration .
• CI/CD & DevOps: Proven track record of designing complex CI/CD workflows and managing automated test infrastructure at scale.

Why Join Interactly.ai?

• Build infrastructure and products powering real-world healthcare AI
• Work with AI-native engineering teams using cutting-edge tools (Claude Code, Cursor, Copilot)
• Mission-driven impact — improve patient outcomes and save thousands of staff hours
• High ownership, rapid learning, and significant career upside
• Global collaboration across India and the US
• Seed-stage growth — scale from Seed to Series A alongside the team

Interview Process

Recruiter Screen → Functional / Technical Round → System Design / Product Deep Dive → Founder Interview