Veritus Voice AI QA Analyst at Veritus to test and monitor AI voice agents for borrower communication. Ensure agent quality and performance before and after client launch.
Responsibilities
Conduct structured UAT on AI voice agents prior to client launch: place and receive test calls, probe edge cases and failure modes, and document defects with clear, reproducible steps
Test across scripted and adversarial scenarios (identity verification, payment negotiation, escalation/handoff triggers) to confirm the agent behaves as expected before go-live
Monitor the ongoing performance of live agents across client deployments, using both call-level QA and aggregate performance metrics
Listen to and grade live agent calls against Veritus's QA scorecard, scoring for accuracy, compliance, and tone
Analyze call data to identify trends, recurring failure modes, outliers, and opportunities to improve agent performance
Track performance metrics over time and investigate changes or anomalies in agent behavior
Partner with Customer Success to prepare quarterly client performance reports, combining quantitative analysis with insights from call reviews
Help translate call and performance data into clear, actionable recommendations for clients and internal teams
Flag systemic issues—including recurring failure patterns, scorecard drift, and emerging edge cases—to customer success and engineering teams
Maintain clear, consistent documentation of test results, call grades, analyses, and findings in our tracking tools
Qualification
2–4 years of QANice-to-have (but not required)
Required
2–4 years of QA, quality analyst, operations analyst, or similar experience, ideally in a consumer lending contact center or voice-based environment
Familiarity with US consumer lending laws and regulations such as FDCPA, TCPA, ECOA, and SCRA
Strong attention to detail and comfort following structured test plans and scorecards
Comfortable working with quantitative data and basic statistics to identify trends and assess performance
Experience working with spreadsheets or data analysis tools to analyze operational or customer interaction data
Ability to turn quantitative and qualitative findings into clear conclusions and recommendations
Clear written English for defect reports, call notes, and client-facing performance summaries
Comfortable giving and receiving detailed feedback on quality issues
Nice-to-have (but not required)
Experience testing AI/voicebot or IVR systems
Prior UAT or pre-release testing experience for a software product
Experience analyzing contact-center or call-performance data
Preferred
Experience testing AI/voicebot or IVR systems
Prior UAT or pre-release testing experience for a software product
