Senior Ai QA Engineer

Description:

Senior AI QA Engineer / Agentic AI QA Engineer | C2C🚨
🛂 Visa: ONLY H1B / H4 EAD / OPT EAD / L2 EAD
📍 Location: Chicago, IL (Hybrid – 3 days onsite per week)
💼 Experience: 10+ Years
📅 Duration: Long-term

We are looking for a highly experienced Senior AI QA professional with hands-on expertise in testing AI agents, LLM-powered applications, and AI-generated outputs.

🔹 Key Skills Required:
✅ Agentic AI / Generative AI / LLM Testing
✅ AI Evaluation and Quality Engineering
✅ Python and Pytest Automation
✅ SQL Validation and AI-Generated SQL Testing
✅ AI Regression Testing and Automated Test Frameworks
✅ MLflow / LangSmith / Weights and Biases
✅ Prompt Testing, Hallucination Detection and Output Validation
✅ Test Strategy, Risk Assessment and QA Leadership
✅ Web Application and API Testing
✅ Data Security, PHI and Multi-Tenant Isolation Testing

🎯 What Were Looking For:
10+ years of QA / Quality Engineering experience.
Strong hands-on experience evaluating AI agents and LLM-generated outputs.
Expertise in Python-based test automation and AI evaluation frameworks.
Ability to design test strategies, define quality metrics, and implement automated regression gates.
Experience identifying AI failure patterns and ensuring reliable, secure, and consistent AI behavior.