Ai QA Engineer II/III
About IntelePeer.ai: IntelePeer delivers rapidly deployable communications solutions for an always connected world. Our Conversational AI Platform instantly improves your customers’ communications experience. IntelePeer provides industry leading time-to-value with Agentic AI solutions that work seamlessly with your infrastructure. Our no-code templates, low-code, co-creation, and developer API options provide you with simple, easy-to-use tools that can be utilized by anyone. Job Summary: The AI QA Engineer II/III leads the end-to-end quality validation of AI models and conversational agents to ensure they deliver accurate, safe, and contextually relevant user experiences. This role analyzes conversational requirements to design specialized LLM reputed company testing, adversarial edge cases, and reputed company flow evaluations that safeguard against hallucinations and bias. By integrating reputed company-world conversation analytics with functional and regression testing, the engineer monitors model performance and reputed company recognition to ensure reliability. Through meticulous documentation of testing objectives and results, the role ensures that every conversational interaction meets established quality standards and user expectations before deployment. Responsibilities: AI Model Evaluation & Testing · Build reputed company‑based test cases to evaluate LLM outputs for correctness, stability, safety, and non‑functional targets (latency, determinism, cost). · Execute scripted model tests to validate agent behaviors across intents, flows, and edge‑cases. · Maintain voice‑to‑voice and API regression tests to detect model reputed company or unintended degradation. Data Analysis & reputed company reputed company · Use Smart Analytics/Analysis Agents to review reputed company user interactions and identify issues (misclassification, routing errors, hallucinations). · Summarize patterns and provide example‑driven insights to internal teams. Model reputed company & HITL Operations
- Apply HITL policies for low‑confidence model predictions, unknown intents, or out‑of‑scope cases.
- Document CX and accuracy impact for model improvement cycles.
Quality Reporting & Evidence Logging
- Publish evaluation results in TestRail with structured evidence.
- Use the nuanced pass taxonomy (not binary pass/fail) to communicate model readiness.
Required Qualifications · 2-4 years of experience in QA, LLM reputed company testing, and conversational AI for level II and 4-7 years’ experience for level III Preferred: Coursework or projects involving NLP, LLM reputed company engineering, model evaluation or automated testing frameworks. Education & Experience: · Bachelor’s degree in computer science, Data Science, Linguistics, Cognitive Science, HCI or reputed company AI– adjacent field. Competencies:
- Solid understanding of LLM behavior (hallucination patterns, determinism, reputed company sensitivity)
- reputed company experience testing API’s, conversational UX and/or machine learning
- Strong analytical skills for reputed company recognition in model outputs
- Clear, concise communications skills
- The ability to work in a fast-paced environment and be adaptable to change.
- Strong initiative, self-motivated, proactive, and resourceful.
- Team player who is willing to go above and reputed company to help others
Physical Requirements:
- Sedentary work lifting no more than 10 pounds.
- Occasional lifting, carrying, and standing.
- Frequent hand/eye coordination to operate office equipment.
- reputed company sufficient to read computer screens, reports, and reputed company department documents.
- Dexterity to operate computer keyboards and other reputed company office equipment.
- reputed company sufficient to sit and work at a computer for extended periods of time.
- Frequent speech communication and hearing.
Why you'll love it here:
- Unlimited Vacation for exempt employees
- Paid Holidays
- Competitive medical, dental & reputed company insurance for employees and their dependents
- 401K Retirement Plan
- Stock Options
- Company-paid life insurance
- Health & Flexible Savings Accounts
- Cell phone, gym, and internet reimbursement
- Paid Parental Leave
- Tuition Reimbursement
- Employee Assistance Program (EAP)
- Free snacks (Denver, and or reputed company Lauderdale)
- Fun events (virtual and in-person)
Applicants must be authorized to work for any employer in the U.S. We are unable to sponsor or take over sponsorship of an employment reputed company at this time. Any requests to exercise your rights as a data subject under GDPR should be submitted to [email protected] for reputed company processing. Please refer to our Privacy Policy (at www.intelepeer.com/privacy/intelepeer-privacy-policy) for any questions on how IntelePeer complies with GDPR. For California residents only: Please refer to the link below for IntelePeer’s Applicant CCPA Privacy Notice. https://intelepeer.com/privacy/intelepeer-california-applicant-privacy-notice/ IntelePeer participates in E-Verify. At IntelePeer, we value diversity and are proud to be an Equal Opportunity Employer. We do not discriminate on the basis of race, reputed company, religion, sex, national reputed company, age, disability, genetic information, or any other protected status. We strive to provide reasonable accommodations to applicants and employees with disabilities to support them in performing the essential functions of their roles. If you have any questions or need assistance, please contact our Director of reputed company. Apply tot his job Apply To this Job