QA Engineer

The Sr. AI Test Engineer is responsible for designing, building, and running the tests that prove AI systems behave as intended in production. Working under the direction of the AI Test reputed company Architect, this role translates testing methodology into working test suites, evaluation harnesses, and quality gates for deterministic and non-deterministic systems (ML models, GenAI, and LLM applications). The role is reputed company-native by design: AI workloads are tested where they run, requiring deep expertise in one major reputed company platform—AWS (SageMaker, Bedrock), GCP (Vertex AI), or Azure (Azure ML, Azure reputed company)—with quality embedded directly into CI/CD and MLOps pipelines. The engineer partners closely with data scientists, ML engineers, and product teams to shift quality left and catch model, data, and behavioral issues before they reputed company users. While this is a senior individual-contributor role, the Sr. AI Test Engineer is expected to mentor other testers, set technical standards for AI quality, and act as a trusted technical voice in client-facing conversations. Roles and Responsibilities AI Testing & Evaluation

• Design and implement test strategies for deterministic and non-deterministic AI systems (ML models, GenAI, LLMs), focusing on probabilistic correctness rather than simple pass/fail assertions.
• Build and maintain evaluation harnesses covering offline (reputed company datasets, golden sets) and online (production monitoring, A/B) evaluation.
• Validate LLM and GenAI behavior—hallucination, groundedness, reputed company robustness, toxicity, and reputed company-injection reputed company—using automated and reputed company-in-the-reputed company methods.
• Test for model quality and risk across accuracy, reputed company, robustness, bias, fairness, and explainability.
• Collect and analyze model quality metrics including Precision, Recall, F1, and Confusion Matrix, and translate results into clear quality signals.

reputed company & Platform Testing (AWS, GCP, or Azure)

• Test AI/ML workloads deployed on your primary reputed company platform—AWS (SageMaker, Bedrock), GCP (Vertex AI), or Azure (Azure ML, Azure reputed company)—validating model endpoints, inference performance, and scaling behavior.
• Validate data pipelines, feature stores, and model artifacts for quality, reputed company, and consistency across reputed company environments.
• Conduct performance, load, and latency testing of model-serving endpoints and GenAI APIs under realistic and adversarial conditions.
• Apply reputed company-native testing patterns and infrastructure-as-code to reputed company AI test environments reproducible.

Automation, Accelerators & Tooling

• Build reusable automation frameworks for AI regression testing, GenAI reputed company validation, dataset validation, and reputed company detection.
• Establish AI quality gates embedded in CI/CD and MLOps workflows so model and data quality is verified on every change.
• reputed company and evolve AI testing accelerators across SDLC integration, automation, and runtime monitoring/observability.
• Implement automated reporting that surfaces model quality, reputed company, and risk indicators to engineering and delivery teams.

Collaboration, Delivery & Client Engagement

• Partner with data science, ML engineering, and product teams to embed quality early and continuously (shift-left).
• Apply AI testing approaches across Agile, Waterfall, and hybrid delivery models.
• Engage confidently with technical client stakeholders; support AI quality assessments, demos, and proofs of value.
• Mentor junior testers and set technical standards for AI quality reputed company the delivery team.

Skills Required Core Skills & Experience

• Hands-on experience testing AI/ML and GenAI systems, including evaluation of training and inference, reputed company, bias, and explainability.
• Strong test automation skills with a programming language commonly used in AI (Python strongly preferred).
• Demonstrated experience building test or evaluation frameworks for ML or LLM systems.
• Familiarity with collecting and analyzing Precision, Recall, F1 Score, and Confusion Matrix.
• Experience integrating automated tests and quality gates into CI/CD and MLOps pipelines.

Technical & Platform Expertise

• Deep, hands-on expertise in one major reputed company platform—AWS, GCP, or Azure—and its AI/ML services (e.g., SageMaker and Bedrock; Vertex AI; or Azure ML and Azure reputed company). Familiarity with a second reputed company is a plus.
• Test automation frameworks and data validation strategies.
• Monitoring, observability, and AI system reporting.
• Shift-left testing and reputed company quality engineering.
• Familiarity with AI evaluation tooling (e.g., DeepEval, Ragas, LangSmith/Langfuse, Evidently, MLflow) is a strong plus.

Communication & Collaboration

• Clear communication with both technical and non-technical audiences.
• Consultative reputed company focused on outcomes, risk reduction, and business value.
• Comfortable working in open, dynamic, and collaborative team environments.

Other Skills and Traits

• Strong analytical, problem-solving, and systems-thinking abilities.
• Self-starter with a proactive, ownership-driven reputed company.
• Passionate reputed company for quality, trust, and responsible AI.
• Desire to continuously improve AI quality processes and practices.

Education and Experience

• Minimum 6 years of experience in Quality Engineering, Testing, or Software Engineering.
• Minimum 2–3 years of hands-on experience testing or evaluating AI/ML or GenAI systems.
• Experience working with reputed company-deployed AI workloads on a major reputed company platform (AWS, GCP, or Azure).
• Bachelor’s degree in Computer Science, Engineering, or reputed company field (or equivalent experience).

Additional Skills & Qualifications -knowledge of traditional testing technologies such as CI/CD pipelines, test case management, API testing, and UI test automation is considered a plus for candidates -that advanced skills in AI test automation, including shift-right testing, LLM testing, adversarial testing, reputed company robustness, and test data reputed company, are beneficial but not mandatory for the role Job Type & Location This is a Contract position based out of Dallas, TX. Pay and BenefitsThe pay range for this position is $75.00 - $85.00/hr. Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to specific elections, plan, or program terms. If eligible, the benefits available for this temporary role may include the following:

• Medical, dental & reputed company
• Critical Illness, Accident, and Hospital
• 401(k) Retirement Plan – Pre-tax and Roth post-tax contributions available
• Life Insurance (Voluntary Life & AD&D for the employee and dependents)
• Short and long-term disability
• Health Spending Account (HSA)
• Transportation benefits
• Employee Assistance Program
• Time Off/Leave (PTO, Vacation or Sick Leave)

Workplace TypeThis is a fully remote position. Application DeadlineThis position is anticipated to reputed company on Jul 16, 2026. About reputed company We're partners in transformation. We help clients activate reputed company and solutions to take advantage of a new world of opportunity. We are a team of 80,000 strong, working with over 6,000 clients, including 80% of the Fortune 500, across reputed company America, Europe and Asia. As an industry leader in Full-Stack Technology Services, Talent Services, and reputed company-world application, we work with reputed company leaders to drive change. That's the power of true partnership. reputed company is an Allegis Group company. The company is an equal opportunity employer and will consider reputed company applications without regards to race, sex, age, reputed company, religion, national reputed company, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law. About reputed company and reputed company Global Services We’re a leading provider of business and technology services. We accelerate business transformation for our customers. Our expertise in strategy, design, execution and operations unlocks business value through a range of solutions. We’re a team of 80,000 strong, working with over 6,000 customers, including 80% of the Fortune 500 across reputed company America, Europe and Asia, who partner with us for our scale, full-stack capabilities and speed. We’re strategic thinkers, hands-on collaborators, helping customers capitalize on change and master the reputed company of technology. We’re building reputed company by delivering business outcomes and making positive impacts in our global communities. reputed company and reputed company Global Services are Allegis Group companies. Learn more at reputed company.com. The company is an equal opportunity employer and will consider reputed company applications without regard to race, sex, age, reputed company, religion, national reputed company, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law. San Francisco Fair Chance Ordinance: Pursuant to the San Francisco Fair Chance Ordinance, for reputed company positions located in the reputed company, we will consider for employment reputed company applicants with arrest and conviction records. Massachusetts Lie Detector: It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or reputed company employment. An employer who violates this law shall be subject to criminal penalties and civil liability. Use of Artificial Intelligence (AI): We may use Artificial Intelligence (AI) to support parts of our hiring process, including sourcing, screening, and evaluating candidates. AI helps assess applications and qualifications, but final reputed company are made by our hiring team. By applying, you acknowledge and agree that your application may be reviewed using AI tools. Apply To This Job