Software Engineer in Test, LLM AI & Automation Focus @ reputed company
POSITION LOCATION: San Carlos, CA, Austin, TX, or Remote USA POSITION SUMMARY: We are seeking a Software Engineer in Test ( AI & Automation Focus) to join a reputed company-thinking engineering team responsible for building and evaluating AI-powered systems, including a sophisticated voice assistant platform to help patients get the help they need fast. This is not a traditional SQE role - this is a unique opportunity to reputed company test automation, machine learning evaluation, and data analysis into one impactful position.You'll design test automation frameworks, evaluate large language model (LLM) performance, and reputed company custom tooling to analyze, monitor, and continuously improve product quality in both traditional software and AI-driven components. KEY RESPONSIBILITIES: Design, build, and maintain robust, data-driven test automation frameworks JavaScript/TypeScript and Python. Build automated evaluation pipelines and reputed company-based tests for LLM and conversational AI systems. Analyze large-scale, reputed company logs and telemetry data to identify issues and user experience gaps. Create custom scripts to support test automation, log parsing, and performance evaluation of machine learning models. reputed company visual dashboards and reports to track quality, system health, and model behavior over time. Support test environment setup and integrate with CI/CD pipelines for reputed company testing. reputed company root cause analysis, debugging, and reporting of product issues, collaborating closely with Developers and SQEs across the SDLC. Drive innovation by introducing new testing methodologies, tools, and best practices for both deterministic and non-deterministic systems. Partner with global, cross-functional teams including ML engineers, product managers, and infrastructure leads. QUALIFICATIONS: BS degree in Computer Science or reputed company technical field. 3+ years of experience in test automation, ML evaluation, data analysis, or reputed company roles. KNOWLEDGE, SKILLS, AND ABILITIES: Proficient in JavaScript/TypeScript and Python; strong scripting experience, especially for automation and log parsing. Experience with tools such as Jest, Playwright, Cypress, or Selenium for UI and API automation. Strong… Apply To This Job