Principal Backend QA Engineer (AI)

Nexaminds is looking for a Senior Backend QA Engineer with strong API automation expertise and a focus on AI System Validation. In this role, you will be responsible for designing automated test frameworks, analyzing upstream service architectures, and validating non-deterministic systems (such as AI agents and chatbots). The ideal candidate excels at structuring high-quality test plans from complex requirements, utilizing modern automation frameworks, and leveraging AI techniques (like LLM as a Judge and scoring evaluations) to test applications beyond traditional deterministic methods.

Location: CANADA

Qualifications we are looking for:

API & Backend Automation: Strong hands-on experience in API test automation using Karate Framework and manual API testing with Postman.

AI & Non-Deterministic Testing: Understanding of testing non-deterministic systems, including response scoring techniques, AI agent/chatbot validation, or evaluation concepts like LLM as a Judge.

Technical Code Analysis: Ability to read and analyze source code to understand backend architectures, microservices, service dependencies, and upstream impacts.

Structured Test Planning: Proven skill in reviewing requirement documents to create concise, precise test plans and automation frameworks without relying on rigid language-specific constraints.

Local Environment Setup: Hands-on experience configuring local dev environments for early-stage (shift-left) test execution.

A proactive approach to learning emerging AI technologies and building dynamic testing solutions where standard deterministic QA falls short.

Clear, structured, and direct technical communication.

Nice to have:

Hands-on experience with the Agent Development Kit (ADK) or MCP applications.

Background in contract testing or microservices architecture validation.

Prior experience working in complex enterprise environments (e.g., retail or large-scale backend systems).

Job duties:

API Testing & Automation: Analyze OpenAPI/Swagger specs to design comprehensive test plans, execute manual testing via Postman, and build robust automated test suites using the Karate Framework.

AI System & Agent Validation: Test and evaluate non-deterministic AI outputs, internal AI agents (built with ADK), and MCP applications using evaluation metrics and scoring techniques.

Architecture & Upstream Analysis: Inspect source code and upstream services to identify critical dependencies, evaluate technical system impacts, and manage test data across pre-production and production environments.

Shift-Left & Local Testing: Configure and launch local development environments to conduct early-stage testing on backend APIs and microservices before deployment.

AI-Integrated Test Frameworks: Design scalable test automation strategies that dynamically integrate AI capabilities to validate scenarios standard testing tools cannot handle.