Sumit Saurav

My Journey

15+ years in, and the thing that's never changed is this: I genuinely love the craft — the puzzle of catching what everyone else missed, the satisfaction of a framework that just works. What's changed is the toolkit. I started in manual and classic automation, grew into deep test-architecture ownership across a 12+ component platform at TELUS Digital, and now I'm building AI systems that test, evaluate, and reason about quality at a scale manual testing never could.

Right now I'm pushing in two directions at once: going deeper into AI Quality Engineering — LLM evaluation, agentic test generation, MCP-based tooling — while also strengthening fundamentals I used to lean on others for: performance engineering, SQL, Docker, CI/CD infrastructure. Real quality ownership means understanding the whole stack, not just the test layer. 70%+ of my time stays genuinely hands-on — architecting, building, and shipping, not just reviewing.

Still learning. Still shipping. Still genuinely enjoying it.

  • 🔭 Currently building: deeper performance + infra skills — k6, Docker, SQL, Appium next
  • 🌱 Always learning: every new tool gets built from scratch, line by line, before I trust it in production
  • 💬 Ask me about: AI-augmented testing, agentic workflows, or why your load test numbers might be lying to you
  • ⚡ Fun fact: ran the same k6 script locally, in Docker, and through BlazeMeter's cloud — just to prove the ~70ms real-network-latency gap most teams never actually measure

Shipped in Production

  • 4-Agent AI Orchestration System — Claude Code + MCP. Agent 1 generates structured test models from requirements, Agent 2 auto-generates Playwright/Cypress scripts, Agent 3 performs LLM-as-judge peer review — hallucination detection, prompt injection testing, AI safety and fairness validation, and coverage gap analysis before merge — Agent 4 triages CI failures via agent trajectory analysis, mapping failures to code changes and generating structured diagnostic reports. Cut manual regression effort ~65%, improved release velocity ~40%.
  • Reflect AI Kit — MCP-based agentic test automation (SmartBear Reflect + Claude Code). Generates working E2E tests from plain-English prompts in minutes, not hours.
  • QACypress — Real-time quality observability system (n8n + Cypress Cloud API + Datadog + Slack). Surfaces net-new regressions, maps failures to process deviations, delivers release-readiness dashboards with zero manual triage. Used daily by 12+ engineers.

Projects

RepositoryDescriptionStack
ai-engineering-chatbot-qa A hands-on LangChain.js chatbot for a fictional telecom provider — real tool-calling, multi-turn memory, and a dedicated AI-QA evaluation suite built with promptfoo TypeScript
supertest-api-agentictool-platform API automation with Supertest/Jest, a Playwright UI layer, staged CI/CD quality gates, and an agentic AI pipeline that generates test specs from a requirement — built from first principles against a real, live API TypeScript
performance-testing k6 load/stress/spike/soak testing framework — containerized with Docker, automated via GitHub Actions CI, validated against real cloud infrastructure through BlazeMeter JavaScript
playwright-ts-test-framework Playwright + TypeScript automation framework built from scratch — custom fixtures, full type-safe page objects TypeScript
test-forge BDD/Gherkin + REST API test automation framework (Playwright + Cucumber), with hands-on SQL data-validation practice TypeScript
fail-forward-slack-bot A Slack bot that parses commands and surfaces CI/test-failure signal directly in Slack TypeScript
ci-regression-diff-bot Compares two CI runs, surfaces only new regressions, filters out already-known failures JavaScript
ai-test-generation-kit Prompt-engineering kit generating browser regression tests from plain English, via an AI agent and a live-session test recorder (MCP-connected) JavaScript
cypress-ai-eval AI evaluation pipeline built from scratch — LLM-as-judge classifier, golden-dataset regression gate Python

Experience

Jan 2022 — Present
4 yrs 8 mos
QA Automation Lead Engineer TELUS Digital · Toronto, ON
  • Owned quality end-to-end across a 12+ component distributed microservices platform — designed and built the Playwright/TypeScript automation framework from scratch (POM architecture, shared utility libraries, fixture-driven data patterns, BDD-aligned tagging, environment-agnostic configuration); achieved 97% E2E coverage across frontend, integration, and regression test layers.
  • Built and shipped a 4-agent AI testing system (Claude Code + MCP) for automated test generation, LLM-as-judge peer review — hallucination detection, prompt injection testing, AI safety and fairness validation, coverage gap analysis — and intelligent CI failure triage via agent trajectory analysis, mapping failures to code changes and generating structured diagnostic reports — ~65% reduction in manual regression effort, ~40% improvement in release velocity.
  • Designed REST API and integration test suites across distributed microservice layers using Postman, PactumJS, and Playwright API testing (request context); contract testing with Pact; SQL backend data verification; established multi-tier CI/CD quality gates in GitHub Actions and Jenkins with staged rollout validation.
  • Mentored and upskilled QA engineers across multiple organisations on Playwright patterns, AI tooling adoption, and quality ownership culture; authored best-practice guides and process documentation.
Oct 2016 — Dec 2021
5 yrs 3 mos
QA Automation Lead (progressed from Test Architect) Accenture · delivering QA for TELUS · Toronto, ON
  • Built Selenium/Java and Karate BDD automation frameworks from scratch for a large-scale enterprise BSS/EPM financial platform (billing, rating engine, invoice generation, payment workflows) — validated end-to-end business process integrity with full SQL backend verification using JUnit/TestNG.
  • Designed REST and SOAP API test suites across financial integration layers; built JMeter performance testing infrastructure with threshold-based CI quality gates; coordinated onshore/offshore QA teams of 8+ engineers — defect triage, release readiness, stakeholder reporting via HP ALM/QC.
May 2012 — Oct 2016
4 yrs 6 mos
Associate Software Engineer / Test Architect Accenture
  • Functional, regression, integration, and performance testing across enterprise financial and telecom applications; validated complex distributed business workflows using HP QC/ALM, Selenium, SQL; built automation scripts across Agile and waterfall SDLC environments.

Technical Skills

  • Automation: Playwright, Cypress, Selenium, WebdriverIO, Node.js, BDD (Cucumber, Karate), Page Object Model, fixture-driven data patterns
  • Programming: TypeScript, JavaScript, Python, Java, SQL, Bash
  • AI & Agentic Testing: LangChain.js, LLM-as-Judge, Hallucination Detection, Prompt Injection Testing, AI Safety & Fairness Validation, Agent Trajectory Analysis, DeepEval, RAGAS, MCP, Claude Code, GitHub Copilot, Prompt Engineering
  • API & Integration Testing: REST API testing, Playwright API testing (request context), Postman, PactumJS, REST Assured, ReadyAPI, Contract testing (Pact)
  • Performance Testing: k6, JMeter, Docker, BlazeMeter, load/stress testing
  • CI/CD & DevOps: GitHub Actions, Jenkins, Docker, AWS, Git, n8n
  • Quality Observability & Governance: Datadog, Cypress Cloud API, Slack API, Quality KPI Definition, Defect Leakage Tracking
  • Test Management & Agile: JIRA, Confluence, Xray, Zephyr, Zephyr Scale, Agile/Scrum, Risk-Based Test Strategy

Education & Certifications

2008 — 2012
B.Tech, Electronics and Electrical Engineering KIIT University, India
Completed
ISTQB Advanced Level — Test Automation Engineer