Test plans and coverage

Individual test cases tell you what to check; a test plan is the level above — what you will test for a given release or feature, what you will not, in what order, and how you will know you are done. It is how a QA turns "test the app" into an organised, communicable piece of work, and how the team agrees on what "tested" means. This lesson is what a test plan is (and is not), and the idea of coverage — knowing what you have and have not tested.

What a test plan is for

A test plan answers, for a piece of work (a release, a feature, a sprint), the practical questions:

What is in scope — which features/areas will be tested.
What is out of scope — what will not be tested, deliberately (and why) — this is as important as what is in, because it makes the risk explicit and agreed.
How — the approach: which levels and types (manual, API, mobile, some automation), what environments and devices.
What is needed — test data, environments, access, devices.
When you are done — the exit criteria (below).
The risks — what could make testing incomplete (a device you cannot get, a feature delivered late).

The point is communication and agreement, not ceremony. A test plan lets the QA, the developers and the product owner agree, before testing starts, on what will and will not be covered — so nobody is surprised at release time by "we never tested that". On a small team a "test plan" may be a short section in the ticket or a checklist, not a formal document — the thinking matters more than the format. Modern teams favour lightweight plans (a one-page test approach, a checklist) over heavy formal documents; match the weight to the project, but always do the thinking.

Scope and the out-of-scope decision

The most valuable part of a plan is often deciding — and stating — what you will not test. You cannot test everything (the risk-based lesson), so being explicit about the boundary is honest and protects everyone:

"In scope: the top-up flow on Android 10 to 14, on wifi and 3G, including failure/retry."
"Out of scope this release: iOS (no changes there), tablets, and load testing (a separate exercise)."

Stating the out-of-scope items before release means the decision to not test them is a team decision made with eyes open — not a gap discovered after a customer hits it. A QA who says "I did not test iOS, and here is why we agreed that was acceptable" is doing their job well; one who silently did not test iOS and lets everyone assume it was covered is not. Make the coverage boundary explicit and agreed.

Coverage: knowing what you have tested

Coverage is the measure of how much of something your tests exercise — and, more usefully for a manual QA, knowing what you have and have not tested. There are different kinds:

Requirements coverage — has every requirement/acceptance-criterion got at least one test? (The most useful for a QA — every stated behaviour is checked.)
Feature/flow coverage — has every screen and flow been exercised, including the alternative and error paths?
Code coverage — what percentage of the code the tests execute (a developer/automation metric, useful but easily misused — see below).

For manual testing, the practical goal is requirements and flow coverage: a simple grid or checklist mapping each requirement/flow to the test cases that cover it, so you can see at a glance what is covered and what is not. That grid is your answer to "are we done?" and "what is the risk?" — the blank cells are the untested areas.

The trap: coverage is not a guarantee

An essential caution, because coverage numbers are easily abused: high coverage does not mean high quality, and 100% coverage does not mean bug-free. You can execute every line of code (100% code coverage) with tests that assert nothing meaningful, or cover every requirement with shallow happy-path checks that miss the edge and error cases where bugs live. Coverage tells you what you touched, not whether you touched it well. So:

Treat coverage as a way to find gaps ("we have no tests for the failure path") — its genuine value — not as a score to maximise.
Never let "we have 90% coverage" stand in for "we have tested the risky things well". A money app with 90% coverage but no tests of the double-charge scenario is dangerously under-tested where it matters.
Depth on the risky areas beats breadth of shallow coverage everywhere.

Coverage is a map of what you have visited, useful for spotting where you have not been — not a proof that the places you visited are sound.

Exit criteria: how you know you are done

A test plan needs a definition of "done" — exit criteria — or testing goes on forever or stops arbitrarily. Sensible exit criteria are risk-based, not "zero bugs" (unrealistic):

All planned test cases for the in-scope features have been executed.
All critical and high-severity bugs are fixed (and re-tested); remaining bugs are low-severity and accepted by the team (the severity/priority lesson).
The risky areas (money flows, security) have been tested to the agreed depth.
Known out-of-scope items are documented and accepted.

"Done" is a risk decision the team makes together, informed by what QA found — not a number and not "perfect". A QA's job at exit is to state the true state clearly ("all critical flows pass; there are three low-severity cosmetic issues; iOS was out of scope") so the release decision is informed. That, again, is the whole point of the role: not to declare it perfect, but to make the release decision an honest one.

Check your work

What a test plan is for. To state, for a release/feature: what is in scope, what is out (and why), the approach, what is needed, when it is done, and the risks — for team agreement, not ceremony. Match the weight to the project (a checklist is fine); do the thinking.

Out-of-scope is key. Stating what you will not test makes the coverage boundary an explicit, agreed team decision — not a gap discovered after a customer hits it.

Coverage. How much your tests exercise — requirements coverage (every acceptance criterion tested, most useful), flow coverage (every screen/flow incl. error paths), code coverage (a dev metric). For manual QA, a grid mapping requirements/flows to cases shows what is and is not tested; blank cells are the risk.

Coverage is not a guarantee. High/100% coverage ≠ bug-free — you can cover shallowly and miss the edge/ error bugs. Use coverage to find gaps, not as a score; depth on risky areas beats shallow breadth.

Exit criteria. Risk-based "done": planned cases run, critical/high bugs fixed and re-tested, risky areas tested to depth, out-of-scope documented. "Done" is a team risk decision informed by QA's honest report of the true state — not "zero bugs".

Practice

Write a one-page test plan for a top-up feature release: in scope, out of scope (with reasons), approach, data/devices needed, exit criteria, risks.
Build a requirements-coverage grid: list the acceptance criteria and map each to the test case(s) that cover it; find the blank cells.
Decide and justify three things you would put out of scope for a given release, and how you would state them to the team.
Argue why a feature at "95% code coverage" might still be badly under-tested; give a concrete gap.
Write risk-based exit criteria for a money-handling release (what must be fixed vs what is acceptable to ship with).
Draft the two-sentence "true state" summary a QA would give at release time for a feature with some low-severity bugs and one out-of-scope platform.

Official documentation

ISTQB — Foundation syllabus (test planning, coverage, exit criteria) — The standard treatment.
Ministry of Testing — Test strategy and planning — Lightweight, practitioner approaches to planning.
Martin Fowler — TestCoverage — Why coverage is a gap-finder, not a target.

Next: exploratory testing, done with a charter.