When to automate, and when not to

Everything so far has been manual testing — a person deciding what to check, doing it, and judging the result. Now the course turns to automation: writing code that runs the tests for you. Before a single line of that code, though, comes the question that decides whether automation helps or hurts: when should you automate, and when should you not? Getting this wrong produces a large, slow, expensive test suite that finds little and that everyone eventually ignores — the single most common failure of test automation. This lesson is when to automate, and when to leave it manual.

What automation is, and is not, for

An automated test is code that performs a check the same way every time and tells you pass or fail. Its strength is repetition: once written, it runs a thousand times for almost no cost, on every code change, faster and more reliably than a human doing the same steps. That makes automation superb for one job:

Regression testing — confirming that things which used to work still work, after every change. This is tedious, endless, and exactly what a person is bad at (attention drifts on the hundredth run) and a machine is good at.

But automation is not a replacement for testing. It only checks what you told it to check — it will never notice the thing you did not anticipate, the layout that looks wrong, the flow that is confusing, the bug in a case you did not think of. It has no judgement. So automation does not replace the exploratory testing, the "does this feel right", the human noticing — it frees up time for them by taking the repetitive checks off your hands. A team that thinks "we automated the tests, so we do not need to test" has misunderstood what automation is.

The real cost of an automated test

Automation is often sold as free testing. It is not. Every automated test has a cost that is easy to underestimate:

Writing it costs time — more than doing the check once by hand.
Running it costs time and infrastructure (CI minutes, devices).
Maintaining it is the big one: every time the app changes, tests that touched the changed part break and must be updated. A large suite is a standing maintenance burden that grows with the app.
A flaky test (one that fails randomly without a real bug — the flaky-tests lesson) costs the most of all, because it destroys trust in the whole suite.

So a test earns its place only if the value of running it many times exceeds the cost of writing and maintaining it. That simple trade-off is the whole basis of deciding what to automate.

When automation pays off

Automate a check when the sums favour it — when it will run often, stay stable, and matter if it breaks:

It runs many times. A regression check on a core flow runs every release — high repetition, high payoff.
It is stable. The feature is settled, not changing weekly — so the test will not need constant rewriting.
It is important. A failure would be serious (login, payment, the money flow) — worth guarding on every change.
It is deterministic. The same input reliably gives the same output, so the test can assert a definite result.
It is tedious or error-prone by hand. Checking 50 input combinations, or a calculation across many cases — a machine does it faster and without slips.

The sweet spot is a stable, important, repetitive, deterministic check. That is where automation is clearly worth it.

When to leave it manual

Just as important — and more often ignored — is knowing when not to automate:

It is still changing. Automating a feature that is being redesigned weekly means rewriting the test weekly; wait until it settles.
It runs rarely. A one-off check, or something tested once a year, is not worth the code.
It needs human judgement. "Does this look right?", "is this confusing?", visual design, usability, exploratory testing — a machine cannot judge these; keep them human.
It is very hard to automate for little gain. Some flows (a real payment gateway, a hardware interaction, a CAPTCHA) cost enormous effort to automate reliably; the effort may not be worth it.
You do not yet understand it. Automate a check only once you know what "correct" is — automating a flow you have not first tested by hand bakes in your misunderstanding.

The honest position, which good teams hold and beginners resist: not everything should be automated, and a smaller suite of well-chosen, reliable tests beats a huge suite of fragile ones. Automation is a tool with a cost, aimed at the repetitive checks — not a goal in itself, and not a substitute for a thinking tester.

Check your work

What automation is for. Code that runs a check identically every time; its strength is cheap repetition, so it excels at regression (confirming what worked still works). It has no judgement — it checks only what you told it to, and never replaces exploratory testing or human noticing; it frees time for them.

The real cost. Writing (more than one manual run), running (CI/devices), maintaining (breaks whenever the app changes — the big one), and flaky tests (the worst). A test is worth it only if the value of many runs exceeds write + maintain cost.

Automate when it runs many times, is stable, is important, is deterministic, and is tedious/error-prone by hand — the stable/important/repetitive/deterministic sweet spot.

Leave manual when it is still changing, runs rarely, needs human judgement (look/feel, usability, exploratory), is very hard to automate for little gain, or you do not yet understand it. A smaller reliable suite beats a huge fragile one; automation is a costed tool, not a goal.

Practice

List five checks for an app you know and, for each, decide automate or manual — and justify it with the cost/repetition trade-off.
Take one feature and argue why automating it now would be a mistake, and what would have to be true to change that.
Explain, to someone who says "we automated the tests so QA is done", what automation does not do.
Give an example of a check that is tedious and error-prone by hand and so is an obvious automation win.
Estimate (roughly) the write + maintain cost of one test versus the cost of doing it manually each release, and say at how many releases automation breaks even.

Official documentation

Google Testing Blog — Just say no to more end-to-end tests — Why more automation is not always better.
Martin Fowler — Test automation — The case for, and the costs of, automated testing.
Ministry of Testing — Automation in testing — Practitioner perspective on what to automate.

Next: the test pyramid, and why most UI suites rot.