Loading
Why do bugs still reach customers when every test is green?
Because a green build only answers the questions the team thought to ask - and customers always ask new ones. A test suite is a list of questions; bugs live in the questions that never made the list. Quality engineering widens the list: readable code that can actually be checked, human testing spent hunting surprises instead of re-clicking scripts, and automation that turns every escaped bug into a question asked forever. Green stays necessary. It was never sufficient.
The release had everything going for it. The feature was small, the team was careful, and the pipeline was a wall of green - every test passing, every check satisfied, every box ticked. It shipped on a quiet afternoon, and everyone moved on to the next thing.
The next morning, a customer found the bug. Not a subtle one. She edited an order after paying for it, the total did not update, and she was charged the old amount. It took her less than an hour of ordinary use to find what the entire pipeline had missed.
The retrospective asked the natural question: how did this get past the tests? But that is the wrong question, and the wrongness is the whole story. The bug did not get past the tests. It was never in front of them. No test had ever asked what happens when an order is edited after payment - so no test ever failed.
Strip away the tooling and a test is a very simple thing: a question, written down. Does the total include tax? Can a signed-out visitor see this page? Does deleting the last item empty the cart? A test suite is the list of questions a team has thought to ask so far, frozen in code so they can be asked again on every change.
Seen that way, a green build has a precise meaning - and a humbling one. Green means every question on the list got a good answer today. It says nothing about the questions that never made the list. The customer, on her first morning, asked one of those. Customers always do. They are not trying to break the product. They are simply using it in ways nobody wrote down.
A green build answers every question you asked. It says nothing about the questions you didn't.
Put the whole picture on one map and the release stops being a mystery.
Does the code actually behave? - It misbehaves → It behaves
Has anyone asked the question? - Never asked → Asked, and automated
Quality is the work of moving questions from left to right - before the customer does it for you.
The pipeline lives entirely in the asked column: it can celebrate the top and catch the bottom, and that is all it will ever see. The dangerous half of the map is the other column. The escape is where the customer found her bug. And the quadrant above it may be the most underrated risk in software: luck - code that behaves today by accident, unprotected, one refactor away from misbehaving with no alarm wired to notice. The customer does not test your code. She tests your imagination.
Quality work, seen on this map, is not about making green greener. It is the patient movement of questions from left to right - before the customer moves them for you.
Every bug is born the same way: at a keyboard, in a moment of ordinary human fallibility. What happens next is what separates teams. The bug either stops close to its birthplace, or it travels.
Each stop outward raises the price. Caught as it is typed, a bug costs a keystroke. Caught by the automated suite, minutes. Caught in code review, a conversation. Caught in a test environment, an afternoon and a redeploy. Caught by monitoring, an incident. And in a customer's hands it costs the one thing that never fully refunds: a little of her confidence that the product can be trusted with her work.
This is where code quality stops being a matter of taste. A bug travels farthest through code nobody can read. Convoluted code smuggles its bugs past the reviewer, who cannot see the intent, and past the test writer, who cannot tell what questions to ask of it. Readable code is not a style preference. It is the first quality gate - the one that decides whether any later gate can work.
Quality is the art of shortening the distance a bug can travel.
The word automation makes some testers nervous and some managers giddy, and both reactions miss what it is for. Automation does not replace the people who test. It splits the work with them - and the split is cleaner than most teams realize.
Both halves are quality. Only one can be automated.
A team that swaps them wastes both: machines cannot wonder, and people should not be replaying scripts.
Machines are unbeatable at yesterday's questions: the regressions, the contracts, the thousand small promises the product has already made. They ask identically, on every change, at any hour, without boredom and without mercy. What they cannot do is wonder. No machine ever asked "what if she edits the order after paying?" A human asked that - too late - and only then could a machine ask it forever.
People are at their best exactly where scripts end: the suspicious what-if, the odd sequence of steps, the empathy to notice that something technically correct is practically wrong. Which is why the saddest waste in software quality is a skilled tester spending the week re-clicking a script a machine should have inherited long ago - a machine's work done expensively by a person, at the cost of the person's real work, which no machine can do.
Put the pieces together and the test suite turns out to be something better than a gate or a chore. It is the team's memory: every lesson the team has ever learned about how this product can go wrong, written down as a question and asked again on every single change.
↺ returns to A surprise escapes
Each trip around the loop makes escapes rarer - and shorter.
This is what should happen to every bug that escapes: it ends its life as a question. Not just fixed - understood, distilled and automated, so the same lesson never has to be learned twice. Teams that do this compound. Every escape makes the suite wiser, every wiser suite frees more human attention, and the freed attention finds the next surprise sooner. A new engineer inherits years of hard-won questions on day one, without having lived through the incidents that taught them.
Teams that skip it also compound, in the other direction. Fix without understanding, promise to be more careful, move on - and the same kind of bug returns wearing a different coat, to a team with no memory of having met it before.
A test suite is a team's memory. Automation is what keeps it from forgetting.
None of this needs a transformation program. It needs three habits, practiced without exception.
When a bug escapes, finish the job. The fix is half the work. The other half is the question - what check would have caught this? - written as a test before the incident is closed. A retrospective that produces only promises has produced nothing but mood.
Spend human testing where machines cannot go. Exploration, odd paths, real-user empathy, the deliberate attempt to be surprised. If a person is executing the same steps for the third time, that is a machine's job being done by hand - automate it, and buy the person back for the work that needs a person.
Treat readability as a quality feature. Code review is a quality gate only when reviewers can see intent. Code that cannot be read cannot be checked - by anyone, human or machine.
Machines ask yesterday's questions perfectly. Humans exist to find tomorrow's.
Green is still necessary. A team without automation is a team whose lessons evaporate, whose people spend their days re-answering old questions, whose every release is a fresh act of hope. But green was never the definition of done. It is the floor.
The customer will always ask a question that is not on the list. That is not a failure of quality; it is what happens when software meets the real world. The failure is when the question stays off the list. Quality, in the end, is the speed with which the questions you missed become the questions you never miss again.
Carefully, if at all. Coverage tells you which lines ran during testing, not which questions were asked - a fully covered function can still fail the question nobody thought of. Coverage is useful as a torch: it shows the code no question has ever visited. It fails as a definition of done. The better habit is behavioral: every escaped bug becomes a test, and every human testing hour goes where machines cannot - those two rules raise real quality whether or not any number moves.
At its best, neither - it is a property of how the whole team works, with specialists who make it sharper. A skilled tester is a professional finder of unasked questions, not the person quality is outsourced to. The moment quality becomes somebody else's phase at the end, the distance a bug can travel grows - and the list of questions stops widening at exactly the point where the product meets reality.