Context driven testing

Lessons Learned in Software Testing (Kaner, Bach & Pettichord)

This skill installs a way of thinking about testing, not a procedure. Where the rest of the repo tells you how to build a good test, this tells you how to be a good tester: testing is a skilled, investigative, thinking activity — you're learning about a product to inform the people who decide its fate, under real constraints of time and uncertainty. The book's stance is the context-driven school: judgment and skill over rote process, heuristics over "best practices."

Reach for this for the human, investigative side of quality: planning how to test a specific situation, doing exploratory testing, reasoning about what could go wrong, deciding what's worth automating, and writing bug reports that change minds. It is the mindset behind Quadrant 3 of agile-testing-quadrants.

This skill is advisory; it produces strategies, charters, and bug reports rather than unit-test code.

---

The seven basic principles

The context-driven school is defined by seven principles (use them as a checklist when someone proposes a "best practice"):

1. The value of any practice depends on its context.
2. There are good practices in context, but there are no best practices.
3. People, working together, are the most important part of any project's context.
4. Projects unfold over time in ways that are often not predictable.
5. The product is a solution. If the problem isn't solved, the product doesn't work.
6. Good software testing is a challenging intellectual process.
7. Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.

The practical upshot: never recommend a testing practice in the abstract. Ask who, when, where, why, and what-if first — the same technique that's essential for an avionics system may be wasteful for a throwaway web app. Detail and illustrations in references/context-driven-principles.md.

---

Testing is investigation, not verification

The central mental shift: testing isn't mechanically confirming the program does what the spec says (that's checking). Testing is investigating to discover what's actually true about the product — including the things no one thought to specify. A tester is an information service: the job is to reduce stakeholders' uncertainty so they can make good decisions (ship / fix / cut), not to be a gatekeeper who "owns" quality. Different missions (qualify it, debug it, sell it, explore it) call for entirely different testing.

This is why the value of a test lies in the information it provides — a test that can only ever pass tells you little; a test aimed at a real risk, that could plausibly fail, tells you a lot.

---

Exploratory testing

Exploratory testing is simultaneous learning, test design, and test execution: you design the next test based on what the last one just taught you, rather than following a script written in advance. It's not "random clicking" — it's disciplined, often organized around a charter (a mission for a time-boxed session: "explore the import feature with malformed files to discover data-loss risks"). It excels exactly where scripted tests are weak: finding the surprises, the unspecified behavior, the "I didn't think of that." Scripted tests and exploration are complementary — scripts confirm known expectations cheaply and repeatably; exploration finds the unknown. Detail (sessions, charters, bug hunting) in references/exploratory-testing-and-heuristics.md.

---

Oracles and heuristics

Two ideas you'll use constantly:

• An oracle is the means by which you decide whether a behavior is a problem — a reference, a comparable product, a spec, a user's reaction, your own judgment. The load-bearing lesson: all oracles are fallible. A program can "pass" your test and still be broken in a way you weren't watching. So hold your pass/fail judgments provisionally, and use multiple oracles (consistency heuristics: is the product consistent with comparable products, with its own past behavior, with its claims, with reasonable user expectations?).
• A heuristic is a rule of thumb — a fallible-but-useful way to generate test ideas fast. Because the space of possible tests is infinite, good testers collect heuristics to make better guesses about which few tests will be worthwhile. The crucial caveat: "there is no wisdom in heuristics — the wisdom is with you." A heuristic only makes a suggestion; understand why it works and when it doesn't, rather than following it blindly. (Every heuristic is also a bias.)

---

Bug advocacy: report to get fixed

Finding a bug is half the job; getting it fixed is the other half. A bug report is an advocacy document — effectively a sales tool arguing that the bug is worth scarce time and money to fix. Your responsibility isn't that every bug gets fixed; it's to report accurately and vividly enough that the reader grasps the full impact and can decide well. Practical moves (full detail in references/bug-advocacy-and-reporting.md):

• Make the impact clear — how it interferes with real use, what data it corrupts, how often users will hit it. State the benefit of fixing so the reader wants to.
• Do follow-up testing — a minor-looking bug often hides a more severe consequence; probe it before reporting so you report the worst, most motivating version.
• Anticipate objections — pre-empt "works on my machine," "edge case," "won't fix."
• Mind your tone — reports shape your credibility; blaming or sloppy reports get you (and your future reports) ignored. You are what you write.

---

Healthy skepticism

The context-driven stance breeds productive skepticism — apply it when steering testing:

• Metrics: invalid metrics are dangerous. Counting test cases, or chasing a coverage percentage, measures activity, not quality, and distorts behavior once it becomes a target. (Same "guide, not target" line as effective-software-testing.)
• "Best practices": there aren't any in the abstract (principle 2). Treat any one-size-fits-all prescription — including the other skills in this repo — as a default to adapt, not a law.
• Automation: automated checking ≠ automated human testing. Be skeptical of "100% automation" claims; automation runs fewer tests more often, and many valuable tests are worth running only once (by a human). Automate to free people for the investigation only people can do.
• Documentation: test artifacts are worth only as much as they serve a real stakeholder need; heavy documentation for its own sake is waste.
• "Good enough": the question is rarely "is it perfect" but "is it good enough for this context, and how would we know if it weren't?" — a risk judgment, made with stakeholders.

---

Where this fits with the other skills

This is the highest-altitude, most human skill in the repo. It complements rather than competes with the others:

• It is the mindset behind Q3 (exploratory/usability) in agile-testing-quadrants.
• Its heuristics for generating test ideas pair with the systematic case-derivation of effective-software-testing (exploration finds risks; spec-based testing covers them methodically) — its combination/all-pairs lesson is the same pairwise idea.
• It supplies the why test this at all / how much is enough judgment that the unit-craft skills (khorikov-unit-testing, art-of-unit-testing) assume.

---

Reference files

• references/context-driven-principles.md — the seven principles in depth, their illustrations, and the context-driven mindset (testing as a service, missions, no best practices).
• references/exploratory-testing-and-heuristics.md — exploratory testing and charters, oracles and their fallibility, consistency heuristics, and using heuristics wisely.
• references/bug-advocacy-and-reporting.md — bug advocacy, the report as a sales tool, follow-up testing, and what makes a report get the bug fixed.

../../EXAMPLES.md (repo root) has worked examples under "Lessons Learned in Software Testing" — investigative artifacts (a charter, a weak-vs-strong bug report) rather than code.