Prompting for Test Generation That Finds Real Bugs

TL;DR: Test generation prompting means steering a model toward tests that would catch a real bug, not tests that merely run and pass. An unconstrained prompt reliably produces green suites that check nothing: implementation-coupled assertions, trivial passes, over-mocked boundaries, and skipped edge cases. This page covers the clauses that stop each one, plus a self-adversarial follow-up prompt.

"Write unit tests for this function" gets you a file that runs. It rarely gets you a file that would notice if the function were wrong. That gap is not a framework problem, so naming Jest or pytest or JUnit in your prompt does not close it, and it is not a coverage problem either, so a green run and a high percentage do not close it. It is a specificity problem: a generic request leaves the model free to satisfy "wrote tests" in the cheapest way that still compiles, which is a suite that passes almost regardless of what the code does. What follows are the exact failure modes that produces, the prompt clauses that stop each one, and a follow-up prompt that catches whatever the first pass still misses.

Why Doesn't "Write Unit Tests for This" Produce Tests That Catch Bugs?

Because the instruction never says what a test is for. Left unconstrained, a model asked to test a function will reliably do the least amount of work that still looks complete: cover the input you already showed it, pass on the first try, and stop. None of that requires understanding what would make the function wrong, only what makes the current version's output reproducible.

The fix is not more tests. It is a different instruction: ask for tests that would fail if the code were subtly different, not tests that pass against the code as written. That single reframing is the difference between a suite that documents today's behavior and a suite that protects tomorrow's refactor.

What Does Mutation Testing Teach You About Prompting for Tests?

Mutation testing is a real, tool-supported technique for measuring exactly this gap, and it gives you the right mental model even if you never run the tool. PIT, a mutation-testing system for Java and the JVM, describes the mechanism plainly: "Faults (or mutations) are automatically seeded into your code, then your tests are run. If your tests fail then the mutation is killed, if your tests pass then the mutation lived." A suite that lets most mutations live is a suite that would not have caught the equivalent real bug.

You cannot run PIT against a JavaScript codebase or a one-off Python script mid-conversation, and that is fine. What you can do is ask the model to play both roles: write the tests, then write a wrong implementation that a careless reviewer would approve, then check its own work. That loop is the subject of the section after next.

What Exact Instruction Stops Each Failure Mode?

Four ways a generated suite passes without checking anything, and the specific clause that closes each one.

It re-derives the answer instead of asserting a known one. A test that repeats the function's own formula will pass no matter which operator inside that formula is wrong, because it copies the mistake along with the logic.

def calculate_shipping_cost(weight_kg, distance_km):
    return round(weight_kg * 0.8 + distance_km * 0.05, 2)

# Passes regardless of what the formula computes — it just re-derives it
def test_calculate_shipping_cost():
    weight, distance = 12, 340
    assert calculate_shipping_cost(weight, distance) == round(weight * 0.8 + distance * 0.05, 2)

The clause: "Assert against a literal expected value you computed independently, never against an expression that repeats the function's own logic." Applied here:

def test_shipping_cost_for_a_12kg_package_over_340km():
    assert calculate_shipping_cost(12, 340) == 26.6

def test_shipping_cost_for_a_zero_weight_package():
    assert calculate_shipping_cost(0, 340) == 17.0

It passes on presence alone. expect(result).toBeTruthy() or assert result is not None passes for almost any non-crashing output, wrong ones included. The clause: "Assert equality against a concrete value or a specific error type, not merely that something was returned or did not throw, unless 'did not throw' is genuinely the whole contract."

It mocks away the thing under test. Mock every collaborator a function touches and the test stops exercising the function's own wiring.

// Every dependency is mocked — the test cannot fail regardless of call order
test("charges the customer", async () => {
  jest.spyOn(inventory, "reserve").mockResolvedValue(true);
  jest.spyOn(gateway, "charge").mockResolvedValue({ ok: true });
  const result = await chargeCustomer(order);
  expect(result.ok).toBe(true);
});

The clause: "Name the true I/O boundary explicitly and mock only that. Let pure functions and in-process collaborators run for real, and assert on what was actually passed to the boundary."

test("charges the customer the order total exactly once", async () => {
  const chargeSpy = jest.spyOn(gateway, "charge").mockResolvedValue({ ok: true });
  await chargeCustomer(order);
  expect(chargeSpy).toHaveBeenCalledWith({ amount: order.total, currency: "USD" });
  expect(chargeSpy).toHaveBeenCalledTimes(1);
});

Now the test checks that the real inventory and order logic produced the right call, not just that two mocks returned success.

It stops at the happy path. The clause here is the whole next section, because "add edge cases too" as an afterthought reliably gets ignored.

How Do You Prompt for Edge Cases the Happy Path Never Touches?

By naming the categories instead of trusting "and edge cases too" to cover them. A model asked for "some tests" reliably produces the happy path plus one obvious failure, which is exactly the coverage a reviewer would send back.

Ask for one case per category, by name: an empty or null input; a boundary value (zero, a negative number where the domain assumes positive, the maximum allowed length); every documented error or exception path, asserted against the specific type or message rather than "it throws"; and, for anything touched by more than one caller at once, a concurrent-execution case.

That last category is the one most prompts skip entirely, because a model reasoning about a single function call has no reason to imagine two of them running at the same time unless you tell it to. Name the shared state and ask for a test that exercises the race window directly:

func TestReserveSeat_ConcurrentRequestsNeverOversell(t *testing.T) {
	pool := NewSeatPool(1) // exactly one seat left
	var wg sync.WaitGroup
	results := make([]bool, 10)
	for i := 0; i < 10; i++ {
		wg.Add(1)
		go func(i int) {
			defer wg.Done()
			results[i] = pool.Reserve()
		}(i)
	}
	wg.Wait()

	reserved := 0
	for _, ok := range results {
		if ok {
			reserved++
		}
	}
	if reserved != 1 {
		t.Errorf("expected exactly 1 successful reservation, got %d", reserved)
	}
}

The prompt clause: "If [the function or service] can be called from multiple goroutines, threads, or requests at once, name the shared mutable state explicitly and require a test that calls it concurrently, asserting an invariant that must hold no matter the interleaving. If the language has a race detector, name it and require the suite to be run under it." For Go, that means asking for the test above to be run with go test -race, which instruments memory access and flags the unsynchronized read-modify-write even in a run where the count happens to come out right by luck. A test that only checks the final count can pass by accident on a run that never triggered the race; the race detector catches the hazard itself, not just its occasional symptom.

How Do You Get the Model to Adversarially Test Its Own Tests?

By asking it to try to break its own work in a second turn, which is the manual version of the mutation-testing loop above. Once you have a test file, paste it back with this:

Here is the test file you just wrote:
[paste the test file]

Now write a DIFFERENT implementation of the same function or class — one
that is subtly wrong, plausible enough that a careless reviewer would
approve it in a code review — and that still passes every one of these
tests without modification.

Then say explicitly:
1. Which specific test SHOULD have caught the wrong implementation and
   didn't.
2. What assertion is missing from that test, precisely.

If every test would genuinely catch every plausible wrong implementation
you can think of, say so instead of inventing a forced failure.

This works because it hands the model a concrete, checkable task instead of an abstract instruction to "be thorough." Writing a wrong-but-plausible implementation is something the model is good at, and comparing that implementation against the existing tests is mechanical rather than a matter of judgment it can hand-wave past. When it succeeds at fooling its own suite, it also tells you exactly which assertion to add, which is a shorter loop than reading the whole file again and guessing.

Run this once, fix the gap it finds, and stop. This is a spot-check for the failure modes above, not a substitute for reading the file, and it will not surface something the two of you cannot conceive of together, only something the first pass conceived of loosely.

Do You Still Need to Read Every Assertion Yourself?

Yes, every one, every time, including the ones the adversarial pass above did not flag anything on. A generated suite that runs green and survives one adversarial round tells you the tests that exist passed and resisted one deliberately-invented wrong implementation. It tells you nothing about a wrong implementation neither of you thought to try, and nothing about whether an assertion checks the actual business rule rather than a plausible-looking stand-in for it. A prompt cannot know that your discount policy caps at 50% regardless of what a formula computes unless you put that rule in the prompt yourself, and no amount of adversarial prompting recovers a business rule nobody stated.

The workable split: the model drafts, the clauses above and the adversarial pass tighten the draft, and a person who understands the business rule reads every assertion before it merges. That is not a lower bar than writing tests by hand. It is the same bar, applied to a draft that already covers the categories a human would otherwise have to remember to write down.

Where Does Test-Generation Prompting Fit Next to Code Review and CI?

Writing tests that would catch a bug is one part of a chain that also includes catching what already shipped without one, and running the tests you trust on every change automatically. If the function under review needs a second set of eyes before it needs a second set of tests, How to Prompt for a Genuinely Useful Code Review covers the same specificity problem from the review side, and 30 AI Prompts for Debugging picks up from here for when a test you wrote starts failing and you need to find out why. The adversarial-pass technique above is a narrow application of a broader habit: Red-Team Your Own Prompt Before You Trust the Output generalizes it past testing.

Once a suite earns your trust, the next step is making sure it actually runs on every push rather than only on the machine that wrote it: Prompting for CI/CD Pipelines covers wiring a test job into a workflow file without leaking a secret or granting it more access than it needs.

None of this requires an account, and every clause above runs as-is in ChatGPT, Claude, or Gemini. If you want the wording of a prompt like these tightened before you send it, that is what our Chrome extension does inside the same chat window, on Pro and above; the free plan gives 5 prompt enhancements a day, forever, per our FAQ page.