Debugging and Tests

Debugging with a model is either the fastest thing you do all week or a long walk in the wrong direction, and which one you get is decided in the first message.

The debugging sequence

1. Reproduce it reliably first. If you cannot trigger it on demand, you cannot tell whether anything fixed it — and neither can the model
2. Paste the complete error and stack trace. The truncated version removes the frame that identifies the cause
3. Include the code the trace points at, plus what it calls. The bug is often one level down from the crash
4. State what you already ruled out. Otherwise the first three suggestions will be the things you tried an hour ago
5. Ask for hypotheses ranked by likelihood, each with a way to test it — not for a fix

Why 'it works now' is not a fix

Intermittent bugs sometimes stop appearing after an unrelated change. That is not evidence of a fix — it is evidence that the timing shifted.

Before closing anything intermittent, you should be able to explain why it happened. If the only story is 'the model changed this and it stopped', it will come back, usually under load and usually on a weekend.

Tests: the strongest use case

Test generation is where these tools shine, because the economics reverse. The test either passes or fails, verification is automatic, and the boring cases you would have skipped are exactly what a model produces tirelessly.

Give it the function and the framework, and ask explicitly for the awkward inputs.

Write tests for this function using Vitest.

Cover:
- the normal case
- empty input, null, undefined
- boundary values (0, negative, maximum)
- malformed input that should throw
- one realistic case from production shape (not toy data)

Each test name states the behaviour, not the implementation.
Do not test private internals.

[function]

The trap: tests that encode the bug

If you ask a model to write tests for code that is already wrong, it will write tests that pass. It infers intent from the implementation, so a bug becomes the expected behaviour, now protected by a green test.

Write the assertion for what the code should do, from the requirement, before you look at what it currently does. When a generated test fails, resist the urge to make it pass — first decide which one is wrong, the test or the code.

Coverage is not correctness

It is easy to generate a hundred tests and move coverage from 40% to 85% in an afternoon. Coverage measures which lines ran, not whether the assertions mean anything. A test that calls a function and asserts it did not throw covers the line and catches nothing.

Ten tests that would fail if the behaviour changed are worth more than a hundred that would not.