Software Engineering with Language Models
Module 8 · Verification through testing and evaluation
- Class time
- 60 minutes
- Slides
- 33, each with teacher’s notes
- Goes with
- Problem set; Assignment 3 (Part IV); Assignment 4 (Part IV); Section problems (Part IV)
Where this module sits
- In Module 7 you put a human approval after each thinking phase.
- Section 7.6 listed verification among the downstream constraints.
- Learning outcome 4 requires evidence at each gate.
- The three objectives are to judge a test, check a grader and read a reported result.
- You apply Rules 1 and 3 throughout the hour.
- Laboratories are graded on evidence, including a list of what was not checked.
Objectives
- Judge whether a test or an evaluation measures behaviour.
- Apply the four-question gate before adding a test.
- Remove a test only with the four recorded items of evidence.
- Calibrate a grader against golden solutions and the unmodified application.
- Compute opportunity and uplift from guided and unguided runs.
- Read a reported result for its protocol, its repetitions and its source.
Agenda
- 4 min: why verification is the scarce skill.
- 18 min: change-detector tests, the gate, what to reject and keep.
- 5 min: tests that use a model.
- 12 min: calibrating a grader; opportunity and uplift; equivalence.
- 13 min: reading results and checking claims; prose.
- 8 min: mistakes, problem set, practical work and recap.
8.1 Why this is the scarce skill
- Generation is cheap, and judgment is the bottleneck.
- Verifying code can be harder than writing it.
- Agent teams most want people who can write evaluations.
- Every "it works" in this course reduces to a verification method.
8.2 Change-detector tests
- A model adds a test for each small change it makes.
- Left unchecked, a suite fills up with tests that restate the implementation.
- Such tests always pass and break on every refactor.
- Projects report deletions of tens or hundreds of thousands of such lines.
- Coverage changed little after the deletions.
- The term "change-detector test" dates from 2015.
Worked example: a change-detector test
def test_welcome():
mailer = Mock()
signup("ana", mailer)
mailer.send.assert_called_once_with(
"welcome", "ana")
8.2 The four-question gate
- Name the observable behaviour, invariant or independent contract it protects.
- Name a credible regression that would make it fail.
- Say why existing coverage does not already catch that failure.
- Check whether it needs a production seam that no production caller needs.
In-class exercise: the gate on one test
- Use the welcome test from the worked example.
- Write a one-line answer to each of the four questions.
- State the verdict: add, do not add, or replace.
- Repeat on the suspect test you brought.
- Three minutes for each test.
8.2 Conditions on a regression test
code before the fix
run test -> FAILS
for the intended reason
apply the fix
run test -> passes
never failed before the fix:
the test proves the mock
8.2 Patterns to reject on sight (1 to 5)
- A probe that asserts nothing.
- A value compared with itself.
- Fixtures or export lists copied into the test.
- An exact grep of the source text.
- A private predicate tested again though a boundary test covers it.
8.2 Patterns to reject on sight (6 to 9)
- Expected values produced by the code under test.
- A mock that implements the behaviour the test asserts.
- A test whose only effect is keeping a test-only export alive.
- A negative control that passes for an unrelated reason.
8.2 Tests to keep and evidence for removal
- Keep tests that independently enforce a contract, even when they look like implementation.
- The contracts are public API, protocol, configuration, migration, storage, security and release.
- Keep call-order tests where order is observable behaviour.
- Being slow is not grounds for deletion.
- To remove a test, record its name, what it detects, the stronger proof remaining and the validation command run.
- Put end-to-end tests at the user's or caller's boundary, and end each with a repeatable artifact.
8.3 Four kinds of step in a test that uses a model
| Kind of step | Invented example |
|---|---|
| Deterministic navigation | Open the checkout page |
| Goal stated in plain language | Add the cheapest blue item to the basket |
| Model-judged assertion | The page confirms the order in plain words |
| Exact locator, deterministic assertion | The total field reads 12.50 |
8.3 Action caching and visual review
- Action caching records the concrete actions the agent took.
- Later runs replay those actions with no model call for as long as they match the application.
- Automated runs use a read-only cache.
- Tests refer to credentials by name, and the environment supplies the values.
- A scheduled visual review agent reports at most five findings, each with a screenshot.
- Motion needs filming, because one screenshot cannot show a one-frame jump.
8.4 Calibrating a grader
- Experts write the guidance and a demonstration.
- Each task sits on a real application and has a golden solution and an automated grader.
- Every golden solution passes the grader and the unmodified application fails it, every time.
- Prompts read like real requests and leave the technique unnamed.
- The same agent is run with the guidance and without it.
Ask the room: four invented graders
- Grader A passes 20 of 20 golden solutions and fails the unmodified application.
- Grader B passes 19 of 20 golden solutions and fails the unmodified application.
- Grader C passes 20 of 20 golden solutions and passes the unmodified application.
- Grader D gives A's results on one run and B's on the next.
- Decide whether each grader is usable and what to do next.
Worked example: opportunity and uplift
unguided pass rate 40%
guided pass rate 70%
opportunity = 100% - 40% = 60%
uplift = 70% - 40%
= 30 points
second task set: 95% and 96%
opportunity 5%, uplift 1 point
8.4 Equivalence proofs re-derived on each run
- The rebuild is clean, and a run that executed zero tests counts as failure.
- The old and new systems run on identical inputs, and the output is compared byte for byte with masked fields listed.
- At least ten unseen inputs cover boundaries and malformed records.
- A deliberate one-line break must turn tests red, and a one-byte change must trip the comparator.
- A test that actually ran names each critical requirement.
- The verdicts are proven, partly proven and not proven.
Worked example: the perfect win rate
- A public library holds fifty trading strategies.
- One strategy shows a hundred per cent win rate.
- The same strategy shows an infinite profit factor.
- Its risk-adjusted return is near zero.
- The library was promoted as leaked proprietary strategies.
- The library is an openly licensed teaching set, and most of it performs poorly.
8.5 Out-of-sample validation
time ------------------------------>
[ train window ] gap [ test window ]
[ train ] gap [ test ]
fit on the train window only
score on the test window only
the gap stops information leaking
8.5 Protocol and repetition
- Good model cards state sampling settings.
- They average three runs and give ranges.
- They run code with the network disabled and list what was not measured.
- They warn that small differences prove nothing.
- In one security audit harness, a single run found roughly half of what repeated runs found.
- A vendor's benchmark on its own tasks is an advertisement until reproduced.
8.6 Checking a claim before acting on it
- List at least three explanations, one of which is that the source is wrong.
- Mark whether each piece of evidence supports or contradicts each explanation.
- Evidence that fits every explanation has no diagnostic value.
- Prefer the explanation with the least evidence against it.
- Name one check you can perform yourself.
- Decide whether to act, wait or verify.
Worked example: competing hypotheses
| Evidence | Fixed as stated | Passed by chance | Test was weakened |
|---|---|---|---|
| One green run | consistent | consistent | consistent |
| Diff replaces a fixed sleep with a wait on a signal | supports | contradicts | contradicts |
| Test file unchanged in the diff | consistent | consistent | contradicts |
| Fifty further runs, all green | supports | contradicts | consistent |
8.7 Measuring prose
- Prose written by models has recognisable faults that can be counted.
- It opens with contrast, negation or hedging where the main clause should be.
- It leans on abstract nouns and metaphorical verbs, so it is unclear who does what.
- It uses short parallel sentences and nominalisation to sound weightier than the content.
- One rule repairs these faults: name the subject, the object and the action.
- Hold the reports that an agent writes for you to the same rule.
Ask the room: the meaning of two sentences
- "Comprehensive validation was performed across the authentication flow."
- "Rather than a simple patch, this is a holistic strengthening of reliability."
- Say what each sentence means in plain words.
- Rewrite each with a subject, an object and an action.
- Name the fault each one shows.
Common mistakes
- Tests or coverage are counted as evidence. Ask what regression each test catches.
- A regression test that never failed is accepted. Run it on the code before the fix.
- An uncalibrated grader is trusted. Golden solutions must pass and the unmodified application must fail.
- One metric from one run is reported. Report several measures, the worst case and repeated runs.
- A model checks its own claim. Add an external check.
- A contract test is deleted for being slow. Slowness is not grounds for deletion.
Problem set
- In problem 1 you run the four-question gate over twenty tests from a project you know.
- State how many tests pass, and delete three tests with the required evidence.
- In problem 2 you write a grader for one task and calibrate it as in 8.4.
- Report how often the grader passes the golden solutions and the unmodified base.
- In problem 3 you choose one benchmark claim published in the last month.
- Name its protocol and who ran it, what went unmeasured, and one other explanation.
Assignment 3 (Part IV): the warm-up and the stage checks
- In the warm-up, write examples for first_failing(results) before any code exists.
- Cover the empty list and an all-passing list.
- An agent writes the function from the examples alone.
- Then make one example wrong on purpose, and the run must fail.
- The staged build has five stages, each with one recorded check.
- Hand in the five checks run again on the final commit.
Assignment 4 (Part IV): the verification work
- The warm-ups are pair(predictions, gold) and a decoder for a compact agent log.
- In Sand, write each rule's expected result as executable examples before any implementation.
- The agent must not edit the examples, and a permission or a hook enforces that.
- Then break one rule by hand and confirm that an example fails.
- In Fire, judge one THRESHOLD over 200 scored items against 50 hand labels.
- In Bytes, test one encoding hypothesis and one decoder by a round trip.
Section problems (Part IV): strings and a budget object
- In problem 1 (Keep only) you strip a reply to its digits and list three replies where that fails.
- In problem 2 (Count the differences) you run one prompt twice at temperature 0 and compare the outputs character by character.
- In problem 3 (Number the lines) you return index and line pairs.
- In problem 4 (Doubling) characters double exactly and tokens do not.
- In problem 7 (A budget object) an agent re-implements ContextBudget from its test file alone.
- You write the expected results before the code in every problem.
Recap
- Judgment is the scarce skill, and every "it works" reduces to a verification method.
- A change-detector test fails on refactors and passes when a collaborator breaks.
- Ask four questions before adding a test, and write a four-item record before removing a test.
- A grader passes every golden solution and fails the unmodified application, every time.
- Report several measures, the worst case, repeated runs and what was not measured.
- The check that adds information is external to whatever made the claim.
Before next class
- Do problem 1: gate the twenty tests, count the tests that pass and delete three with evidence.
- Finish the Assignment 3 warm-up and begin stage 1 with its recorded check.
- Read Module 9, sections 9.1 to 9.6.
- Read the requirements of Laboratory 4.
- Decide what small game you would build in a browser.