Syllabus Module 8 · Verification through testing and evaluation
NSW 3011 / 33
Software Engineering with Language Models

Module 8 · Verification through testing and evaluation

Class time
60 minutes
Slides
33, each with teacher’s notes
Goes with
Problem set; Assignment 3 (Part IV); Assignment 4 (Part IV); Section problems (Part IV)
Module 82 / 33

Where this module sits

  • In Module 7 you put a human approval after each thinking phase.
  • Section 7.6 listed verification among the downstream constraints.
  • Learning outcome 4 requires evidence at each gate.
  • The three objectives are to judge a test, check a grader and read a reported result.
  • You apply Rules 1 and 3 throughout the hour.
  • Laboratories are graded on evidence, including a list of what was not checked.
Module 83 / 33

Objectives

  • Judge whether a test or an evaluation measures behaviour.
  • Apply the four-question gate before adding a test.
  • Remove a test only with the four recorded items of evidence.
  • Calibrate a grader against golden solutions and the unmodified application.
  • Compute opportunity and uplift from guided and unguided runs.
  • Read a reported result for its protocol, its repetitions and its source.
Module 84 / 33

Agenda

  • 4 min: why verification is the scarce skill.
  • 18 min: change-detector tests, the gate, what to reject and keep.
  • 5 min: tests that use a model.
  • 12 min: calibrating a grader; opportunity and uplift; equivalence.
  • 13 min: reading results and checking claims; prose.
  • 8 min: mistakes, problem set, practical work and recap.
Module 85 / 33

8.1 Why this is the scarce skill

  • Generation is cheap, and judgment is the bottleneck.
  • Verifying code can be harder than writing it.
  • Agent teams most want people who can write evaluations.
  • Every "it works" in this course reduces to a verification method.
Module 86 / 33

8.2 Change-detector tests

  • A model adds a test for each small change it makes.
  • Left unchecked, a suite fills up with tests that restate the implementation.
  • Such tests always pass and break on every refactor.
  • Projects report deletions of tens or hundreds of thousands of such lines.
  • Coverage changed little after the deletions.
  • The term "change-detector test" dates from 2015.
Module 87 / 33

Worked example: a change-detector test

def test_welcome():
    mailer = Mock()
    signup("ana", mailer)
    mailer.send.assert_called_once_with(
        "welcome", "ana")
Module 88 / 33

8.2 The four-question gate

  1. Name the observable behaviour, invariant or independent contract it protects.
  2. Name a credible regression that would make it fail.
  3. Say why existing coverage does not already catch that failure.
  4. Check whether it needs a production seam that no production caller needs.
Module 89 / 33

In-class exercise: the gate on one test

  • Use the welcome test from the worked example.
  • Write a one-line answer to each of the four questions.
  • State the verdict: add, do not add, or replace.
  • Repeat on the suspect test you brought.
  • Three minutes for each test.
Module 810 / 33

8.2 Conditions on a regression test

code before the fix
  run test -> FAILS
  for the intended reason
apply the fix
  run test -> passes
never failed before the fix:
  the test proves the mock
Module 811 / 33

8.2 Patterns to reject on sight (1 to 5)

  1. A probe that asserts nothing.
  2. A value compared with itself.
  3. Fixtures or export lists copied into the test.
  4. An exact grep of the source text.
  5. A private predicate tested again though a boundary test covers it.
Module 812 / 33

8.2 Patterns to reject on sight (6 to 9)

  1. Expected values produced by the code under test.
  2. A mock that implements the behaviour the test asserts.
  3. A test whose only effect is keeping a test-only export alive.
  4. A negative control that passes for an unrelated reason.
Module 813 / 33

8.2 Tests to keep and evidence for removal

  • Keep tests that independently enforce a contract, even when they look like implementation.
  • The contracts are public API, protocol, configuration, migration, storage, security and release.
  • Keep call-order tests where order is observable behaviour.
  • Being slow is not grounds for deletion.
  • To remove a test, record its name, what it detects, the stronger proof remaining and the validation command run.
  • Put end-to-end tests at the user's or caller's boundary, and end each with a repeatable artifact.
Module 814 / 33

8.3 Four kinds of step in a test that uses a model

Kind of stepInvented example
Deterministic navigationOpen the checkout page
Goal stated in plain languageAdd the cheapest blue item to the basket
Model-judged assertionThe page confirms the order in plain words
Exact locator, deterministic assertionThe total field reads 12.50
Module 815 / 33

8.3 Action caching and visual review

  • Action caching records the concrete actions the agent took.
  • Later runs replay those actions with no model call for as long as they match the application.
  • Automated runs use a read-only cache.
  • Tests refer to credentials by name, and the environment supplies the values.
  • A scheduled visual review agent reports at most five findings, each with a screenshot.
  • Motion needs filming, because one screenshot cannot show a one-frame jump.
Module 816 / 33

8.4 Calibrating a grader

  1. Experts write the guidance and a demonstration.
  2. Each task sits on a real application and has a golden solution and an automated grader.
  3. Every golden solution passes the grader and the unmodified application fails it, every time.
  4. Prompts read like real requests and leave the technique unnamed.
  5. The same agent is run with the guidance and without it.
Module 817 / 33

Ask the room: four invented graders

  • Grader A passes 20 of 20 golden solutions and fails the unmodified application.
  • Grader B passes 19 of 20 golden solutions and fails the unmodified application.
  • Grader C passes 20 of 20 golden solutions and passes the unmodified application.
  • Grader D gives A's results on one run and B's on the next.
  • Decide whether each grader is usable and what to do next.
Module 818 / 33

Worked example: opportunity and uplift

unguided pass rate   40%
guided pass rate     70%

opportunity = 100% - 40% = 60%
uplift      = 70% - 40%
            = 30 points

second task set: 95% and 96%
opportunity 5%, uplift 1 point
Module 819 / 33

8.4 Equivalence proofs re-derived on each run

  • The rebuild is clean, and a run that executed zero tests counts as failure.
  • The old and new systems run on identical inputs, and the output is compared byte for byte with masked fields listed.
  • At least ten unseen inputs cover boundaries and malformed records.
  • A deliberate one-line break must turn tests red, and a one-byte change must trip the comparator.
  • A test that actually ran names each critical requirement.
  • The verdicts are proven, partly proven and not proven.
Module 820 / 33

Worked example: the perfect win rate

  • A public library holds fifty trading strategies.
  • One strategy shows a hundred per cent win rate.
  • The same strategy shows an infinite profit factor.
  • Its risk-adjusted return is near zero.
  • The library was promoted as leaked proprietary strategies.
  • The library is an openly licensed teaching set, and most of it performs poorly.
Module 821 / 33

8.5 Out-of-sample validation

time ------------------------------>
[ train window ] gap [ test window ]
                 [ train ] gap [ test ]

fit on the train window only
score on the test window only
the gap stops information leaking
Module 822 / 33

8.5 Protocol and repetition

  • Good model cards state sampling settings.
  • They average three runs and give ranges.
  • They run code with the network disabled and list what was not measured.
  • They warn that small differences prove nothing.
  • In one security audit harness, a single run found roughly half of what repeated runs found.
  • A vendor's benchmark on its own tasks is an advertisement until reproduced.
Module 823 / 33

8.6 Checking a claim before acting on it

  1. List at least three explanations, one of which is that the source is wrong.
  2. Mark whether each piece of evidence supports or contradicts each explanation.
  3. Evidence that fits every explanation has no diagnostic value.
  4. Prefer the explanation with the least evidence against it.
  5. Name one check you can perform yourself.
  6. Decide whether to act, wait or verify.
Module 824 / 33

Worked example: competing hypotheses

EvidenceFixed as statedPassed by chanceTest was weakened
One green runconsistentconsistentconsistent
Diff replaces a fixed sleep with a wait on a signalsupportscontradictscontradicts
Test file unchanged in the diffconsistentconsistentcontradicts
Fifty further runs, all greensupportscontradictsconsistent
Module 825 / 33

8.7 Measuring prose

  • Prose written by models has recognisable faults that can be counted.
  • It opens with contrast, negation or hedging where the main clause should be.
  • It leans on abstract nouns and metaphorical verbs, so it is unclear who does what.
  • It uses short parallel sentences and nominalisation to sound weightier than the content.
  • One rule repairs these faults: name the subject, the object and the action.
  • Hold the reports that an agent writes for you to the same rule.
Module 826 / 33

Ask the room: the meaning of two sentences

  • "Comprehensive validation was performed across the authentication flow."
  • "Rather than a simple patch, this is a holistic strengthening of reliability."
  • Say what each sentence means in plain words.
  • Rewrite each with a subject, an object and an action.
  • Name the fault each one shows.
Module 827 / 33

Common mistakes

  • Tests or coverage are counted as evidence. Ask what regression each test catches.
  • A regression test that never failed is accepted. Run it on the code before the fix.
  • An uncalibrated grader is trusted. Golden solutions must pass and the unmodified application must fail.
  • One metric from one run is reported. Report several measures, the worst case and repeated runs.
  • A model checks its own claim. Add an external check.
  • A contract test is deleted for being slow. Slowness is not grounds for deletion.
Module 828 / 33

Problem set

  • In problem 1 you run the four-question gate over twenty tests from a project you know.
  • State how many tests pass, and delete three tests with the required evidence.
  • In problem 2 you write a grader for one task and calibrate it as in 8.4.
  • Report how often the grader passes the golden solutions and the unmodified base.
  • In problem 3 you choose one benchmark claim published in the last month.
  • Name its protocol and who ran it, what went unmeasured, and one other explanation.
Module 829 / 33

Assignment 3 (Part IV): the warm-up and the stage checks

  • In the warm-up, write examples for first_failing(results) before any code exists.
  • Cover the empty list and an all-passing list.
  • An agent writes the function from the examples alone.
  • Then make one example wrong on purpose, and the run must fail.
  • The staged build has five stages, each with one recorded check.
  • Hand in the five checks run again on the final commit.
Module 830 / 33

Assignment 4 (Part IV): the verification work

  • The warm-ups are pair(predictions, gold) and a decoder for a compact agent log.
  • In Sand, write each rule's expected result as executable examples before any implementation.
  • The agent must not edit the examples, and a permission or a hook enforces that.
  • Then break one rule by hand and confirm that an example fails.
  • In Fire, judge one THRESHOLD over 200 scored items against 50 hand labels.
  • In Bytes, test one encoding hypothesis and one decoder by a round trip.
Module 831 / 33

Section problems (Part IV): strings and a budget object

  • In problem 1 (Keep only) you strip a reply to its digits and list three replies where that fails.
  • In problem 2 (Count the differences) you run one prompt twice at temperature 0 and compare the outputs character by character.
  • In problem 3 (Number the lines) you return index and line pairs.
  • In problem 4 (Doubling) characters double exactly and tokens do not.
  • In problem 7 (A budget object) an agent re-implements ContextBudget from its test file alone.
  • You write the expected results before the code in every problem.
Module 832 / 33

Recap

  • Judgment is the scarce skill, and every "it works" reduces to a verification method.
  • A change-detector test fails on refactors and passes when a collaborator breaks.
  • Ask four questions before adding a test, and write a four-item record before removing a test.
  • A grader passes every golden solution and fails the unmodified application, every time.
  • Report several measures, the worst case, repeated runs and what was not measured.
  • The check that adds information is external to whatever made the claim.
Module 833 / 33

Before next class

  • Do problem 1: gate the twenty tests, count the tests that pass and delete three with evidence.
  • Finish the Assignment 3 warm-up and begin stage 1 with its recorded check.
  • Read Module 9, sections 9.1 to 9.6.
  • Read the requirements of Laboratory 4.
  • Decide what small game you would build in a browser.