Syllabus Module 11 · Laboratory: reverse engineering
NSW 3011 / 31
Software Engineering with Language Models

Module 11 · Laboratory: reverse engineering

Class time
60 minutes
Slides
31, each with teacher’s notes
Goes with
Laboratory 6; Assignment 4 (Part IV)
Module 112 / 31

Where this module sits

  • Module 10 ended with pages that agents can read (10.5).
  • In section 8.1 every claim that something works reduces to a verification method.
  • In Module 11 you recover behaviour from the artifact alone.
  • You state each finding with its evidence and its limits.
  • You use the traffic capture of 11.6 again in section 12.4.
Module 113 / 31

Objectives

  • State the scope, which is what you own or are authorised to test.
  • Turn a vague aim into one answerable question.
  • Apply the seven-step method, with static analysis before dynamic analysis.
  • Keep a findings log with evidence and a limit on every entry.
  • Match a target type to its engine, and isolate untrusted input.
  • Prove a reconstruction by differential testing and, where possible, byte-identical output.
Module 114 / 31

Agenda

  • 5 min: where this sits, objectives, scope and law (11.1)
  • 14 min: the seven-step method and the findings log (11.2)
  • 9 min: the toolchain and target types (11.3)
  • 7 min: the standard of proof (11.4)
  • 17 min: hardware and radio, black-box audits, closed toolchains (11.5 to 11.7)
  • 8 min: mistakes, hand-ins, recap
Module 115 / 31

Scope and law

  • Reverse engineering is how interoperability and security research get done.
  • Preservation and repair depend on reverse engineering as well.
  • Reverse engineering is also how software is cloned.
  • Work only on targets you own or are authorised to test.
  • Respect licence terms and local law.
  • Cheap analysis by agents is one reason value moves from code to data and distribution.
Module 116 / 31

The seven-step method

  1. State the question.
  2. Inventory without executing.
  3. Static analysis.
  4. Dynamic analysis.
  5. Hypothesise and test.
  6. Reconstruct.
  7. Prove.
Module 117 / 31

Step 1 · Stating the question

  • An answerable question is "How does feature X compute its result."
  • An unanswerable question is "Understand this program."
  • The answerable question names one feature and one result.
  • The question decides when the work is finished.
  • Laboratory 6 asks for one stated question about behaviour.
Module 118 / 31

Ask the room: answerable and unanswerable questions

  • "Understand this program."
  • "Find out what the thermostat does."
  • "How the export feature computes its checksum."
  • Mark each statement as answerable or unanswerable.
  • Rewrite the statements that are unanswerable.
Module 119 / 31

Steps 2 and 3 · Inventory and static analysis

  • The inventory lists file type, sections, strings, imports, embedded resources and signatures.
  • Nothing is executed during the inventory.
  • In static analysis you disassemble and decompile.
  • Static analysis starts from a known string or API call.
  • Cross-references lead from that string or call to the code of interest.
Module 1110 / 31

Steps 4 and 5 · Dynamic analysis and hypothesis testing

  • In dynamic analysis you run the target under observation in an isolated environment.
  • You observe system calls, files touched and network traffic.
  • You place a debugger at the function found by static analysis.
  • You write down what you believe the code does.
  • You also write down an experiment that would show the belief to be wrong.
Module 1111 / 31

The findings log

FieldWhat goes in it
Number and dateA numbered, dated entry
FindingOne claim about behaviour
EvidenceWhat was observed
Tool and versionWhat observed it
LimitWhat could not be determined
Module 1112 / 31

Worked example: a log entry for one hypothesis

Entry 4, day 3, toy save file
Finding: bytes 0-3 hold the score,
  least significant byte first
Evidence: score 140 -> 141 changed
  byte 0 from 8C to 8D, no other
Tool: hex dump, two files compared
Limit: scores above 255 not tried;
  bytes 4-15 unknown
Module 1113 / 31

The Ghidra engine

  • Ghidra is open source and is maintained by the research directorate of the NSA.
  • It offers disassembly, decompilation, graphing and scripting.
  • It covers many processors and formats.
  • It runs interactively or headless, and you can extend it in Java and Python.
  • The Ghidra documentation warns of known vulnerabilities in some versions.
  • Analyse untrusted samples in a virtual machine.
Module 1114 / 31

Agents and the tool server

  • Agents drive these engines through a tool server.
  • With one open package a coding agent reaches many target types through a single interface.
  • Each result comes back with its evidence.
  • Static inspection does not execute the target.
  • Runtime capture executes the target with your permissions, which is why you use a sandbox.
Module 1115 / 31

Targets and engines (part 1)

TargetWhat is recoveredEngine
Native binariesPseudocode, assembly, strings, symbols, cross-referencesGhidra, Hopper or IDA
JavaScript and Electron appsModules, imports, source maps, routes, inter-process messagesnone needed
Android packagesManifest, classes, decompiled methodsJADX, Apktool
.NET assembliesMetadata, intermediate languagestatic only
Module 1116 / 31

Targets and engines (part 2)

TargetWhat is recoveredEngine
WebAssemblySections, imports and exports, text formWABT
Firmware imagesRegions and extracted file systemsBinwalk, Unblob
Network capturesRequests, responses, payloadsHAR files, mitmproxy
Running processesOutput, exit status, file system effectsa pseudo-terminal
Module 1117 / 31

Ask the room: an engine for each target

  • Target A is a desktop chat application built on Electron, which you are licensed to use.
  • Target B is the firmware image of a router you own.
  • Target C is an Android package you are licensed to use.
  • Target D is a recording of the requests your own thermostat makes.
  • For each target, name what is recovered and the engine that recovers it.
Module 1118 / 31

Steps 6 and 7 · The standard of proof

  • A reconstruction with no test is a plausible story.
  • The first test is differential testing against the original.
  • The second test is byte-identical output, where a compiler is available.
  • One case is a sound-panning calculation from an old game, which passed 3,205 test cases and compiled to the same 63 bytes.
  • Another case is a 16-bit DOS calculation rebuilt in C++ and compared with the historical compiler's output.
Module 1119 / 31

Worked example: proving a reconstruction

3,205 inputs from original x86
        |             |
   original     reconstruction
        |             |
        +-- compare --+
        all 3,205 equal

compile reconstruction
  -> 63 bytes == original 63 bytes
Module 1120 / 31

The cheapest signal in hardware and radio

  • What looks like magic in a device is often a single simple signal.
  • First, list every sensor and radio the device has.
  • Then find the cheapest signal that accounts for the behaviour.
  • For an appliance, capture traffic while pressing each control.
  • Then write a thin client for the recovered protocol.
  • An agent writes the client in minutes, and the capture and the hypothesis remain your work.
Module 1121 / 31

Worked example: a shape that jumps between two boards

board A  ---- packets ---->  board B
board A  <--- packets -----  board B
each reads the other's signal
strength

far  ........................  near
signal    low  ---------->  high
animation  0%  ---------->  100%
jump fires at a threshold
Module 1122 / 31

The method of a black-box audit

  • A peer-reviewed audit of smart televisions studied devices that cannot be opened.
  • You put the device behind a network you control.
  • You capture its traffic.
  • You vary one condition at a time.
  • The audit varied four conditions: login, tracking opt-in, input source and country.
Module 1123 / 31

Worked example: the television audit

condition varied       traffic
HDMI display only      continues
tracking opted out     stops
country changed        differs

reported capture interval:
10 to 500 ms, by maker
Module 1124 / 31

In-class exercise: a plan for your own audit

  • Choose a device you own that connects to your network.
  • List the conditions you can vary, one at a time.
  • For each condition, write what you expect to see in the traffic.
  • Write the observation that would prove the expectation wrong.
  • State what the capture cannot determine.
Module 1125 / 31

Three constraints on replacing a closed toolchain

  • The case is an open project that takes iPhone applications from build to shipping on Linux, with no Mac.
  • One step may be irreducible.
  • The platform SDK exists only in the vendor's distribution, and the project fetches it once.
  • Version matching is a hard constraint.
  • The compiler in the SDK's release must match the installed toolchain.
  • Most failures come from the environment.
Module 1126 / 31

Three practices for replacing a closed toolchain

  • Validate offline before upload.
  • The release script stops on any failed check.
  • The project replaces the vendor's asset compiler and interface compiler with stand-ins.
  • The project judges the stand-ins by byte-identical output on real applications.
  • A numbered findings log records what broke and how it was fixed.
  • Each entry has a date and the exact versions on which it was verified.
Module 1127 / 31

Common mistakes

  • Starting from "understand it". Fix: ask one question, with an experiment that could refute the answer.
  • Running the target first. Fix: do the inventory and static analysis before any execution.
  • Analysing untrusted samples on the host. Fix: use a virtual machine, and a sandbox for runtime capture.
  • Logging conclusions without evidence. Fix: put the observation, the tool and the limit on every entry.
  • Calling a reconstruction done when it reads well. Fix: run a differential test on many inputs.
  • Repeating an unverified device claim. Fix: record it as a hypothesis and test it.
Module 1128 / 31

Laboratory 6: what is due

  • The target is a licensed web or Electron application, a device you own, or a public exercise binary.
  • One stated question about the target's behaviour, and its answer.
  • A findings log giving evidence and limits on every entry.
  • A reconstruction.
  • A differential test of the reconstruction against the original on at least 100 inputs.
  • A statement of what the agent did and what you checked yourself.
Module 1129 / 31

Assignment 4 · Sand and fire with byte decoding: what is due

  • The warm-ups are pair(predictions, gold) and a decoder for a compact log.
  • For Sand, write the expected results for every rule as executable examples before any implementation.
  • A permission or a hook stops the agent editing the examples.
  • For Fire, take one threshold, try three values and label 50 items by hand.
  • For Bytes, which belongs to this module, state one hypothesis about an unknown format, write a decoder and run a round-trip test.
  • Record each failed hypothesis with the evidence that killed it.
Module 1130 / 31

Recap

  • Start with scope, which is what you own or are authorised to test.
  • Ask one answerable question, then follow the seven steps, and execute nothing before step 4.
  • Record every finding with its evidence, its tool and its limit.
  • Run a tool that parses hostile input in a virtual machine.
  • Proof is differential testing, and byte-identical output where a compiler exists.
  • For devices, list sensors and radios, find the cheapest signal and vary one condition at a time.
Module 1131 / 31

Before next class

  • Choose your Laboratory 6 target and question, and open the findings log at entry 1.
  • Do the Bytes part of Assignment 4 as a rehearsal.
  • Read Module 12 on network protection.
  • List every device on a network you administer, with its maker and purpose.
  • Confirm that you can change your router's settings.