# Pre9 controlled manual QEMU evidence audit

Date: 2026-08-13

## Scope and evidence status

This report audits the already completed manual run at
`/work/tests-dev/temp/pre9-manual-20260813-run3/`.  No QEMU test was rerun.
The long-lived guard VM (PID 26318, host port 9222) was not touched.

The automated TC004 implementation remains `PENDING_FIX/BLOCKED` in
`/work/tests-dev/fix-todo/tc004-automation.md`.  Run3 is a manual substitute;
it is not evidence that TC004 automation passes.

Run history:

- Run1: infrastructure failure; no retained result directory is available.
- Run2: `INFRA`. Guest SSH authentication failed for Pre7 and Pre8, then the
  host runner was interrupted while preparing Pre9. It produced no DUT result.
- Run3: valid manual run. All three DUTs built, loaded, mounted, executed the
  bounded probes, produced a live SHA stack sample, and cleaned up.

## Verdict

| Question | Verdict | Evidence |
| --- | --- | --- |
| 4 KiB, 16 KiB, 64 KiB and 1 MiB single-syscall read correctness | PASS | Every version returned the requested byte count, `errno=0`, matched the source-range SHA-256, exited zero, and has exactly one target-file `pread()` in truss. |
| First 1 MiB sequential `dd` read | PASS | Every version returned zero and copied 1 MiB within the 20-second bound. This is a bounded smoke check, not proof of one 1 MiB kernel read. |
| Full-file SHA correctness | BLOCKED | No version retained a hash or an exit status for the background SHA. `sha_after` is empty and process disappearance is not correctness evidence. |
| Full-file read liveness/performance | FAIL | After an eight-second observation delay, SHA was still running in the LZMA decode path and had advanced only about 1.0-1.25 MiB. This is unacceptable for the 8 MiB smoke fixture and reproduces across all three versions. |
| Pre7 to Pre8/Pre9 regression attribution | PASS: no Pre9 regression demonstrated | Pre8 and Pre9 have the same `src` tree and identical KLD hash. Pre7 has a different source tree/KLD but shows the same symptom and stack. The evidence attributes the issue to shared behavior, not a Pre9-only change. |

Overall verdict: **PARTIAL**. Bounded reads are correct, but the full-file
correctness verdict is blocked and full-file liveness fails.

## Runner semantics audit

The runner creates a fresh qcow2 overlay per version over the read-only base,
archives the selected repository subtree, builds the KLD in the guest, mounts
the same LZMA fixture, and runs four probes. Each probe invokes a C program that
contains one target-file `pread()` and writes the returned bytes to a file.
The runner then hashes that file and compares it with bytes read from the
uncompressed source fixture.

Important field meanings:

- `commit`: Git tree object for the version directory at the enclosing repo
  HEAD. It is not a standalone commit ID.
- `src_tree`: Git tree object for that version's `src` directory.
- `probes[].elapsed`: host wall time for the complete SSH command, including
  SSH and truss overhead. It is not pure kernel decompression time.
- `metadata.hash_match`: equality between the mounted output-range hash and
  the uncompressed source-range hash.
- `signal=0`: runner shorthand for probe return code zero. It is not a signal
  collected through `waitpid()`.
- `dd_1m`: a userspace `dd bs=1m count=1` result. It does not establish the
  size or count of VOP/kernel reads.
- `sha_pid`: PID printed after starting a background `sha256` command.
- `diagnostic_sample`: process table, kernel stack, descriptor offset, mount,
  md device and dmesg captured after an unconditional eight-second sleep.
- `sha_after`: process status followed by `sha.out` and `sha.err`. Empty output
  means neither a process row nor captured SHA/error output was available.
- `status=PARTIAL`: runner-generated fallback when any bounded probe succeeds.
  It does not mean full SHA correctness passed.

The full SHA was not run under `timeout`. The runner waited eight seconds,
sampled it, sent TERM, slept two seconds, then attempted KILL, and finally read
the output files. No start/end timestamp or exit status was recorded. Therefore
the exact SHA lifetime is unknown; the only defensible timing statement is that
it was still active approximately eight seconds after launch.

`sha_after` contains only `,state=,command=` for all versions. The subsequent
KILL reports `No such process`. This establishes only that the sampled PID no
longer existed after TERM plus the two-second delay. It does not distinguish a
successful completion from TERM handling, and the absent `sha.out` means no
hash can be validated. Process disappearance must not be reported as PASS.

## Controlled identities and setup

The common fixture hashes were:

- Image: `32107a084b27362a093768b88746c37c2e99a74b9f1301a9d4046988479defe9`
- Source: `ddda39737f0f6093e828a032ec161511fefbb1fa361bc6cbffdbc91e48e4c461`

| Version | Repository tree | `src` tree | KLD SHA-256 | Build/load/mount |
| --- | --- | --- | --- | --- |
| Pre7 | `b8f9af2cd55225c4348b79ff5910ae6fc83cd517` | `dc203534d7b5721905f8538026e4c59346106020` | `33a52f2f16a94afda0501d305b238678b96c71e420d4b8bbcbeec6ffcdf1aae5` | PASS/PASS/PASS |
| Pre8 | `1e24c992a5b071e6fdabfdc42d342b6ed5a91cf5` | `cbf93a19df8d1ce2fb66e2aeb7276a4ea6d4dcc2` | `348cb1f5a91d47e1412747d285c795c49375a61a66cfb3d35d22f38ecfde6ea0` | PASS/PASS/PASS |
| Pre9 | `dc0c01157b5c925684bd77c57ed6703904af7453` | `cbf93a19df8d1ce2fb66e2aeb7276a4ea6d4dcc2` | `348cb1f5a91d47e1412747d285c795c49375a61a66cfb3d35d22f38ecfde6ea0` | PASS/PASS/PASS |

Pre8 and Pre9 are runtime-identical for this test according to both the source
tree and the produced KLD hash.

## Single-syscall probes

All probes used offset zero. Times below include SSH/truss overhead.

| Version | Length | Returned | Hash match | Elapsed | Exactly one target `pread()` |
| --- | ---: | ---: | --- | ---: | --- |
| Pre7 | 4 KiB | 4 KiB | yes | 1.299 s | yes |
| Pre7 | 16 KiB | 16 KiB | yes | 1.291 s | yes |
| Pre7 | 64 KiB | 64 KiB | yes | 1.438 s | yes |
| Pre7 | 1 MiB | 1 MiB | yes | 1.432 s | yes |
| Pre8 | 4 KiB | 4 KiB | yes | 1.216 s | yes |
| Pre8 | 16 KiB | 16 KiB | yes | 1.327 s | yes |
| Pre8 | 64 KiB | 64 KiB | yes | 1.430 s | yes |
| Pre8 | 1 MiB | 1 MiB | yes | 1.442 s | yes |
| Pre9 | 4 KiB | 4 KiB | yes | 1.791 s | yes |
| Pre9 | 16 KiB | 16 KiB | yes | 1.341 s | yes |
| Pre9 | 64 KiB | 64 KiB | yes | 1.374 s | yes |
| Pre9 | 1 MiB | 1 MiB | yes | 1.737 s | yes |

The target-file truss lines are preserved in
`pre9-manual-evidence/probe-summary.txt`. Dynamic-loader `pread()` calls are
not counted as target-file calls.

The 1 MiB `dd` results were:

- Pre7: 1 MiB in 0.195526 s.
- Pre8: 1 MiB in 0.179484 s.
- Pre9: 1 MiB in 0.367417 s.

These values are not stable enough for version performance ranking, but all
three bounded operations completed.

## Full SHA sample

| Version | State at sample | File offset | Kernel stack | Final hash |
| --- | --- | ---: | --- | --- |
| Pre7 | running (`RC`) | 1,245,184 | `lzma2_lzma -> erofs_xz_dec_microlzma_run -> lzma_decompress -> z_erofs_decompress -> z_erofs_do_read -> z_erofs_read_uio` | absent |
| Pre8 | running (`RC`) | 1,150,976 | same | absent |
| Pre9 | running (`RC`) | 1,044,480 | same | absent |

The stack is direct evidence of active CPU-side decompression, not an EROFS
lock wait. It does not by itself prove why decompression is slow. Combined with
the static review, the leading explanation remains repeated full mapped-extent
decompression as `z_erofs_read_uio()` segments reads at `MAXPHYS`. There is no
direct evidence here of an XZ infinite loop or an EROFS lock deadlock.

## Infrastructure integrity and cleanup

The base image SHA-256 before and after run3 was identical and matched the
expected value:

`67f359621f23a1d745f0889370cbb99a096cee3e99a0b2f3bb18fc7a91bf6fef`

For each version, `umount`, `mdconfig -d` and `kldunload` returned zero. Each
QEMU PID was no longer alive and each overlay was deleted. The run3 result set
therefore supports successful test-VM cleanup. This report does not infer the
state of the unrelated long-lived guard VM and did not inspect or modify it.

## Required follow-up

A supplemental manual test is still required if full-file correctness must be
closed. It should run one full SHA with an explicit wall-clock deadline, record
start/end timestamps and exit status, capture `sha.out` before cleanup, compare
the hash to the source fixture, and separately record whether TERM/KILL was
used. Until then, full-file SHA correctness remains `BLOCKED`.