Pre9 controlled manual QEMU evidence audit

Date: 2026-08-13

Scope and evidence status

This report audits the already completed manual run at /work/tests-dev/temp/pre9-manual-20260813-run3/. No QEMU test was rerun. The long-lived guard VM (PID 26318, host port 9222) was not touched.

The automated TC004 implementation remains PENDING_FIX/BLOCKED in /work/tests-dev/fix-todo/tc004-automation.md. Run3 is a manual substitute; it is not evidence that TC004 automation passes.

Run history:

Run1: infrastructure failure; no retained result directory is available.
Run2: INFRA. Guest SSH authentication failed for Pre7 and Pre8, then the host runner was interrupted while preparing Pre9. It produced no DUT result.
Run3: valid manual run. All three DUTs built, loaded, mounted, executed the bounded probes, produced a live SHA stack sample, and cleaned up.

Verdict

Question	Verdict	Evidence
4 KiB, 16 KiB, 64 KiB and 1 MiB single-syscall read correctness	PASS	Every version returned the requested byte count, `errno=0`, matched the source-range SHA-256, exited zero, and has exactly one target-file `pread()` in truss.
First 1 MiB sequential `dd` read	PASS	Every version returned zero and copied 1 MiB within the 20-second bound. This is a bounded smoke check, not proof of one 1 MiB kernel read.
Full-file SHA correctness	BLOCKED	No version retained a hash or an exit status for the background SHA. `sha_after` is empty and process disappearance is not correctness evidence.
Full-file read liveness/performance	FAIL	After an eight-second observation delay, SHA was still running in the LZMA decode path and had advanced only about 1.0-1.25 MiB. This is unacceptable for the 8 MiB smoke fixture and reproduces across all three versions.
Pre7 to Pre8/Pre9 regression attribution	PASS: no Pre9 regression demonstrated	Pre8 and Pre9 have the same `src` tree and identical KLD hash. Pre7 has a different source tree/KLD but shows the same symptom and stack. The evidence attributes the issue to shared behavior, not a Pre9-only change.

Overall verdict: PARTIAL. Bounded reads are correct, but the full-file correctness verdict is blocked and full-file liveness fails.

Runner semantics audit

The runner creates a fresh qcow2 overlay per version over the read-only base, archives the selected repository subtree, builds the KLD in the guest, mounts the same LZMA fixture, and runs four probes. Each probe invokes a C program that contains one target-file pread() and writes the returned bytes to a file. The runner then hashes that file and compares it with bytes read from the uncompressed source fixture.

Important field meanings:

commit: Git tree object for the version directory at the enclosing repo HEAD. It is not a standalone commit ID.
src_tree: Git tree object for that version's src directory.
probes[].elapsed: host wall time for the complete SSH command, including SSH and truss overhead. It is not pure kernel decompression time.
metadata.hash_match: equality between the mounted output-range hash and the uncompressed source-range hash.
signal=0: runner shorthand for probe return code zero. It is not a signal collected through waitpid().
dd_1m: a userspace dd bs=1m count=1 result. It does not establish the size or count of VOP/kernel reads.
sha_pid: PID printed after starting a background sha256 command.
diagnostic_sample: process table, kernel stack, descriptor offset, mount, md device and dmesg captured after an unconditional eight-second sleep.
sha_after: process status followed by sha.out and sha.err. Empty output means neither a process row nor captured SHA/error output was available.
status=PARTIAL: runner-generated fallback when any bounded probe succeeds. It does not mean full SHA correctness passed.

The full SHA was not run under timeout. The runner waited eight seconds, sampled it, sent TERM, slept two seconds, then attempted KILL, and finally read the output files. No start/end timestamp or exit status was recorded. Therefore the exact SHA lifetime is unknown; the only defensible timing statement is that it was still active approximately eight seconds after launch.

sha_after contains only ,state=,command= for all versions. The subsequent KILL reports No such process. This establishes only that the sampled PID no longer existed after TERM plus the two-second delay. It does not distinguish a successful completion from TERM handling, and the absent sha.out means no hash can be validated. Process disappearance must not be reported as PASS.

Controlled identities and setup

The common fixture hashes were:

Image: 32107a084b27362a093768b88746c37c2e99a74b9f1301a9d4046988479defe9
Source: ddda39737f0f6093e828a032ec161511fefbb1fa361bc6cbffdbc91e48e4c461

Version	Repository tree	`src` tree	KLD SHA-256	Build/load/mount
Pre7	`b8f9af2cd55225c4348b79ff5910ae6fc83cd517`	`dc203534d7b5721905f8538026e4c59346106020`	`33a52f2f16a94afda0501d305b238678b96c71e420d4b8bbcbeec6ffcdf1aae5`	PASS/PASS/PASS
Pre8	`1e24c992a5b071e6fdabfdc42d342b6ed5a91cf5`	`cbf93a19df8d1ce2fb66e2aeb7276a4ea6d4dcc2`	`348cb1f5a91d47e1412747d285c795c49375a61a66cfb3d35d22f38ecfde6ea0`	PASS/PASS/PASS
Pre9	`dc0c01157b5c925684bd77c57ed6703904af7453`	`cbf93a19df8d1ce2fb66e2aeb7276a4ea6d4dcc2`	`348cb1f5a91d47e1412747d285c795c49375a61a66cfb3d35d22f38ecfde6ea0`	PASS/PASS/PASS

Pre8 and Pre9 are runtime-identical for this test according to both the source tree and the produced KLD hash.

Single-syscall probes

All probes used offset zero. Times below include SSH/truss overhead.

Version	Length	Returned	Hash match	Elapsed	Exactly one target `pread()`
Pre7	4 KiB	4 KiB	yes	1.299 s	yes
Pre7	16 KiB	16 KiB	yes	1.291 s	yes
Pre7	64 KiB	64 KiB	yes	1.438 s	yes
Pre7	1 MiB	1 MiB	yes	1.432 s	yes
Pre8	4 KiB	4 KiB	yes	1.216 s	yes
Pre8	16 KiB	16 KiB	yes	1.327 s	yes
Pre8	64 KiB	64 KiB	yes	1.430 s	yes
Pre8	1 MiB	1 MiB	yes	1.442 s	yes
Pre9	4 KiB	4 KiB	yes	1.791 s	yes
Pre9	16 KiB	16 KiB	yes	1.341 s	yes
Pre9	64 KiB	64 KiB	yes	1.374 s	yes
Pre9	1 MiB	1 MiB	yes	1.737 s	yes

The target-file truss lines are preserved in pre9-manual-evidence/probe-summary.txt. Dynamic-loader pread() calls are not counted as target-file calls.

The 1 MiB dd results were:

Pre7: 1 MiB in 0.195526 s.
Pre8: 1 MiB in 0.179484 s.
Pre9: 1 MiB in 0.367417 s.

These values are not stable enough for version performance ranking, but all three bounded operations completed.

Full SHA sample

Version	State at sample	File offset	Kernel stack	Final hash
Pre7	running (`RC`)	1,245,184	`lzma2_lzma -> erofs_xz_dec_microlzma_run -> lzma_decompress -> z_erofs_decompress -> z_erofs_do_read -> z_erofs_read_uio`	absent
Pre8	running (`RC`)	1,150,976	same	absent
Pre9	running (`RC`)	1,044,480	same	absent

The stack is direct evidence of active CPU-side decompression, not an EROFS lock wait. It does not by itself prove why decompression is slow. Combined with the static review, the leading explanation remains repeated full mapped-extent decompression as z_erofs_read_uio() segments reads at MAXPHYS. There is no direct evidence here of an XZ infinite loop or an EROFS lock deadlock.

Infrastructure integrity and cleanup

The base image SHA-256 before and after run3 was identical and matched the expected value:

67f359621f23a1d745f0889370cbb99a096cee3e99a0b2f3bb18fc7a91bf6fef

For each version, umount, mdconfig -d and kldunload returned zero. Each QEMU PID was no longer alive and each overlay was deleted. The run3 result set therefore supports successful test-VM cleanup. This report does not infer the state of the unrelated long-lived guard VM and did not inspect or modify it.

Required follow-up

A supplemental manual test is still required if full-file correctness must be closed. It should run one full SHA with an explicit wall-clock deadline, record start/end timestamps and exit status, capture sha.out before cleanup, compare the hash to the source fixture, and separately record whether TERM/KILL was used. Until then, full-file SHA correctness remains BLOCKED.

8.4 KiB Raw Blame History