348 lines
16 KiB
Markdown
348 lines
16 KiB
Markdown
# Pre10 Completion and Targeted Smoke Report
|
|
|
|
Status: **PRE10 IMPLEMENTATION COMPLETE; TARGETED SMOKE PASS; FULL FEATURE TEST NOT RUN**
|
|
|
|
Date: 2026-08-13
|
|
|
|
## Scope and goals
|
|
|
|
Pre10 is a controlled maintenance-alignment release based on the final Pre9
|
|
tree. Its purpose is to make the FreeBSD EROFS implementation easier to compare
|
|
and maintain alongside the independent Linux EROFS repository without copying
|
|
Linux-only lifecycle or memory-management models.
|
|
|
|
The planned work was limited to three source batches:
|
|
|
|
1. Remove unused Linux-shaped compatibility declarations and improve source
|
|
ordering.
|
|
2. Extract bounded helpers where Linux and FreeBSD responsibilities are
|
|
comparable, while retaining FreeBSD ownership and errno behavior.
|
|
3. Introduce a BSD-native synchronous decompressor request and descriptor
|
|
boundary with Linux-comparable codec names.
|
|
|
|
Full feature validation, CI execution, performance redesign, Linux page/folio/
|
|
bio/workqueue integration, and new format support were outside Pre10. This
|
|
report does not claim that every EROFS feature was tested.
|
|
|
|
## Repository history and identities
|
|
|
|
The relevant pushed commits are:
|
|
|
|
| Commit | Purpose |
|
|
| --- | --- |
|
|
| `cdfcd79640f252830c4e507f65aa7387eaa98ae0` | Snapshot `repo-pre-10` from the current `repo-pre-9`. |
|
|
| `b5d6f5ce0696407f9f1cb3f0a9f8cbab643eea49` | Add the Pre10 execution plan under `planning/pre10/`. |
|
|
| `3c7c774b296a2ee90578a78f372aad5fd2edbec6` | Batch A compatibility cleanup and ordering. |
|
|
| `8c246ea3c4dbf94b92d1f27a9d1b6b6eda582c31` | Batch B bounded metadata helper extraction. |
|
|
| `d3c1cb90b21ae8a97f549bc256d120268e5d09e9` | Record the discovered `insmntque()` vnode lifecycle issue. |
|
|
| `5c8a47e916b1521cb8a35ec4f546e5e16fb8c2dc` | Batch C decompressor request and descriptor alignment. |
|
|
| `9fbf0d7e24084d2342ecd61eff21592b1a83d225` | Fix `insmntque()` failure ownership. |
|
|
| `ff6586c2b24e155a3cbdf55b2ed80edb36d3e406` | Clarify the issue status after the source fix. This is the commit tested by the final smoke run. |
|
|
|
|
The snapshot report records the Pre9 source tree inherited by Pre10 as
|
|
`e657e8a63b097b061670f75ca01947a568ef33d5`. The final smoke runner independently
|
|
recorded these tested identities:
|
|
|
|
```text
|
|
commit=ff6586c2b24e155a3cbdf55b2ed80edb36d3e406
|
|
repo_pre_10_tree=3387e32efbcb0bec7e0bb57ecfc42daf9b962d28
|
|
src_tree=ec684d8dbd7662070234da1fe947aba48a293db8
|
|
```
|
|
|
|
Evidence: `/work/tests-dev/temp/pre10-smoke-final-20260813T102543Z/dut-identities.txt`.
|
|
|
|
## Batch A: compatibility cleanup and ordering
|
|
|
|
Batch A changed `src/internal.h`, `src/data.c`, and `src/zmap.c`.
|
|
|
|
Repository-wide consumer searches proved that the following declarations did
|
|
not represent implemented FreeBSD behavior and had no source, Makefile, ABI,
|
|
initializer, or field-access consumer:
|
|
|
|
- `EROFS_SYNC_DECOMPRESS_*`;
|
|
- `EROFS_ZIP_CACHE_*`;
|
|
- `struct erofs_buf` and `__EROFS_BUF_INITIALIZER`;
|
|
- the unused `erofs_map_blocks.buf` member; and
|
|
- `erofs_is_fileio_mode()`.
|
|
|
|
Removing these declarations improves maintainability because the FreeBSD tree
|
|
no longer advertises Linux mechanisms that it does not implement. The real
|
|
FreeBSD synchronous contiguous-buffer path and the Pre9 mount-owned decoded
|
|
LZMA extent cache remain explicit and unchanged.
|
|
|
|
Declarations and existing function definitions were grouped by actual
|
|
responsibility, closer to the Linux source's readable organization where the
|
|
responsibilities match. Static call topology was preserved, and moved function
|
|
bodies were compared for equivalence. No return value, allocation flag, lock,
|
|
mapping flag, `bread`/`brelse` ownership rule, or runtime structure field was
|
|
changed.
|
|
|
|
Detailed evidence: `repo-pre-10/docs/pre10-batch-a.md`.
|
|
|
|
## Batch B: bounded FreeBSD helper boundaries
|
|
|
|
Batch B changed `src/super.c`, `src/data.c`, and `src/inode.c`.
|
|
|
|
It extracted three `static` helpers with one direct caller each:
|
|
|
|
- `erofs_read_superblock()` isolates superblock read, fixed-size copy,
|
|
validation, field decode, checksum, generation seed, and compression
|
|
configuration parsing. `erofs_mountfs()` still owns mount allocation, GEOM
|
|
transfer, device scanning, internal inode setup, publication, and the single
|
|
mount cleanup path.
|
|
- `erofs_fill_map_dev()` performs only the final `m_em`, `m_dif`, and `m_pa`
|
|
assignments. Device selection, range checks, overflow checks, flat-device
|
|
semantics, and `ENODEV`/`EINTEGRITY` decisions remain in `erofs_map_dev()`.
|
|
- `erofs_fill_vnode()` performs only inode-derived vnode field setup. FreeBSD
|
|
vnode allocation, locking, mount association, hash insertion, race handling,
|
|
construction state, downgrade, publication, and cleanup remain in
|
|
`erofs_vget()`.
|
|
|
|
These boundaries improve Linux/FreeBSD visual comparability at the function
|
|
responsibility level while deliberately retaining the FreeBSD GEOM, vnode,
|
|
hash, lock, and cleanup contracts. Linux lifecycle names such as `iget` or
|
|
`fill_super` were not adopted where their semantics would be misleading.
|
|
|
|
Detailed evidence: `repo-pre-10/docs/pre10-batch-b.md`.
|
|
|
|
## Batch C: BSD decompressor request dispatch
|
|
|
|
Batch C changed `src/internal.h`, `src/decompressor.c`, `src/lz4.c`,
|
|
`src/decompressor_lzma.c`, `src/decompressor_deflate.c`, and
|
|
`src/decompressor_zstd.c`.
|
|
|
|
It introduced a stack-owned, synchronous, contiguous-buffer
|
|
`z_erofs_decompress_req` and a descriptor table covering LZ4, LZMA, DEFLATE,
|
|
ZSTD, SHIFTED, and INTERLACED. Codec entry points now use the Linux-comparable
|
|
names `z_erofs_lz4_decompress`, `z_erofs_lzma_decompress`,
|
|
`z_erofs_deflate_decompress`, and `z_erofs_zstd_decompress` because their
|
|
functional roles are equivalent.
|
|
|
|
The interface remains BSD-native:
|
|
|
|
- no page, folio, bio, workqueue, XArray, shrinker, asynchronous request, or
|
|
Linux ownership wrapper was introduced;
|
|
- request fields borrow the existing compressed and decoded buffers only for
|
|
the synchronous callback;
|
|
- `zdata.c` still owns buffer release and LZMA decoded-cache publication;
|
|
- plain SHIFTED/INTERLACED transforms remain in the existing dispatch model;
|
|
- LZ4 padding behavior, partial-decoding source, LZMA dictionary validation,
|
|
ZSTD conditional availability, and codec resource release remain unchanged;
|
|
- configuration I/O errors preserve priority and backend failures remain
|
|
normalized to the existing `EIO` result.
|
|
|
|
This gives third-party maintainers a recognizable request/descriptor and codec
|
|
naming shape without importing Linux kernel runtime assumptions.
|
|
|
|
Detailed evidence: `repo-pre-10/docs/pre10-batch-c.md`.
|
|
|
|
## P1 vnode lifecycle finding and fix
|
|
|
|
Independent Batch B review found a pre-existing P1 ownership violation in
|
|
`erofs_vget()`: after `insmntque()` returned an error, the old branch wrote
|
|
through `vp->v_data` even though FreeBSD may already have executed `vgone()`
|
|
and `vput()` and ended caller ownership of that vnode.
|
|
|
|
Commit `9fbf0d7e24084d2342ecd61eff21592b1a83d225` applied the minimal ownership
|
|
fix. The error branch now frees only the independently allocated `en`, clears
|
|
`*vpp`, returns the original error, and performs no further access, unlock,
|
|
reclaim, or release operation on `vp`. The successful path, vnode hash race,
|
|
lock state, errno, and later reclaim path were not changed.
|
|
|
|
Static ownership review confirms that the kernel failure path clears
|
|
`v_data`, installs `dead_vnodeops`, and owns vnode cleanup; dead vnode reclaim
|
|
does not release the filesystem's separately allocated `en`, so exactly one
|
|
caller-side `free(en, M_EROFS)` remains necessary.
|
|
|
|
The source defect is fixed, but runtime closure is still pending. Closing the
|
|
issue requires a dedicated mount/unmount race test that repeatedly creates or
|
|
looks up uncached vnodes while normal and forced unmount are attempted, with
|
|
kernel diagnostics capable of detecting stale vnode access, memory corruption,
|
|
lock misuse, and double release. The ordinary smoke test below does not trigger
|
|
or prove this teardown race.
|
|
|
|
Issue record:
|
|
`repo-pre-10/issues/erofs-vget-insmntque-failure-use-after-release.md`.
|
|
|
|
## Static review result
|
|
|
|
The final static review accepted Batches A, B, and C and the P1 fix. Checks
|
|
included changed-path allowlists, zero-consumer proof, prototype and backend
|
|
symbol uniqueness, descriptor coverage and bounds, normalized moved-function
|
|
body comparison, cleanup/ownership matrices, errno preservation, mount and
|
|
vnode lifecycle ordering, cache ownership, forbidden Linux mechanism searches,
|
|
conflict-marker searches, clean-worktree checks, `git diff --check`, and local
|
|
`HEAD`/`xdm/main` agreement at each pushed boundary.
|
|
|
|
No new P0 or P1 defect was found in the three planned batches. Static review is
|
|
not runtime proof. In particular, it does not close the dedicated vnode
|
|
teardown race test or the older explicit-extent positive-fixture gap recorded
|
|
under `repo-pre-10/issues/`.
|
|
|
|
## Initial inconclusive smoke attempts
|
|
|
|
Two isolated attempts preceded the final PASS. Neither is treated as a DUT
|
|
failure or a PASS.
|
|
|
|
### Attempt 1
|
|
|
|
Evidence directory:
|
|
`/work/tests-dev/temp/pre10-smoke-20260813T100326Z`.
|
|
|
|
The readiness probe began before the QEMU process had actually started, then
|
|
performed repeated SSH handshakes while the TCG guest was still becoming
|
|
responsive. Sources and fixtures were eventually transferred and the build log
|
|
reached the final module link/strip stage, but the long SSH build session ended
|
|
with transport status `255`. No reliable KLD SHA, `kldload`, or `kldstat`
|
|
evidence was produced.
|
|
|
|
The first runner incorrectly continued into the plain case after the failed
|
|
build/load stage. The guest reported `Invalid fstype`, while `results.txt`
|
|
incorrectly recorded `mount_status=0` because the exit status was not captured
|
|
immediately after `mount`. Since module loading was never proven and the
|
|
runner's status recording was unsound, the mount result is not attributable to
|
|
Pre10 and the attempt is **INCONCLUSIVE**, not FAIL or PASS.
|
|
|
|
### Attempt 2
|
|
|
|
Evidence directory:
|
|
`/work/tests-dev/temp/pre10-smoke-retry-20260813T101136Z`.
|
|
|
|
The fresh guest booted and returned a valid FreeBSD identity after intermittent
|
|
connection refusals, banner timeouts, and resets. The orchestration then exited
|
|
almost immediately. The claimed SCP failure was not backed by an actual
|
|
`scp-images.log`; only a failed attempt to tail that missing file remained.
|
|
No build, module load, mount, or hash result exists. This attempt is also
|
|
**INCONCLUSIVE** and provides no DUT verdict.
|
|
|
|
Both attempts stopped only their owned QEMU process, released ports `10040`
|
|
and `10041`, deleted their disposable overlays, preserved the immutable base,
|
|
and left guard PID `26318` and port `9222` unchanged.
|
|
|
|
The infrastructure diagnosis was recorded at
|
|
`/work/tests-dev/fix-todo/pre10-smoke-ssh-banner-timeout.md`. Its original
|
|
`PENDING_FIX` verdict describes those two attempts; the corrected final runner
|
|
below subsequently completed the required targeted smoke.
|
|
|
|
## Corrected fail-fast runner
|
|
|
|
The successful run used a new directory and port `10042`, a fresh disposable
|
|
overlay, 4096 MiB and two TCG CPUs, and a strictly ordered runner:
|
|
|
|
1. Verify exact Git, base-image, fixture, guard, and port identities before
|
|
starting QEMU.
|
|
2. Start QEMU and record its owned PID before beginning readiness checks.
|
|
3. Wait for TCP, require two consecutive successful FreeBSD SSH identity
|
|
checks, and establish one persistent SSH ControlMaster connection.
|
|
4. Transfer the source archive, guest scripts, and all fixtures in one batch,
|
|
then verify fixture hashes in the guest.
|
|
5. Run the guest build as a background job that writes `build.rc` and a
|
|
completion sentinel; poll the sentinel rather than depending on one long
|
|
SSH session.
|
|
6. Stop immediately unless the build return is zero, `erofs.ko` exists, its
|
|
hash is recorded, `kldload` succeeds, and `kldstat -n erofs` proves the
|
|
module is loaded.
|
|
7. Run each filesystem case separately and immediately record attach, mount,
|
|
mount-proof, hash, unmount, and detach return codes. A missing proof or
|
|
nonzero result stops the runner and cannot be converted to PASS.
|
|
8. Review guest cleanup and `dmesg`, unload the module, and prove it is absent.
|
|
9. Use the cleanup trap to stop only the recorded QEMU PID, release only the
|
|
run-owned port and overlay, and revalidate the immutable base and guard VM.
|
|
|
|
Runner evidence:
|
|
`/work/tests-dev/temp/pre10-smoke-final-20260813T102543Z/runner.sh` and
|
|
`runner.log`.
|
|
|
|
## Final targeted smoke result
|
|
|
|
Final evidence directory:
|
|
`/work/tests-dev/temp/pre10-smoke-final-20260813T102543Z`.
|
|
|
|
Overall result: **PASS** for the targeted Pre10 smoke scope.
|
|
|
|
### Isolation
|
|
|
|
- Immutable base image mode before and after: `0444`.
|
|
- Immutable base image SHA-256 before and after:
|
|
`67f359621f23a1d745f0889370cbb99a096cee3e99a0b2f3bb18fc7a91bf6fef`.
|
|
- The `.bp` was not booted or modified directly; the run used a disposable
|
|
qcow2 overlay.
|
|
- Guard PID: `26318`; guard SSH port: `9222`.
|
|
- Guard command identity and `/proc` start value `1102996924` matched before
|
|
and after, and port `9222` remained reachable.
|
|
|
|
Evidence: `base-before.txt`, `cleanup-status.txt`, `guard-before.txt`,
|
|
`guard-command-before.txt`, `guard-command-after.txt`,
|
|
`guard-start-before.txt`, and `guard-start-after.txt` in the final evidence
|
|
directory.
|
|
|
|
### KLD build, load, and unload
|
|
|
|
- FreeBSD guest build return: `0`.
|
|
- Built KLD SHA-256:
|
|
`4ce2d44a0902502a40f07805606485c99869371b3cbab1c70ee19a8ac29d90f1`.
|
|
- The loaded module hash matched the built module hash.
|
|
- `kldstat` showed `erofs.ko` loaded as module ID `5`.
|
|
- `kldunload` succeeded, and the final `kldstat -n erofs` correctly reported
|
|
that no such loaded module remained.
|
|
|
|
Evidence: `build-result.out`, `build-result.rc`, `module-load.out`,
|
|
`module-load.rc`, and `guest-evidence.tar.gz`.
|
|
|
|
### Filesystem cases
|
|
|
|
| Case | Mounted file SHA-256 | Time | Result |
|
|
| --- | --- | ---: | --- |
|
|
| Plain `/data.txt` | `056f8f7585667dc695e2edf936deaf84cfee0f88671ba6cd5be22ce890763433` | `0s` | PASS |
|
|
| LZ4 `/compressed.bin` | `3ff012b76087c4da65ce0b69813a76f47ea367e95782edf8b8cd6cd3ec4d1880` | `5s` | PASS |
|
|
| LZMA full `/level.dat` | `ddda39737f0f6093e828a032ec161511fefbb1fa361bc6cbffdbc91e48e4c461` | `2s` | PASS |
|
|
|
|
Each case recorded zero returns for md attach, EROFS mount, mount proof,
|
|
full-file hash, unmount, and md detach. The exact fixture image hashes were:
|
|
|
|
```text
|
|
plain image 3785ca07e7bd16f6c611191596ae0314253c0ae9b7217a4b22de25799b0f08ac
|
|
LZ4 image 967cc1b625546f9f9f881472e71cc3d849c65feb4e98e67fbfdc9b90a4be6dda
|
|
LZMA image 32107a084b27362a093768b88746c37c2e99a74b9f1301a9d4046988479defe9
|
|
```
|
|
|
|
Evidence: `fixture-host-sha256.txt`, `results.txt`, and the per-case files in
|
|
`guest-evidence.tar.gz`.
|
|
|
|
### Kernel messages and cleanup
|
|
|
|
The post-test `dmesg` delta and suspect scan were empty: no EROFS mount,
|
|
integrity, decompression, panic, or assertion error was found. After all cases,
|
|
only the guest root filesystem and `devfs` remained mounted, no test md device
|
|
remained, and the EROFS module was unloaded.
|
|
|
|
Host cleanup also passed:
|
|
|
|
- the run-owned QEMU process stopped;
|
|
- port `10042` was released;
|
|
- the disposable overlay was deleted;
|
|
- the base image mode and hash were unchanged; and
|
|
- the guard VM remained unchanged and reachable.
|
|
|
|
Evidence: `dmesg-delta.txt`, `dmesg-suspect.txt`, `final-guest-state.out`,
|
|
`cleanup-status.txt`, and `guest-evidence.tar.gz`.
|
|
|
|
## Validation limits and conclusion
|
|
|
|
Pre10's planned source changes are complete, independently statically reviewed,
|
|
committed, pushed, built in a FreeBSD guest, and covered by the targeted plain,
|
|
LZ4, and LZMA smoke run. That targeted scope passed at tested commit
|
|
`ff6586c2b24e155a3cbdf55b2ed80edb36d3e406`.
|
|
|
|
No full feature test and no `/work/tests-dev/test_all.py` run was performed.
|
|
This report therefore does not claim all features, codecs, mapping layouts,
|
|
multi-device behavior, partial references, tailpacking, memory pressure, or
|
|
concurrent teardown behavior passed.
|
|
|
|
A feature-specific test is still required to close the fixed `insmntque()`
|
|
issue's runtime-validation status: the dedicated normal/forced mount-unmount
|
|
race with uncached vnode creation and kernel diagnostics described above. It
|
|
is desirable and required for issue closure, but it was not run because the
|
|
instruction for Pre10 was smoke testing only and explicitly excluded feature
|
|
testing.
|