update
This commit is contained in:
@@ -0,0 +1,347 @@
|
||||
# Pre10 Completion and Targeted Smoke Report
|
||||
|
||||
Status: **PRE10 IMPLEMENTATION COMPLETE; TARGETED SMOKE PASS; FULL FEATURE TEST NOT RUN**
|
||||
|
||||
Date: 2026-08-13
|
||||
|
||||
## Scope and goals
|
||||
|
||||
Pre10 is a controlled maintenance-alignment release based on the final Pre9
|
||||
tree. Its purpose is to make the FreeBSD EROFS implementation easier to compare
|
||||
and maintain alongside the independent Linux EROFS repository without copying
|
||||
Linux-only lifecycle or memory-management models.
|
||||
|
||||
The planned work was limited to three source batches:
|
||||
|
||||
1. Remove unused Linux-shaped compatibility declarations and improve source
|
||||
ordering.
|
||||
2. Extract bounded helpers where Linux and FreeBSD responsibilities are
|
||||
comparable, while retaining FreeBSD ownership and errno behavior.
|
||||
3. Introduce a BSD-native synchronous decompressor request and descriptor
|
||||
boundary with Linux-comparable codec names.
|
||||
|
||||
Full feature validation, CI execution, performance redesign, Linux page/folio/
|
||||
bio/workqueue integration, and new format support were outside Pre10. This
|
||||
report does not claim that every EROFS feature was tested.
|
||||
|
||||
## Repository history and identities
|
||||
|
||||
The relevant pushed commits are:
|
||||
|
||||
| Commit | Purpose |
|
||||
| --- | --- |
|
||||
| `cdfcd79640f252830c4e507f65aa7387eaa98ae0` | Snapshot `repo-pre-10` from the current `repo-pre-9`. |
|
||||
| `b5d6f5ce0696407f9f1cb3f0a9f8cbab643eea49` | Add the Pre10 execution plan under `planning/pre10/`. |
|
||||
| `3c7c774b296a2ee90578a78f372aad5fd2edbec6` | Batch A compatibility cleanup and ordering. |
|
||||
| `8c246ea3c4dbf94b92d1f27a9d1b6b6eda582c31` | Batch B bounded metadata helper extraction. |
|
||||
| `d3c1cb90b21ae8a97f549bc256d120268e5d09e9` | Record the discovered `insmntque()` vnode lifecycle issue. |
|
||||
| `5c8a47e916b1521cb8a35ec4f546e5e16fb8c2dc` | Batch C decompressor request and descriptor alignment. |
|
||||
| `9fbf0d7e24084d2342ecd61eff21592b1a83d225` | Fix `insmntque()` failure ownership. |
|
||||
| `ff6586c2b24e155a3cbdf55b2ed80edb36d3e406` | Clarify the issue status after the source fix. This is the commit tested by the final smoke run. |
|
||||
|
||||
The snapshot report records the Pre9 source tree inherited by Pre10 as
|
||||
`e657e8a63b097b061670f75ca01947a568ef33d5`. The final smoke runner independently
|
||||
recorded these tested identities:
|
||||
|
||||
```text
|
||||
commit=ff6586c2b24e155a3cbdf55b2ed80edb36d3e406
|
||||
repo_pre_10_tree=3387e32efbcb0bec7e0bb57ecfc42daf9b962d28
|
||||
src_tree=ec684d8dbd7662070234da1fe947aba48a293db8
|
||||
```
|
||||
|
||||
Evidence: `/work/tests-dev/temp/pre10-smoke-final-20260813T102543Z/dut-identities.txt`.
|
||||
|
||||
## Batch A: compatibility cleanup and ordering
|
||||
|
||||
Batch A changed `src/internal.h`, `src/data.c`, and `src/zmap.c`.
|
||||
|
||||
Repository-wide consumer searches proved that the following declarations did
|
||||
not represent implemented FreeBSD behavior and had no source, Makefile, ABI,
|
||||
initializer, or field-access consumer:
|
||||
|
||||
- `EROFS_SYNC_DECOMPRESS_*`;
|
||||
- `EROFS_ZIP_CACHE_*`;
|
||||
- `struct erofs_buf` and `__EROFS_BUF_INITIALIZER`;
|
||||
- the unused `erofs_map_blocks.buf` member; and
|
||||
- `erofs_is_fileio_mode()`.
|
||||
|
||||
Removing these declarations improves maintainability because the FreeBSD tree
|
||||
no longer advertises Linux mechanisms that it does not implement. The real
|
||||
FreeBSD synchronous contiguous-buffer path and the Pre9 mount-owned decoded
|
||||
LZMA extent cache remain explicit and unchanged.
|
||||
|
||||
Declarations and existing function definitions were grouped by actual
|
||||
responsibility, closer to the Linux source's readable organization where the
|
||||
responsibilities match. Static call topology was preserved, and moved function
|
||||
bodies were compared for equivalence. No return value, allocation flag, lock,
|
||||
mapping flag, `bread`/`brelse` ownership rule, or runtime structure field was
|
||||
changed.
|
||||
|
||||
Detailed evidence: `repo-pre-10/docs/pre10-batch-a.md`.
|
||||
|
||||
## Batch B: bounded FreeBSD helper boundaries
|
||||
|
||||
Batch B changed `src/super.c`, `src/data.c`, and `src/inode.c`.
|
||||
|
||||
It extracted three `static` helpers with one direct caller each:
|
||||
|
||||
- `erofs_read_superblock()` isolates superblock read, fixed-size copy,
|
||||
validation, field decode, checksum, generation seed, and compression
|
||||
configuration parsing. `erofs_mountfs()` still owns mount allocation, GEOM
|
||||
transfer, device scanning, internal inode setup, publication, and the single
|
||||
mount cleanup path.
|
||||
- `erofs_fill_map_dev()` performs only the final `m_em`, `m_dif`, and `m_pa`
|
||||
assignments. Device selection, range checks, overflow checks, flat-device
|
||||
semantics, and `ENODEV`/`EINTEGRITY` decisions remain in `erofs_map_dev()`.
|
||||
- `erofs_fill_vnode()` performs only inode-derived vnode field setup. FreeBSD
|
||||
vnode allocation, locking, mount association, hash insertion, race handling,
|
||||
construction state, downgrade, publication, and cleanup remain in
|
||||
`erofs_vget()`.
|
||||
|
||||
These boundaries improve Linux/FreeBSD visual comparability at the function
|
||||
responsibility level while deliberately retaining the FreeBSD GEOM, vnode,
|
||||
hash, lock, and cleanup contracts. Linux lifecycle names such as `iget` or
|
||||
`fill_super` were not adopted where their semantics would be misleading.
|
||||
|
||||
Detailed evidence: `repo-pre-10/docs/pre10-batch-b.md`.
|
||||
|
||||
## Batch C: BSD decompressor request dispatch
|
||||
|
||||
Batch C changed `src/internal.h`, `src/decompressor.c`, `src/lz4.c`,
|
||||
`src/decompressor_lzma.c`, `src/decompressor_deflate.c`, and
|
||||
`src/decompressor_zstd.c`.
|
||||
|
||||
It introduced a stack-owned, synchronous, contiguous-buffer
|
||||
`z_erofs_decompress_req` and a descriptor table covering LZ4, LZMA, DEFLATE,
|
||||
ZSTD, SHIFTED, and INTERLACED. Codec entry points now use the Linux-comparable
|
||||
names `z_erofs_lz4_decompress`, `z_erofs_lzma_decompress`,
|
||||
`z_erofs_deflate_decompress`, and `z_erofs_zstd_decompress` because their
|
||||
functional roles are equivalent.
|
||||
|
||||
The interface remains BSD-native:
|
||||
|
||||
- no page, folio, bio, workqueue, XArray, shrinker, asynchronous request, or
|
||||
Linux ownership wrapper was introduced;
|
||||
- request fields borrow the existing compressed and decoded buffers only for
|
||||
the synchronous callback;
|
||||
- `zdata.c` still owns buffer release and LZMA decoded-cache publication;
|
||||
- plain SHIFTED/INTERLACED transforms remain in the existing dispatch model;
|
||||
- LZ4 padding behavior, partial-decoding source, LZMA dictionary validation,
|
||||
ZSTD conditional availability, and codec resource release remain unchanged;
|
||||
- configuration I/O errors preserve priority and backend failures remain
|
||||
normalized to the existing `EIO` result.
|
||||
|
||||
This gives third-party maintainers a recognizable request/descriptor and codec
|
||||
naming shape without importing Linux kernel runtime assumptions.
|
||||
|
||||
Detailed evidence: `repo-pre-10/docs/pre10-batch-c.md`.
|
||||
|
||||
## P1 vnode lifecycle finding and fix
|
||||
|
||||
Independent Batch B review found a pre-existing P1 ownership violation in
|
||||
`erofs_vget()`: after `insmntque()` returned an error, the old branch wrote
|
||||
through `vp->v_data` even though FreeBSD may already have executed `vgone()`
|
||||
and `vput()` and ended caller ownership of that vnode.
|
||||
|
||||
Commit `9fbf0d7e24084d2342ecd61eff21592b1a83d225` applied the minimal ownership
|
||||
fix. The error branch now frees only the independently allocated `en`, clears
|
||||
`*vpp`, returns the original error, and performs no further access, unlock,
|
||||
reclaim, or release operation on `vp`. The successful path, vnode hash race,
|
||||
lock state, errno, and later reclaim path were not changed.
|
||||
|
||||
Static ownership review confirms that the kernel failure path clears
|
||||
`v_data`, installs `dead_vnodeops`, and owns vnode cleanup; dead vnode reclaim
|
||||
does not release the filesystem's separately allocated `en`, so exactly one
|
||||
caller-side `free(en, M_EROFS)` remains necessary.
|
||||
|
||||
The source defect is fixed, but runtime closure is still pending. Closing the
|
||||
issue requires a dedicated mount/unmount race test that repeatedly creates or
|
||||
looks up uncached vnodes while normal and forced unmount are attempted, with
|
||||
kernel diagnostics capable of detecting stale vnode access, memory corruption,
|
||||
lock misuse, and double release. The ordinary smoke test below does not trigger
|
||||
or prove this teardown race.
|
||||
|
||||
Issue record:
|
||||
`repo-pre-10/issues/erofs-vget-insmntque-failure-use-after-release.md`.
|
||||
|
||||
## Static review result
|
||||
|
||||
The final static review accepted Batches A, B, and C and the P1 fix. Checks
|
||||
included changed-path allowlists, zero-consumer proof, prototype and backend
|
||||
symbol uniqueness, descriptor coverage and bounds, normalized moved-function
|
||||
body comparison, cleanup/ownership matrices, errno preservation, mount and
|
||||
vnode lifecycle ordering, cache ownership, forbidden Linux mechanism searches,
|
||||
conflict-marker searches, clean-worktree checks, `git diff --check`, and local
|
||||
`HEAD`/`xdm/main` agreement at each pushed boundary.
|
||||
|
||||
No new P0 or P1 defect was found in the three planned batches. Static review is
|
||||
not runtime proof. In particular, it does not close the dedicated vnode
|
||||
teardown race test or the older explicit-extent positive-fixture gap recorded
|
||||
under `repo-pre-10/issues/`.
|
||||
|
||||
## Initial inconclusive smoke attempts
|
||||
|
||||
Two isolated attempts preceded the final PASS. Neither is treated as a DUT
|
||||
failure or a PASS.
|
||||
|
||||
### Attempt 1
|
||||
|
||||
Evidence directory:
|
||||
`/work/tests-dev/temp/pre10-smoke-20260813T100326Z`.
|
||||
|
||||
The readiness probe began before the QEMU process had actually started, then
|
||||
performed repeated SSH handshakes while the TCG guest was still becoming
|
||||
responsive. Sources and fixtures were eventually transferred and the build log
|
||||
reached the final module link/strip stage, but the long SSH build session ended
|
||||
with transport status `255`. No reliable KLD SHA, `kldload`, or `kldstat`
|
||||
evidence was produced.
|
||||
|
||||
The first runner incorrectly continued into the plain case after the failed
|
||||
build/load stage. The guest reported `Invalid fstype`, while `results.txt`
|
||||
incorrectly recorded `mount_status=0` because the exit status was not captured
|
||||
immediately after `mount`. Since module loading was never proven and the
|
||||
runner's status recording was unsound, the mount result is not attributable to
|
||||
Pre10 and the attempt is **INCONCLUSIVE**, not FAIL or PASS.
|
||||
|
||||
### Attempt 2
|
||||
|
||||
Evidence directory:
|
||||
`/work/tests-dev/temp/pre10-smoke-retry-20260813T101136Z`.
|
||||
|
||||
The fresh guest booted and returned a valid FreeBSD identity after intermittent
|
||||
connection refusals, banner timeouts, and resets. The orchestration then exited
|
||||
almost immediately. The claimed SCP failure was not backed by an actual
|
||||
`scp-images.log`; only a failed attempt to tail that missing file remained.
|
||||
No build, module load, mount, or hash result exists. This attempt is also
|
||||
**INCONCLUSIVE** and provides no DUT verdict.
|
||||
|
||||
Both attempts stopped only their owned QEMU process, released ports `10040`
|
||||
and `10041`, deleted their disposable overlays, preserved the immutable base,
|
||||
and left guard PID `26318` and port `9222` unchanged.
|
||||
|
||||
The infrastructure diagnosis was recorded at
|
||||
`/work/tests-dev/fix-todo/pre10-smoke-ssh-banner-timeout.md`. Its original
|
||||
`PENDING_FIX` verdict describes those two attempts; the corrected final runner
|
||||
below subsequently completed the required targeted smoke.
|
||||
|
||||
## Corrected fail-fast runner
|
||||
|
||||
The successful run used a new directory and port `10042`, a fresh disposable
|
||||
overlay, 4096 MiB and two TCG CPUs, and a strictly ordered runner:
|
||||
|
||||
1. Verify exact Git, base-image, fixture, guard, and port identities before
|
||||
starting QEMU.
|
||||
2. Start QEMU and record its owned PID before beginning readiness checks.
|
||||
3. Wait for TCP, require two consecutive successful FreeBSD SSH identity
|
||||
checks, and establish one persistent SSH ControlMaster connection.
|
||||
4. Transfer the source archive, guest scripts, and all fixtures in one batch,
|
||||
then verify fixture hashes in the guest.
|
||||
5. Run the guest build as a background job that writes `build.rc` and a
|
||||
completion sentinel; poll the sentinel rather than depending on one long
|
||||
SSH session.
|
||||
6. Stop immediately unless the build return is zero, `erofs.ko` exists, its
|
||||
hash is recorded, `kldload` succeeds, and `kldstat -n erofs` proves the
|
||||
module is loaded.
|
||||
7. Run each filesystem case separately and immediately record attach, mount,
|
||||
mount-proof, hash, unmount, and detach return codes. A missing proof or
|
||||
nonzero result stops the runner and cannot be converted to PASS.
|
||||
8. Review guest cleanup and `dmesg`, unload the module, and prove it is absent.
|
||||
9. Use the cleanup trap to stop only the recorded QEMU PID, release only the
|
||||
run-owned port and overlay, and revalidate the immutable base and guard VM.
|
||||
|
||||
Runner evidence:
|
||||
`/work/tests-dev/temp/pre10-smoke-final-20260813T102543Z/runner.sh` and
|
||||
`runner.log`.
|
||||
|
||||
## Final targeted smoke result
|
||||
|
||||
Final evidence directory:
|
||||
`/work/tests-dev/temp/pre10-smoke-final-20260813T102543Z`.
|
||||
|
||||
Overall result: **PASS** for the targeted Pre10 smoke scope.
|
||||
|
||||
### Isolation
|
||||
|
||||
- Immutable base image mode before and after: `0444`.
|
||||
- Immutable base image SHA-256 before and after:
|
||||
`67f359621f23a1d745f0889370cbb99a096cee3e99a0b2f3bb18fc7a91bf6fef`.
|
||||
- The `.bp` was not booted or modified directly; the run used a disposable
|
||||
qcow2 overlay.
|
||||
- Guard PID: `26318`; guard SSH port: `9222`.
|
||||
- Guard command identity and `/proc` start value `1102996924` matched before
|
||||
and after, and port `9222` remained reachable.
|
||||
|
||||
Evidence: `base-before.txt`, `cleanup-status.txt`, `guard-before.txt`,
|
||||
`guard-command-before.txt`, `guard-command-after.txt`,
|
||||
`guard-start-before.txt`, and `guard-start-after.txt` in the final evidence
|
||||
directory.
|
||||
|
||||
### KLD build, load, and unload
|
||||
|
||||
- FreeBSD guest build return: `0`.
|
||||
- Built KLD SHA-256:
|
||||
`4ce2d44a0902502a40f07805606485c99869371b3cbab1c70ee19a8ac29d90f1`.
|
||||
- The loaded module hash matched the built module hash.
|
||||
- `kldstat` showed `erofs.ko` loaded as module ID `5`.
|
||||
- `kldunload` succeeded, and the final `kldstat -n erofs` correctly reported
|
||||
that no such loaded module remained.
|
||||
|
||||
Evidence: `build-result.out`, `build-result.rc`, `module-load.out`,
|
||||
`module-load.rc`, and `guest-evidence.tar.gz`.
|
||||
|
||||
### Filesystem cases
|
||||
|
||||
| Case | Mounted file SHA-256 | Time | Result |
|
||||
| --- | --- | ---: | --- |
|
||||
| Plain `/data.txt` | `056f8f7585667dc695e2edf936deaf84cfee0f88671ba6cd5be22ce890763433` | `0s` | PASS |
|
||||
| LZ4 `/compressed.bin` | `3ff012b76087c4da65ce0b69813a76f47ea367e95782edf8b8cd6cd3ec4d1880` | `5s` | PASS |
|
||||
| LZMA full `/level.dat` | `ddda39737f0f6093e828a032ec161511fefbb1fa361bc6cbffdbc91e48e4c461` | `2s` | PASS |
|
||||
|
||||
Each case recorded zero returns for md attach, EROFS mount, mount proof,
|
||||
full-file hash, unmount, and md detach. The exact fixture image hashes were:
|
||||
|
||||
```text
|
||||
plain image 3785ca07e7bd16f6c611191596ae0314253c0ae9b7217a4b22de25799b0f08ac
|
||||
LZ4 image 967cc1b625546f9f9f881472e71cc3d849c65feb4e98e67fbfdc9b90a4be6dda
|
||||
LZMA image 32107a084b27362a093768b88746c37c2e99a74b9f1301a9d4046988479defe9
|
||||
```
|
||||
|
||||
Evidence: `fixture-host-sha256.txt`, `results.txt`, and the per-case files in
|
||||
`guest-evidence.tar.gz`.
|
||||
|
||||
### Kernel messages and cleanup
|
||||
|
||||
The post-test `dmesg` delta and suspect scan were empty: no EROFS mount,
|
||||
integrity, decompression, panic, or assertion error was found. After all cases,
|
||||
only the guest root filesystem and `devfs` remained mounted, no test md device
|
||||
remained, and the EROFS module was unloaded.
|
||||
|
||||
Host cleanup also passed:
|
||||
|
||||
- the run-owned QEMU process stopped;
|
||||
- port `10042` was released;
|
||||
- the disposable overlay was deleted;
|
||||
- the base image mode and hash were unchanged; and
|
||||
- the guard VM remained unchanged and reachable.
|
||||
|
||||
Evidence: `dmesg-delta.txt`, `dmesg-suspect.txt`, `final-guest-state.out`,
|
||||
`cleanup-status.txt`, and `guest-evidence.tar.gz`.
|
||||
|
||||
## Validation limits and conclusion
|
||||
|
||||
Pre10's planned source changes are complete, independently statically reviewed,
|
||||
committed, pushed, built in a FreeBSD guest, and covered by the targeted plain,
|
||||
LZ4, and LZMA smoke run. That targeted scope passed at tested commit
|
||||
`ff6586c2b24e155a3cbdf55b2ed80edb36d3e406`.
|
||||
|
||||
No full feature test and no `/work/tests-dev/test_all.py` run was performed.
|
||||
This report therefore does not claim all features, codecs, mapping layouts,
|
||||
multi-device behavior, partial references, tailpacking, memory pressure, or
|
||||
concurrent teardown behavior passed.
|
||||
|
||||
A feature-specific test is still required to close the fixed `insmntque()`
|
||||
issue's runtime-validation status: the dedicated normal/forced mount-unmount
|
||||
race with uncached vnode creation and kernel diagnostics described above. It
|
||||
is desirable and required for issue closure, but it was not run because the
|
||||
instruction for Pre10 was smoke testing only and explicitly excluded feature
|
||||
testing.
|
||||
Reference in New Issue
Block a user