test code v1

This commit is contained in:
2026-08-13 10:44:59 +02:00
commit f3b1165f19
301 changed files with 37885 additions and 0 deletions
+15
View File
@@ -0,0 +1,15 @@
# Issue Status
This directory preserves both resolved investigations and approved validation
gaps. Resolved files remain historical evidence; they are not current failures.
| Issue | Status | Current conclusion |
| --- | --- | --- |
| [TC010 48-bit statfs](TC010-48bit-statfs-large-provider.md) | RESOLVED | A qualified 16 TiB-plus sparse provider completed mount and 64-bit `statfs` validation. |
| [TC060 pathconf](TC060-pathconf-standard-values.md) | RESOLVED | FreeBSD-standard values were implemented and dynamically revalidated. |
| [TC153 large directory index](TC153-large-directory-block-index-validation.md) | RESOLVED | The exact fixed KLD returned `EINTEGRITY` on the multi-TiB sparse-provider path. |
| [Explicit mapped extent payload](extent-metadata-fixture-unavailable.md) | SHELVED | TC146 remains PARTIAL because no independently validated positive mapped-payload fixture is available. |
The remaining shelved item records its trigger, affected feature, analysis,
attempts, results, progress, and acceptance criteria. It is a fixture/tooling
gap, not an observed kernel failure.
@@ -0,0 +1,92 @@
# TC010 48-bit statfs Large Provider
Status: **RESOLVED - qualified sparse vnode provider validated**
Resolved: 2026-08-09
Baseline: `cd0e985b5ac54a4b7acb7042422329ad1729fb3e`
## Original Problem
TC010 requires a real EROFS mount whose 48-bit superblock block count has a
nonzero high word. A nonzero `blocks_hi` declares at least `2^32` filesystem
blocks, so a 4 KiB filesystem needs a provider larger than 16 TiB. Earlier
vnode-md work used only small regular images and correctly could not claim PASS.
This affects:
- 48-bit superblock block-count decoding;
- primary-device media-size validation;
- FreeBSD `statfs(2)` and `df(1)` 64-bit totals;
- the union selector between `rb.blocks_hi` and `rb.rootnid_2b`.
## Exact Trigger
1. `EROFS_FEATURE_INCOMPAT_48BIT` is set.
2. `rootnid_8b` is nonzero, selecting `rb.blocks_hi`.
3. `blocks_hi` is nonzero.
4. Provider media size is at least `total_blocks << blkszbits`.
5. Superblock CRC32C, root inode, and normal mount checks pass.
A short provider fails with `ENXIO`; that negative guard cannot replace the
positive statfs test. TC150's `rootnid_8b == 0` fallback and multidevice totals
also select different production paths.
## Earlier Attempts
1. Structured metadata patching and CRC32C recomputation succeeded.
2. The normal small vnode provider failed media-size validation as expected.
3. Static source comparison confirmed Linux's `48BIT && rootnid_8b` selector,
but static arithmetic was not counted as dynamic coverage.
4. No earlier run had attached and mounted a qualified 16 TiB provider, so the
issue remained SHELVED.
## Resolution
The isolated FreeBSD 15.0-RELEASE-p8 guest uses UFS2, which supports a sparse
regular file large enough for vnode md without allocating 16 TiB physically:
```text
truncate -s 17592193667072 TC010-provider.raw
stat: size=17592193667072 allocated_blocks=15232 block_size=32768
mdconfig -a -t vnode -f TC010-provider.raw: md0
diskinfo mediasize: 17592193667072 bytes
```
The structured fixture encoded:
```text
blocks_lo=1861
blocks_hi=1
rootnid_8b=36
total_blocks=4294969157
required_provider_length=17592193667072
```
Prefix SHA256:
`d3ffadc6fc9e73f85a8a5973b4ac5ee2a2da57b4e8baeccd259fa4325a2146cf`.
The exact prefix was copied to the sparse provider, then extended with zeros.
Mount succeeded with KLD SHA256
`8349c97c7ced253fff32a9e706f29313f309cb63a270e4e39a4501426f2b95c6`.
`df` reported:
```text
Filesystem Type 1024-blocks Used Avail Capacity
/dev/md0 erofs 17179876628 17179876628 0 100%
```
`17179876628 == 4294969157 * 4`; no 32-bit wrap occurred. `f_bfree` and
available blocks were zero, so Used correctly equaled total. The control file
matched, dmesg had no new fault, and the mount, md unit, sparse provider, and
KLD were cleaned up.
## Related Coverage
- TC017 separately repeated the small-provider `ENXIO` guard and qualified
sparse-provider mount, proving the nonzero high-word parse dynamically.
- TC018 separately read exact bytes from physical offset
`17592191561728`, above 16 TiB; it did not substitute a large `df` result for
high-address data I/O.
No repo22 kernel source change was made during this resolution.
+119
View File
@@ -0,0 +1,119 @@
# TC060 Standard pathconf Values
Status: **RESOLVED - standard FreeBSD pathconf values validated**
Last updated: 2026-08-09
Resolved: 2026-08-09
Priority: High
Implementation status: Fixed by `cdba7e54fb9e9d82980a06f178e68d0bbc1663ac`
and dynamically validated
## Problem
The exact repo22 9ae22009f KLD returns EINVAL for two standard pathconf names on
an otherwise valid mounted EROFS directory:
- _PC_NO_TRUNC
- _PC_CHOWN_RESTRICTED
The same call correctly returns NAME_MAX=255, PATH_MAX=1024,
FILESIZEBITS=64, and LINK_MAX=2147483647. TC060 therefore fails at the kernel
behavior level; this is not a missing guest tool or fixture limitation.
## Trigger and Evidence
Environment:
- FreeBSD 15.0-RELEASE-p8 amd64 guest on port 9222.
- Exact source commit: 9ae22009f23a65320730072a780998e80aa9b728.
- KLD SHA256:
19ad086bd2508cbb4c57b1a51f38c93057d438b84fbcfa2e637ff2519f5bc590.
- Fixture: vfs-plain.erofs SHA256
79f0b5f4aa8e7ea532b711ee8fb466f14f35d0952446d41ba4bbc4d6e008b8ff.
Direct command:
/tmp/repo22-g3/g3_vfs_probe pathconf \
/tmp/repo22-g3/mnt/testdir
Observed output and process status:
name_max=255 path_max=1024 filesizebits=64 link_max=2147483647 \
no_trunc=error:22 chown_restricted=error:22
g3_vfs_probe: unexpected EROFS pathconf values
probe_status=1
The test used a fresh dynamic md0, a read-only EROFS mount, and the exact KLD.
After capture, umount and md detach both returned zero. Final guest state was
zero EROFS mounts, zero md units, and no loaded EROFS KLD.
## Analysis
erofs_pathconf() handles NAME_MAX, PATH_MAX, FILESIZEBITS, LINK_MAX, and ACL
queries directly, then delegates other names to vop_stdpathconf(). In this
FreeBSD 15 configuration the delegation returns EINVAL for both names above.
The mounted filesystem is immutable and enforces 255-byte names, so reporting
NO_TRUNC=1 and CHOWN_RESTRICTED=1 is consistent with the implemented behavior
and with the TC contract.
This failure affects applications that use pathconf(2) to discover pathname
truncation and ownership-change restrictions. It does not affect the four
values that repo22 already handles directly.
## Original Required Resolution
Add explicit FreeBSD vnode pathconf handling for both standard names, then
rerun TC060 with the direct syscall helper. Acceptance requires both values to
be 1, all six queries to have errno 0, unchanged mount/read behavior, no new
dmesg diagnostics, and complete mount/md/KLD cleanup.
## Resolution
FreeBSD 15's `vop_stdpathconf()` intentionally provides only generic values
such as `_PC_ASYNC_IO`, `_PC_PATH_MAX`, and zero-valued optional features; its
default case returns `EINVAL`. UFS, tmpfs, and ext2fs therefore implement
`_PC_CHOWN_RESTRICTED` and `_PC_NO_TRUNC` in their filesystem pathconf methods
before delegating all remaining names to `vop_stdpathconf()`.
EROFS now follows that FreeBSD pattern: the two names return 1, while the
existing NAME_MAX, PATH_MAX, FILESIZEBITS, LINK_MAX, and ACL cases remain
unchanged and the default branch still calls `vop_stdpathconf()`. This matches
the implemented filesystem behavior: uid/gid mutation is rejected with EROFS,
and lookup of a component longer than `EROFS_NAME_LEN` returns ENAMETOOLONG.
No lock, allocation, reference, or cleanup path was added.
Linux EROFS was used only as a maintenance comparison for the 255-byte name
limit and ENAMETOOLONG behavior. FreeBSD 15 VOP and syscall semantics were the
authority for the returned values and errno behavior.
## Qualified Retest
The exact source archive for the fix commit built natively on FreeBSD
15.0-RELEASE-p8 with kernel `-Werror` in both configurations:
- `WITH_ZSTDIO=0`: KLD SHA256
`68536e03ce93c6c04aab9cf29dab81ee5802d4b94401bd1a4bd752de40c0e504`.
- `WITH_ZSTDIO=1`: KLD SHA256
`598f171d355af78c64407a6d513d20f053530620da92df7f8ccfdf9a804e463d`.
The dependency-minimal `WITH_ZSTDIO=0` KLD and the original TC060 helper
reported:
```text
name_max=255 path_max=1024 filesizebits=64 link_max=2147483647
no_trunc=1 chown_restricted=1
```
All six queries had errno 0. Additional direct checks returned ASYNC_IO=200112,
ACL_EXTENDED=1, ACL_PATH_MAX=254, and ACL_NFS4=0 with errno 0. An unknown name
returned `-1/EINVAL(22)`, and a 256-byte component returned
`ENAMETOOLONG(63)`. The mounted file hash and read-only EROFS behavior were
unchanged, dmesg was byte-identical before and after, and final mount/md/KLD
and guest artifact counts were zero.
Complete build, value, smoke, discarded-attempt, review, and cleanup evidence
is in
`tests/results/manual/2026-08-09T1120Z-tc060-fix/manual-test-report.md`.
@@ -0,0 +1,193 @@
# TC153 Large Directory Block Index Validation
Status: **RESOLVED - exact-source kernel validation passed**
Last updated: 2026-08-09
Priority: Critical validation gap
Implementation status: Source fix present, build-verified, and dynamically
validated
## Problem
`erofs_find_target_block()` used signed `int` values for the binary-search
front, back, and midpoint indexes. A directory whose logical size describes
`2147483649` 4 KiB blocks produces a final block index of `2147483648`.
Converting that value to `int` can make the initial back bound negative. The
search can then return `ENOENT` without reading any directory block, and the
FreeBSD namecache can retain a false negative result for corrupt media.
The review change uses `uint64_t` for block-search bounds and midpoints, checks
the midpoint multiplication, and handles the `mid == 0` lower-bound case. The
within-block search remains `uint32_t` because a validated EROFS directory
block can contain only a block-sized number of entries.
## Affected Features
- Cold pathname lookup in very large or maliciously sized directories.
- Corruption reporting and `EINTEGRITY` propagation from directory reads.
- Negative namecache correctness after a failed lookup.
- Linux/FreeBSD behavioral review of EROFS's two-level directory search.
Normal directories are not expected to approach this bound. The practical
risk is a crafted image causing an incorrect `ENOENT`, not an ordinary mkfs
image losing entries.
## Trigger Conditions
All of the following are needed to exercise the original width error:
1. A directory inode is accepted with a logical block count greater than
`INT_MAX`.
2. The directory layout reaches `erofs_find_target_block()` rather than being
rejected by an earlier inline-data bounds check.
3. A cold lookup is issued for a name that is not satisfied by an already
cached vnode or negative namecache entry.
4. The backing image is too short for the computed midpoint read, so the fixed
code should return `EINTEGRITY` instead of a synthetic miss.
For 4 KiB blocks, TC153 encodes a size of `8796093026304` bytes,
`2147483649` blocks, and a final index of `2147483648`.
## Current Analysis
The source change is mechanically narrow and has passed both repo22 build
configurations. The checked multiplication protects a future block-size or
index change even though the current `OFF_MAX` inode limit already bounds the
product. The `mid == 0` guard prevents unsigned subtraction from wrapping when
the search moves below the first block.
Dynamic proof must show more than a large `st_size`. It must prove that the
lookup enters the block binary search. An image rejected in
`erofs_read_inode()` is not evidence for this fix, and an `ENOENT` observed
after a warm negative cache is ambiguous.
## Attempts and Results
### Attempt 1: Small FLAT_INLINE image
- Artifact directory:
`/work/build/repo22-review-artifact.LtqNxH`
- Base image: `large-dir-base.erofs`, 4096 bytes, SHA256
`973d2a1c90ac1bf776ed76a1fc6a7e88b2156fac3bf04ddbae19cfafebbd562e`
- Patched image: `large-dir-intmax.erofs`, 4096 bytes, SHA256
`ce5cad3eff34ea50ca89eedf93d8ef90e6cea96858a138685fae45f39084de1f`
- Recorded inode: NID 40 at byte 1280.
The directory was Layout 2 (`FLAT_INLINE`). Patching only `i_size` made its
declared inline tail range invalid, so inode validation rejected it before
`erofs_find_target_block()`. This attempt is invalid as TC153 dynamic evidence
and must never be marked PASS.
### Attempt 2: More entries to force block data
- Artifact directory:
`/work/build/repo22-review-artifact2.FmMTup`
- Generated base: `large-dir-base.erofs`, 65536 bytes, SHA256
`50eb28351788ab1801408aca0a6fca6352755f5ca56efe6e2bb3e2fe08305663`
- Directory: NID 40, 21052 bytes, still Layout 2.
The generation process was interrupted before producing a patched image or a
FreeBSD result. A later deterministic rerun reproduced the same base hash and
showed why the approach failed: erofs-utils 1.8.6 tail-packs directories even
when they span multiple data blocks. Its `noinline_data` option does not turn
directory tail packing off.
### Attempt 3: Follow-up agents
Multiple follow-up agents were assigned the fixture and guest validation.
They either stopped while preparing the second image or returned no executed
commands, files, hashes, or guest output. These attempts added no evidence and
are recorded to prevent their elapsed time from being mistaken for progress.
### Attempt 4: Sparse-provider design
A guest-side sparse vnode provider was considered so a valid directory could
declare the complete multi-terabyte range. It was not executed. Such a scheme
must prove the GEOM media size, avoid materializing terabytes, preserve valid
initial directory blocks, and still force a cold read at the binary-search
midpoint. No result exists for this approach.
### Attempt 5: Reproducible FLAT_PLAIN conversion
The tracked generator now copies all 21052 bytes of the generated directory
to appended blocks, changes only that inode to Layout 0 (`FLAT_PLAIN`), updates
the image block count and CRC32C, and then applies the large logical size.
Host reproduction on 2026-08-09 produced:
- `large-dir-base.erofs`: 65536 bytes, SHA256
`50eb28351788ab1801408aca0a6fca6352755f5ca56efe6e2bb3e2fe08305663`
- `large-dir-intmax.erofs`: 90112 bytes, SHA256
`0f90d3d57adbbbd946e41b225c1f6c464915c6abb0b13478ec9b2a318def1f72`
- Patched inode: NID 40 at byte 1280, Layout 0, raw block 16.
- Declared image blocks: 22.
This resolved the fixture-construction problem. The qualified kernel run is
recorded below.
### Attempt 6: Qualified sparse GEOM provider
The G3 takeover generated a checksum-valid sparse prefix from Attempt 5 and
performed one targeted FreeBSD 15 run against the exact source requested by
the assignment.
- Source commit:
9ae22009f23a65320730072a780998e80aa9b728.
- KLD SHA256:
19ad086bd2508cbb4c57b1a51f38c93057d438b84fbcfa2e637ff2519f5bc590.
- Sparse-prefix SHA256:
f9337f83b1f568f6d904331a7749e6362a691055cd5af95b59bc89772f04e3b0.
- Prefix qualification: NID 40, inode offset 1280, extended Layout 0,
start block 16, 2147483649 directory blocks, final block index 2147483648,
valid superblock checksum.
- Sparse provider: 8796093091840 logical bytes, 448 allocated 512-byte
sectors, and the first 90112 bytes retained the prefix SHA256.
- GEOM diskinfo size: 8796093091840 bytes.
- Mounted /huge: size 8796093026304, NID 40.
The first two cold stat operations for /huge/missing each returned EINTEGRITY
(97). Root readdir then returned three complete entries and validated all
kernel/libc restart cookies. A third lookup after that readdir again returned
EINTEGRITY. No lookup returned ENOENT, so no false negative namecache entry
masked the corruption.
Unmount, md detach, sparse-provider removal, and KLD unload all succeeded.
The only new dmesg lines in the complete G3 run were expected SIGBUS child
exits from the unrelated assigned mmap tests; TC153 added no diagnostic.
## Historical Remaining Validation
Attempt 6 completed steps 1-5 and 7 below. The assignment mandated the exact
fixed source, so the optional pre-fix comparison in step 6 was not run.
1. Regenerate the images with
`tests/results/manual/2026-08-09T0124Z-final-review/prepare-fixtures.sh` and
verify the hashes and Layout 0 assertion.
2. Load the exact module under test in a clean FreeBSD 15 guest.
3. Mount the 90112-byte image and confirm `/huge` reports
`8796093026304` bytes.
4. With a cold parent cache, run two lookups of `/huge/missing` and capture
direct command exit codes plus `truss` errno.
5. Require `EINTEGRITY` on every fixed-module lookup, including after root
readdir. Confirm no negative cache changes the result.
6. Repeat with the pre-fix module to demonstrate the erroneous `ENOENT` path.
7. Record dmesg, mount/md/KLD cleanup, module and fixture hashes in a dated
manual report.
## Original Acceptance Criteria
TC153 can move from SHELVED to PASS only after the fixed FreeBSD KLD returns
`EINTEGRITY` repeatedly from the valid Layout 0 fixture, the pre-fix behavior
is distinguished, no trap or panic occurs, and complete cleanup evidence is
recorded. Host fixture generation and static source review alone are
insufficient.
## Resolution
TC153 is PASS because the exact fixed FreeBSD KLD returned EINTEGRITY
repeatedly from the valid Layout 0 fixture before and after root readdir, no
trap or panic occurred, and complete hash/provider/cleanup evidence is recorded
in tests/results/manual/2026-08-09T1059Z-g3/manual-test-report.md. Host fixture
generation and static source review alone remain insufficient.
@@ -0,0 +1,169 @@
# Explicit Extent Metadata Fixture Unavailable
Status: **SHELVED - positive mapped payload unavailable**
Last updated: 2026-08-09, final aggregation
Latest reviewed baseline: `fcc85b93d5f8fd9671686bd68bf3b086c8bd25cf`
Priority: High validation gap
Implementation status: ABI and control flow implemented; positive mapped
kernel read pending
## Problem
repo22 implements the newer compressed-extent metadata path selected by
`Z_EROFS_ADVISE_EXTENTS`. Dynamic positive validation requires an image with a
valid extent table that maps a real payload. erofs-utils 1.8.6 cannot emit or
validate this on-disk format, and the repository's structured helpers cannot
currently relocate a legacy compressed inode into a validated positive extent
table.
HEAD2, interlaced pclusters, legacy full/compact indexes, partial references,
fragments, and all four decoders have separate dynamic coverage. None of those
substitutes for entering `z_erofs_map_blocks_ext()` with a mapped payload.
## Required Trigger
1. The inode uses compressed-full layout.
2. The map header sets `Z_EROFS_ADVISE_EXTENTS`.
3. The selected 4-, 8-, 16-, or 32-byte records form a complete valid table.
4. Physical fields identify real payload bytes for the declared algorithm.
5. Logical starts, physical lengths, partial flags, and final-record semantics
agree with the source file.
6. A FreeBSD read reaches the explicit mapper and returns source-identical
bytes.
Zero-count tables, overflowing physical bases, holes, and ordinary full-index
images are negative or adjacent coverage, not this trigger.
## G5 Independent Attempts
All inputs below were generated in new G5 output directories. No old image or
report was consumed as a fixture.
### 1. erofs-utils 1.8.6 capability scan
Observed tool output:
```text
mkfs.erofs (erofs-utils) 1.8.6
available compressors: lz4, lz4hc, lzma, deflate, libdeflate, zstd
```
The helper searched the installed 1.8.6 `include/` and `lib/` trees for these
on-disk symbols:
```text
Z_EROFS_ADVISE_EXTENTS: 0 matches
z_erofs_extent_recsize: 0 matches
struct z_erofs_extent {: 0 matches
```
The mkfs help has no explicit-record option. `--max-extent-bytes` only limits
decompressed extent size.
Result: **generator unavailable**.
### 2. `--max-extent-bytes` generation attempt
`g5_fixtures.py` generated a fresh LZ4 full-index image with:
```text
-zlz4 -C65536 -Elegacy-compress --max-extent-bytes=65536
```
Structured reopen and `dump.erofs` produced:
```text
image = images/extent-attempt.erofs
image SHA256 = ee7472c23ccffd5eef6f4a3171ea53ae2e840f8a1ea417b2630065175ecf3225
source SHA256 = 5be510e6b43f1ed0cda4261288ea70df11607b6ced29bc90d32ed2849ecc5e35
inode layout = 1 (compressed full)
map header offset = 1312
h_advise = 2
HEAD1 algorithm = 0 (LZ4)
explicit bit selected = no
```
`h_advise=2` is ordinary HEAD1 big-pcluster metadata. Naming or sizing the
image as an extent attempt does not enter the explicit mapper.
Result: **not an explicit fixture**.
### 3. Structured conversion attempt
The G5 transformer parsed the source inode, map header, full lcluster indexes,
physical extents, and payload bounds. It refused to emit a purported positive
fixture because conversion requires relocating variable-size extent records,
subsequent inode metadata, and possibly payload blocks. erofs-utils 1.8.6
cannot reopen and validate the resulting ABI, so an ad hoc rewrite would not
provide trustworthy evidence.
The existing `tests/review_fixtures.py` helper was also reviewed. Its
`extent-pa-wrap.erofs` conversion deliberately writes two 4-byte records and a
physical base whose `pa + plen` overflows. That is a negative overflow trigger
for TC155. It does not describe a valid mapped payload and cannot close this
issue.
Result: **structured positive conversion unavailable**.
### 4. ABI and control-flow comparison
The G5 review compared repo22 `src/erofs_fs.h`/`src/zmap.c` with the Linux EROFS
definitions and mapper. Both sides define:
- record sizes 4, 8, 16, and 32 bytes;
- implicit 64-bit physical bases for 4-byte records;
- per-record low physical starts for 8-byte records;
- explicit counts and binary search for 16/32-byte records;
- high physical and logical words where the record size carries them;
- partial-reference, shifted/interlaced format, and final fragment handling;
- malformed-count and arithmetic bounds checks.
This reduces implementation risk but is static evidence only.
## Impact
- TC146 HEAD2: dynamic PASS.
- TC146 interlaced: dynamic PASS.
- TC146 explicit mapped payload: SHELVED.
- TC146 overall: **PARTIAL**, never overall PASS while this issue remains.
- Record-size variants, positive binary-search transitions, mapped high words,
and explicit-record partial references remain dynamically unvalidated.
No repo22 kernel source was changed because this is a fixture/tooling gap, not
an observed kernel failure.
## Progress
- [x] Reproduce tool limitation against exactly erofs-utils 1.8.6.
- [x] Record tool output and zero symbol matches.
- [x] Generate and inspect a fresh `--max-extent-bytes` attempt.
- [x] Record source/image hashes and map-header fields.
- [x] Audit the existing structured negative helper.
- [x] Compare FreeBSD and Linux record layouts/control flow.
- [x] Reject globally unordered 16-byte and 32-byte explicit tables through
TC157 on FreeBSD 15 without reading the referenced payload.
- [ ] Obtain a producer or validator for positive mapped extent records.
- [ ] Generate at least one source-identical mapped payload.
- [ ] Cover 4/8/16/32-byte positive records where applicable.
- [ ] Exercise binary-search transitions, partial refs, high words, and final
fragments dynamically.
## Acceptance Criteria
This issue can close only after a reproducible helper emits a valid mapped
compressed payload, records its structural fields and hashes, validates it
with an independent format-aware implementation, and FreeBSD full/boundary
reads match the source. Negative overflow/hole fixtures and static review do
not satisfy acceptance.
## Final Residual Risk
TC157 closes the known binary-search ordering bug for descending, duplicate,
and cross-branch 16-byte and 32-byte tables. It does not supply the missing
positive mapped payload, dynamically cover every record-size variant, or
benchmark extremely large extent counts. Those limits do not change the issue
status or TC146's PARTIAL result.