170 lines
6.3 KiB
Markdown
170 lines
6.3 KiB
Markdown
# Explicit Extent Metadata Fixture Unavailable
|
|
|
|
Status: **SHELVED - positive mapped payload unavailable**
|
|
|
|
Last updated: 2026-08-09, final aggregation
|
|
|
|
Latest reviewed baseline: `fcc85b93d5f8fd9671686bd68bf3b086c8bd25cf`
|
|
|
|
Priority: High validation gap
|
|
|
|
Implementation status: ABI and control flow implemented; positive mapped
|
|
kernel read pending
|
|
|
|
## Problem
|
|
|
|
repo22 implements the newer compressed-extent metadata path selected by
|
|
`Z_EROFS_ADVISE_EXTENTS`. Dynamic positive validation requires an image with a
|
|
valid extent table that maps a real payload. erofs-utils 1.8.6 cannot emit or
|
|
validate this on-disk format, and the repository's structured helpers cannot
|
|
currently relocate a legacy compressed inode into a validated positive extent
|
|
table.
|
|
|
|
HEAD2, interlaced pclusters, legacy full/compact indexes, partial references,
|
|
fragments, and all four decoders have separate dynamic coverage. None of those
|
|
substitutes for entering `z_erofs_map_blocks_ext()` with a mapped payload.
|
|
|
|
## Required Trigger
|
|
|
|
1. The inode uses compressed-full layout.
|
|
2. The map header sets `Z_EROFS_ADVISE_EXTENTS`.
|
|
3. The selected 4-, 8-, 16-, or 32-byte records form a complete valid table.
|
|
4. Physical fields identify real payload bytes for the declared algorithm.
|
|
5. Logical starts, physical lengths, partial flags, and final-record semantics
|
|
agree with the source file.
|
|
6. A FreeBSD read reaches the explicit mapper and returns source-identical
|
|
bytes.
|
|
|
|
Zero-count tables, overflowing physical bases, holes, and ordinary full-index
|
|
images are negative or adjacent coverage, not this trigger.
|
|
|
|
## G5 Independent Attempts
|
|
|
|
All inputs below were generated in new G5 output directories. No old image or
|
|
report was consumed as a fixture.
|
|
|
|
### 1. erofs-utils 1.8.6 capability scan
|
|
|
|
Observed tool output:
|
|
|
|
```text
|
|
mkfs.erofs (erofs-utils) 1.8.6
|
|
available compressors: lz4, lz4hc, lzma, deflate, libdeflate, zstd
|
|
```
|
|
|
|
The helper searched the installed 1.8.6 `include/` and `lib/` trees for these
|
|
on-disk symbols:
|
|
|
|
```text
|
|
Z_EROFS_ADVISE_EXTENTS: 0 matches
|
|
z_erofs_extent_recsize: 0 matches
|
|
struct z_erofs_extent {: 0 matches
|
|
```
|
|
|
|
The mkfs help has no explicit-record option. `--max-extent-bytes` only limits
|
|
decompressed extent size.
|
|
|
|
Result: **generator unavailable**.
|
|
|
|
### 2. `--max-extent-bytes` generation attempt
|
|
|
|
`g5_fixtures.py` generated a fresh LZ4 full-index image with:
|
|
|
|
```text
|
|
-zlz4 -C65536 -Elegacy-compress --max-extent-bytes=65536
|
|
```
|
|
|
|
Structured reopen and `dump.erofs` produced:
|
|
|
|
```text
|
|
image = images/extent-attempt.erofs
|
|
image SHA256 = ee7472c23ccffd5eef6f4a3171ea53ae2e840f8a1ea417b2630065175ecf3225
|
|
source SHA256 = 5be510e6b43f1ed0cda4261288ea70df11607b6ced29bc90d32ed2849ecc5e35
|
|
inode layout = 1 (compressed full)
|
|
map header offset = 1312
|
|
h_advise = 2
|
|
HEAD1 algorithm = 0 (LZ4)
|
|
explicit bit selected = no
|
|
```
|
|
|
|
`h_advise=2` is ordinary HEAD1 big-pcluster metadata. Naming or sizing the
|
|
image as an extent attempt does not enter the explicit mapper.
|
|
|
|
Result: **not an explicit fixture**.
|
|
|
|
### 3. Structured conversion attempt
|
|
|
|
The G5 transformer parsed the source inode, map header, full lcluster indexes,
|
|
physical extents, and payload bounds. It refused to emit a purported positive
|
|
fixture because conversion requires relocating variable-size extent records,
|
|
subsequent inode metadata, and possibly payload blocks. erofs-utils 1.8.6
|
|
cannot reopen and validate the resulting ABI, so an ad hoc rewrite would not
|
|
provide trustworthy evidence.
|
|
|
|
The existing `tests/review_fixtures.py` helper was also reviewed. Its
|
|
`extent-pa-wrap.erofs` conversion deliberately writes two 4-byte records and a
|
|
physical base whose `pa + plen` overflows. That is a negative overflow trigger
|
|
for TC155. It does not describe a valid mapped payload and cannot close this
|
|
issue.
|
|
|
|
Result: **structured positive conversion unavailable**.
|
|
|
|
### 4. ABI and control-flow comparison
|
|
|
|
The G5 review compared repo22 `src/erofs_fs.h`/`src/zmap.c` with the Linux EROFS
|
|
definitions and mapper. Both sides define:
|
|
|
|
- record sizes 4, 8, 16, and 32 bytes;
|
|
- implicit 64-bit physical bases for 4-byte records;
|
|
- per-record low physical starts for 8-byte records;
|
|
- explicit counts and binary search for 16/32-byte records;
|
|
- high physical and logical words where the record size carries them;
|
|
- partial-reference, shifted/interlaced format, and final fragment handling;
|
|
- malformed-count and arithmetic bounds checks.
|
|
|
|
This reduces implementation risk but is static evidence only.
|
|
|
|
## Impact
|
|
|
|
- TC146 HEAD2: dynamic PASS.
|
|
- TC146 interlaced: dynamic PASS.
|
|
- TC146 explicit mapped payload: SHELVED.
|
|
- TC146 overall: **PARTIAL**, never overall PASS while this issue remains.
|
|
- Record-size variants, positive binary-search transitions, mapped high words,
|
|
and explicit-record partial references remain dynamically unvalidated.
|
|
|
|
No repo22 kernel source was changed because this is a fixture/tooling gap, not
|
|
an observed kernel failure.
|
|
|
|
## Progress
|
|
|
|
- [x] Reproduce tool limitation against exactly erofs-utils 1.8.6.
|
|
- [x] Record tool output and zero symbol matches.
|
|
- [x] Generate and inspect a fresh `--max-extent-bytes` attempt.
|
|
- [x] Record source/image hashes and map-header fields.
|
|
- [x] Audit the existing structured negative helper.
|
|
- [x] Compare FreeBSD and Linux record layouts/control flow.
|
|
- [x] Reject globally unordered 16-byte and 32-byte explicit tables through
|
|
TC157 on FreeBSD 15 without reading the referenced payload.
|
|
- [ ] Obtain a producer or validator for positive mapped extent records.
|
|
- [ ] Generate at least one source-identical mapped payload.
|
|
- [ ] Cover 4/8/16/32-byte positive records where applicable.
|
|
- [ ] Exercise binary-search transitions, partial refs, high words, and final
|
|
fragments dynamically.
|
|
|
|
## Acceptance Criteria
|
|
|
|
This issue can close only after a reproducible helper emits a valid mapped
|
|
compressed payload, records its structural fields and hashes, validates it
|
|
with an independent format-aware implementation, and FreeBSD full/boundary
|
|
reads match the source. Negative overflow/hole fixtures and static review do
|
|
not satisfy acceptance.
|
|
|
|
## Final Residual Risk
|
|
|
|
TC157 closes the known binary-search ordering bug for descending, duplicate,
|
|
and cross-branch 16-byte and 32-byte tables. It does not supply the missing
|
|
positive mapped payload, dynamically cover every record-size variant, or
|
|
benchmark extremely large extent counts. Those limits do not change the issue
|
|
status or TC146's PARTIAL result.
|