erofs-freebsd-out-tree/issues/extent-metadata-fixture-unavailable.md

Explicit Extent Metadata Fixture Unavailable

Status: SHELVED - positive mapped payload unavailable

Last updated: 2026-08-09, final aggregation

Latest reviewed baseline: fcc85b93d5f8fd9671686bd68bf3b086c8bd25cf

Priority: High validation gap

Implementation status: ABI and control flow implemented; positive mapped kernel read pending

Problem

repo22 implements the newer compressed-extent metadata path selected by Z_EROFS_ADVISE_EXTENTS. Dynamic positive validation requires an image with a valid extent table that maps a real payload. erofs-utils 1.8.6 cannot emit or validate this on-disk format, and the repository's structured helpers cannot currently relocate a legacy compressed inode into a validated positive extent table.

HEAD2, interlaced pclusters, legacy full/compact indexes, partial references, fragments, and all four decoders have separate dynamic coverage. None of those substitutes for entering z_erofs_map_blocks_ext() with a mapped payload.

Required Trigger

1. The inode uses compressed-full layout.
2. The map header sets Z_EROFS_ADVISE_EXTENTS.
3. The selected 4-, 8-, 16-, or 32-byte records form a complete valid table.
4. Physical fields identify real payload bytes for the declared algorithm.
5. Logical starts, physical lengths, partial flags, and final-record semantics agree with the source file.
6. A FreeBSD read reaches the explicit mapper and returns source-identical bytes.

Zero-count tables, overflowing physical bases, holes, and ordinary full-index images are negative or adjacent coverage, not this trigger.

G5 Independent Attempts

All inputs below were generated in new G5 output directories. No old image or report was consumed as a fixture.

1. erofs-utils 1.8.6 capability scan

Observed tool output:

mkfs.erofs (erofs-utils) 1.8.6
available compressors: lz4, lz4hc, lzma, deflate, libdeflate, zstd

The helper searched the installed 1.8.6 include/ and lib/ trees for these on-disk symbols:

Z_EROFS_ADVISE_EXTENTS: 0 matches
z_erofs_extent_recsize: 0 matches
struct z_erofs_extent {: 0 matches

The mkfs help has no explicit-record option. --max-extent-bytes only limits decompressed extent size.

Result: generator unavailable.

2. --max-extent-bytes generation attempt

g5_fixtures.py generated a fresh LZ4 full-index image with:

-zlz4 -C65536 -Elegacy-compress --max-extent-bytes=65536

Structured reopen and dump.erofs produced:

image                 = images/extent-attempt.erofs
image SHA256          = ee7472c23ccffd5eef6f4a3171ea53ae2e840f8a1ea417b2630065175ecf3225
source SHA256         = 5be510e6b43f1ed0cda4261288ea70df11607b6ced29bc90d32ed2849ecc5e35
inode layout          = 1 (compressed full)
map header offset     = 1312
h_advise              = 2
HEAD1 algorithm       = 0 (LZ4)
explicit bit selected = no

h_advise=2 is ordinary HEAD1 big-pcluster metadata. Naming or sizing the image as an extent attempt does not enter the explicit mapper.

Result: not an explicit fixture.

3. Structured conversion attempt

The G5 transformer parsed the source inode, map header, full lcluster indexes, physical extents, and payload bounds. It refused to emit a purported positive fixture because conversion requires relocating variable-size extent records, subsequent inode metadata, and possibly payload blocks. erofs-utils 1.8.6 cannot reopen and validate the resulting ABI, so an ad hoc rewrite would not provide trustworthy evidence.

The existing tests/review_fixtures.py helper was also reviewed. Its extent-pa-wrap.erofs conversion deliberately writes two 4-byte records and a physical base whose pa + plen overflows. That is a negative overflow trigger for TC155. It does not describe a valid mapped payload and cannot close this issue.

Result: structured positive conversion unavailable.

4. ABI and control-flow comparison

The G5 review compared repo22 src/erofs_fs.h/src/zmap.c with the Linux EROFS definitions and mapper. Both sides define:

• record sizes 4, 8, 16, and 32 bytes;
• implicit 64-bit physical bases for 4-byte records;
• per-record low physical starts for 8-byte records;
• explicit counts and binary search for 16/32-byte records;
• high physical and logical words where the record size carries them;
• partial-reference, shifted/interlaced format, and final fragment handling;
• malformed-count and arithmetic bounds checks.

This reduces implementation risk but is static evidence only.

Impact

• TC146 HEAD2: dynamic PASS.
• TC146 interlaced: dynamic PASS.
• TC146 explicit mapped payload: SHELVED.
• TC146 overall: PARTIAL, never overall PASS while this issue remains.
• Record-size variants, positive binary-search transitions, mapped high words, and explicit-record partial references remain dynamically unvalidated.

No repo22 kernel source was changed because this is a fixture/tooling gap, not an observed kernel failure.

Progress

• Reproduce tool limitation against exactly erofs-utils 1.8.6.
• Record tool output and zero symbol matches.
• Generate and inspect a fresh --max-extent-bytes attempt.
• Record source/image hashes and map-header fields.
• Audit the existing structured negative helper.
• Compare FreeBSD and Linux record layouts/control flow.
• Reject globally unordered 16-byte and 32-byte explicit tables through TC157 on FreeBSD 15 without reading the referenced payload.
• Obtain a producer or validator for positive mapped extent records.
• Generate at least one source-identical mapped payload.
• Cover 4/8/16/32-byte positive records where applicable.
• Exercise binary-search transitions, partial refs, high words, and final fragments dynamically.

Acceptance Criteria

This issue can close only after a reproducible helper emits a valid mapped compressed payload, records its structural fields and hashes, validates it with an independent format-aware implementation, and FreeBSD full/boundary reads match the source. Negative overflow/hole fixtures and static review do not satisfy acceptance.

Final Residual Risk

TC157 closes the known binary-search ordering bug for descending, duplicate, and cross-branch 16-byte and 32-byte tables. It does not supply the missing positive mapped payload, dynamically cover every record-size variant, or benchmark extremely large extent counts. Those limits do not change the issue status or TC146's PARTIAL result.