194 lines
8.5 KiB
Markdown
194 lines
8.5 KiB
Markdown
# TC153 Large Directory Block Index Validation
|
|
|
|
Status: **RESOLVED - exact-source kernel validation passed**
|
|
|
|
Last updated: 2026-08-09
|
|
|
|
Priority: Critical validation gap
|
|
|
|
Implementation status: Source fix present, build-verified, and dynamically
|
|
validated
|
|
|
|
## Problem
|
|
|
|
`erofs_find_target_block()` used signed `int` values for the binary-search
|
|
front, back, and midpoint indexes. A directory whose logical size describes
|
|
`2147483649` 4 KiB blocks produces a final block index of `2147483648`.
|
|
Converting that value to `int` can make the initial back bound negative. The
|
|
search can then return `ENOENT` without reading any directory block, and the
|
|
FreeBSD namecache can retain a false negative result for corrupt media.
|
|
|
|
The review change uses `uint64_t` for block-search bounds and midpoints, checks
|
|
the midpoint multiplication, and handles the `mid == 0` lower-bound case. The
|
|
within-block search remains `uint32_t` because a validated EROFS directory
|
|
block can contain only a block-sized number of entries.
|
|
|
|
## Affected Features
|
|
|
|
- Cold pathname lookup in very large or maliciously sized directories.
|
|
- Corruption reporting and `EINTEGRITY` propagation from directory reads.
|
|
- Negative namecache correctness after a failed lookup.
|
|
- Linux/FreeBSD behavioral review of EROFS's two-level directory search.
|
|
|
|
Normal directories are not expected to approach this bound. The practical
|
|
risk is a crafted image causing an incorrect `ENOENT`, not an ordinary mkfs
|
|
image losing entries.
|
|
|
|
## Trigger Conditions
|
|
|
|
All of the following are needed to exercise the original width error:
|
|
|
|
1. A directory inode is accepted with a logical block count greater than
|
|
`INT_MAX`.
|
|
2. The directory layout reaches `erofs_find_target_block()` rather than being
|
|
rejected by an earlier inline-data bounds check.
|
|
3. A cold lookup is issued for a name that is not satisfied by an already
|
|
cached vnode or negative namecache entry.
|
|
4. The backing image is too short for the computed midpoint read, so the fixed
|
|
code should return `EINTEGRITY` instead of a synthetic miss.
|
|
|
|
For 4 KiB blocks, TC153 encodes a size of `8796093026304` bytes,
|
|
`2147483649` blocks, and a final index of `2147483648`.
|
|
|
|
## Current Analysis
|
|
|
|
The source change is mechanically narrow and has passed both repo22 build
|
|
configurations. The checked multiplication protects a future block-size or
|
|
index change even though the current `OFF_MAX` inode limit already bounds the
|
|
product. The `mid == 0` guard prevents unsigned subtraction from wrapping when
|
|
the search moves below the first block.
|
|
|
|
Dynamic proof must show more than a large `st_size`. It must prove that the
|
|
lookup enters the block binary search. An image rejected in
|
|
`erofs_read_inode()` is not evidence for this fix, and an `ENOENT` observed
|
|
after a warm negative cache is ambiguous.
|
|
|
|
## Attempts and Results
|
|
|
|
### Attempt 1: Small FLAT_INLINE image
|
|
|
|
- Artifact directory:
|
|
`/work/build/repo22-review-artifact.LtqNxH`
|
|
- Base image: `large-dir-base.erofs`, 4096 bytes, SHA256
|
|
`973d2a1c90ac1bf776ed76a1fc6a7e88b2156fac3bf04ddbae19cfafebbd562e`
|
|
- Patched image: `large-dir-intmax.erofs`, 4096 bytes, SHA256
|
|
`ce5cad3eff34ea50ca89eedf93d8ef90e6cea96858a138685fae45f39084de1f`
|
|
- Recorded inode: NID 40 at byte 1280.
|
|
|
|
The directory was Layout 2 (`FLAT_INLINE`). Patching only `i_size` made its
|
|
declared inline tail range invalid, so inode validation rejected it before
|
|
`erofs_find_target_block()`. This attempt is invalid as TC153 dynamic evidence
|
|
and must never be marked PASS.
|
|
|
|
### Attempt 2: More entries to force block data
|
|
|
|
- Artifact directory:
|
|
`/work/build/repo22-review-artifact2.FmMTup`
|
|
- Generated base: `large-dir-base.erofs`, 65536 bytes, SHA256
|
|
`50eb28351788ab1801408aca0a6fca6352755f5ca56efe6e2bb3e2fe08305663`
|
|
- Directory: NID 40, 21052 bytes, still Layout 2.
|
|
|
|
The generation process was interrupted before producing a patched image or a
|
|
FreeBSD result. A later deterministic rerun reproduced the same base hash and
|
|
showed why the approach failed: erofs-utils 1.8.6 tail-packs directories even
|
|
when they span multiple data blocks. Its `noinline_data` option does not turn
|
|
directory tail packing off.
|
|
|
|
### Attempt 3: Follow-up agents
|
|
|
|
Multiple follow-up agents were assigned the fixture and guest validation.
|
|
They either stopped while preparing the second image or returned no executed
|
|
commands, files, hashes, or guest output. These attempts added no evidence and
|
|
are recorded to prevent their elapsed time from being mistaken for progress.
|
|
|
|
### Attempt 4: Sparse-provider design
|
|
|
|
A guest-side sparse vnode provider was considered so a valid directory could
|
|
declare the complete multi-terabyte range. It was not executed. Such a scheme
|
|
must prove the GEOM media size, avoid materializing terabytes, preserve valid
|
|
initial directory blocks, and still force a cold read at the binary-search
|
|
midpoint. No result exists for this approach.
|
|
|
|
### Attempt 5: Reproducible FLAT_PLAIN conversion
|
|
|
|
The tracked generator now copies all 21052 bytes of the generated directory
|
|
to appended blocks, changes only that inode to Layout 0 (`FLAT_PLAIN`), updates
|
|
the image block count and CRC32C, and then applies the large logical size.
|
|
|
|
Host reproduction on 2026-08-09 produced:
|
|
|
|
- `large-dir-base.erofs`: 65536 bytes, SHA256
|
|
`50eb28351788ab1801408aca0a6fca6352755f5ca56efe6e2bb3e2fe08305663`
|
|
- `large-dir-intmax.erofs`: 90112 bytes, SHA256
|
|
`0f90d3d57adbbbd946e41b225c1f6c464915c6abb0b13478ec9b2a318def1f72`
|
|
- Patched inode: NID 40 at byte 1280, Layout 0, raw block 16.
|
|
- Declared image blocks: 22.
|
|
|
|
This resolved the fixture-construction problem. The qualified kernel run is
|
|
recorded below.
|
|
|
|
### Attempt 6: Qualified sparse GEOM provider
|
|
|
|
The G3 takeover generated a checksum-valid sparse prefix from Attempt 5 and
|
|
performed one targeted FreeBSD 15 run against the exact source requested by
|
|
the assignment.
|
|
|
|
- Source commit:
|
|
9ae22009f23a65320730072a780998e80aa9b728.
|
|
- KLD SHA256:
|
|
19ad086bd2508cbb4c57b1a51f38c93057d438b84fbcfa2e637ff2519f5bc590.
|
|
- Sparse-prefix SHA256:
|
|
f9337f83b1f568f6d904331a7749e6362a691055cd5af95b59bc89772f04e3b0.
|
|
- Prefix qualification: NID 40, inode offset 1280, extended Layout 0,
|
|
start block 16, 2147483649 directory blocks, final block index 2147483648,
|
|
valid superblock checksum.
|
|
- Sparse provider: 8796093091840 logical bytes, 448 allocated 512-byte
|
|
sectors, and the first 90112 bytes retained the prefix SHA256.
|
|
- GEOM diskinfo size: 8796093091840 bytes.
|
|
- Mounted /huge: size 8796093026304, NID 40.
|
|
|
|
The first two cold stat operations for /huge/missing each returned EINTEGRITY
|
|
(97). Root readdir then returned three complete entries and validated all
|
|
kernel/libc restart cookies. A third lookup after that readdir again returned
|
|
EINTEGRITY. No lookup returned ENOENT, so no false negative namecache entry
|
|
masked the corruption.
|
|
|
|
Unmount, md detach, sparse-provider removal, and KLD unload all succeeded.
|
|
The only new dmesg lines in the complete G3 run were expected SIGBUS child
|
|
exits from the unrelated assigned mmap tests; TC153 added no diagnostic.
|
|
|
|
## Historical Remaining Validation
|
|
|
|
Attempt 6 completed steps 1-5 and 7 below. The assignment mandated the exact
|
|
fixed source, so the optional pre-fix comparison in step 6 was not run.
|
|
|
|
1. Regenerate the images with
|
|
`tests/results/manual/2026-08-09T0124Z-final-review/prepare-fixtures.sh` and
|
|
verify the hashes and Layout 0 assertion.
|
|
2. Load the exact module under test in a clean FreeBSD 15 guest.
|
|
3. Mount the 90112-byte image and confirm `/huge` reports
|
|
`8796093026304` bytes.
|
|
4. With a cold parent cache, run two lookups of `/huge/missing` and capture
|
|
direct command exit codes plus `truss` errno.
|
|
5. Require `EINTEGRITY` on every fixed-module lookup, including after root
|
|
readdir. Confirm no negative cache changes the result.
|
|
6. Repeat with the pre-fix module to demonstrate the erroneous `ENOENT` path.
|
|
7. Record dmesg, mount/md/KLD cleanup, module and fixture hashes in a dated
|
|
manual report.
|
|
|
|
## Original Acceptance Criteria
|
|
|
|
TC153 can move from SHELVED to PASS only after the fixed FreeBSD KLD returns
|
|
`EINTEGRITY` repeatedly from the valid Layout 0 fixture, the pre-fix behavior
|
|
is distinguished, no trap or panic occurs, and complete cleanup evidence is
|
|
recorded. Host fixture generation and static source review alone are
|
|
insufficient.
|
|
|
|
## Resolution
|
|
|
|
TC153 is PASS because the exact fixed FreeBSD KLD returned EINTEGRITY
|
|
repeatedly from the valid Layout 0 fixture before and after root readdir, no
|
|
trap or panic occurred, and complete hash/provider/cleanup evidence is recorded
|
|
in tests/results/manual/2026-08-09T1059Z-g3/manual-test-report.md. Host fixture
|
|
generation and static source review alone remain insufficient.
|