This commit is contained in:
2026-08-18 09:20:44 +02:00
commit b826cd721a
522 changed files with 93730 additions and 0 deletions
@@ -0,0 +1,112 @@
# repo22 Cold Nested Namei Regression Report
- Date: 2026-08-08 UTC
- Baseline: `29a215dd4519579f6313b3df67d524a6b4bdf3ca`
- Guest: FreeBSD 15.0-RELEASE-p8 amd64, QEMU TCG
- Final module SHA-256:
`50a19ea96c7414f73d92049671e66a9eacc9ff5abe27950a46d16c8e6c763baf`
- Result: PASS
## Root Cause
FreeBSD pathname lookup passes a component as `cn_nameptr` plus
`cn_namelen`. An intermediate component is followed by `/` in the pathname
buffer and is not NUL-terminated at `cn_namelen`. The imported Linux EROFS
comparison assumed Linux dentry-name termination, ignored the supplied length,
and tested `qn_name[i] == '\0'` after matching the on-disk name. Therefore a
final component worked, while the same name used as an intermediate component
compared greater than the on-disk entry and returned `ENOENT`. Looking up the
parent as a final component first populated the FreeBSD namecache and hid the
bug on the next nested lookup.
The old error path also inserted a negative cache entry for every lookup
error, including integrity and I/O errors, which could mask later corruption
as `ENOENT`.
## Implementation
- `src/namei.c`
- Compares pathname components by explicit length without reading beyond
`cn_namelen`.
- Uses unsigned-byte ordering compatible with EROFS directory sorting.
- Validates the minimum block size before reading the first dirent.
- Validates the dirent-array boundary, strictly increasing name offsets,
name-slot bounds, name length, and zero-only NUL padding.
- Inserts negative namecache entries only for real `ENOENT` misses.
- `src/dir.c`
- Applies the same directory-block and name-padding validation to `readdir`.
- Determines the actual last-name length before enforcing `EROFS_NAME_LEN`,
so valid full-block zero padding is accepted.
- Keeps on-disk offsets unchanged and appends a synthetic `.` at `i_size`
for `dot_omitted`, matching Linux EROFS.
- Aligns restart positions relative to each directory block and preserves
the `i_size` cookie needed to resume the synthetic dot entry.
- Initializes returned cookie-array outputs before allocation.
## Deterministic Fixture
`prepare-fixtures.sh` creates the same image twice and requires `cmp` success.
The base image contains a cold multi-level path and a 320-file, multi-block
directory. Fixed timestamp, ownership, UUID, worker count, and uncompressed
layout are used.
Final image hashes:
```text
2e9fd75159011ced31646a38f942fa0822d5b3e8e19b7df55b844575fba991ea corrupt-nameoff.erofs
63d12232f9ea0dc7f7ff477d20b95a8412d191fc10bf4b67dacc47e050c379fb corrupt-padding.erofs
77f8a56f969e173d291ccbd957821f1ba284d3e54d875dbcdb9bc398024dfa5a corrupt-short-block.erofs
11064ffbb814ff41027aebc8f36e56d2f24affb7b50836948b3ffbf6d79dee0e dot-omitted.erofs
7a2c244ec10e9a43531b0e8b92e26b11f52d67bab00528b8ba2f584e20e57420 nested-repeat.erofs
7a2c244ec10e9a43531b0e8b92e26b11f52d67bab00528b8ba2f584e20e57420 nested.erofs
```
## Build Results
- `./build.sh`: PASS.
- Final `build/erofs.ko` SHA-256 remained
`50a19ea96c7414f73d92049671e66a9eacc9ff5abe27950a46d16c8e6c763baf`.
- `nm -u build/erofs.ko | grep -w bcmp`: no match.
- `git diff --check` for all scoped source and test files: PASS.
The final integration rerun used the same combined module and these guest
commands:
```sh
cd /root/repo22-namei
cc -Wall -Wextra -O2 -o readdir_probe readdir_probe.c
./vm-regression.sh
```
`vm-regression.sh` performs `kldload`, creates each vnode-backed md device,
mounts it with `mount -t erofs`, executes the cold lookup and readdir probes,
unmounts and detaches each image, and finishes with exact module unload.
## FreeBSD VM Results
- `kldload`: PASS.
- Cold direct read of
`/alpha/bravo/charlie/payload.txt` without parent lookup or `readdir`: PASS.
- Repeated lookup and sibling nested lookup: PASS.
- Two negative lookups followed by an existing nested lookup: PASS.
- Multi-block `wide` readdir: 322 dirents (`.`, `..`, 320 files), PASS.
- Resume from every one of the 322 returned `d_off` cookies: PASS.
- `dot_omitted` root cookies: `12`, `24`, `47`, `48`; resume at `47`
returns only `.`, and resume at `48` returns EOF: PASS.
- Short directory block: two lookups and direct `getdirentries` all return
`EINTEGRITY`, PASS.
- Non-monotonic `nameoff`: two lookups and direct `getdirentries` all return
`EINTEGRITY`, PASS.
- Nonzero data after NUL padding: two lookups and direct `getdirentries` all
return `EINTEGRITY`, PASS.
- `kldunload`: PASS.
- Post-test EROFS module, mount, and md-device state: clean.
- Final explicit state check: zero EROFS modules, zero EROFS mounts, zero md
devices, and no recent panic or fatal trap in dmesg.
## Remaining Scope
No unresolved issue remains for the requested cold lookup and directory
regression. The NFS-specific `a_cookies` consumer path was not exercised by an
NFS export; the tested `d_off` restart-cookie sequence uses the same generated
cookie values.
@@ -0,0 +1,199 @@
#!/bin/sh
set -eu
script_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
fixture_dir="$script_dir/fixture"
artifact_dir="$script_dir/artifacts"
mkdir -p \
"$fixture_dir/alpha/bravo/charlie" \
"$fixture_dir/alpha/sibling" \
"$fixture_dir/wide" \
"$artifact_dir"
printf '%s\n' 'cold nested lookup payload' > \
"$fixture_dir/alpha/bravo/charlie/payload.txt"
printf '%s\n' 'repeat lookup payload' > \
"$fixture_dir/alpha/bravo/repeat.txt"
printf '%s\n' 'sibling marker' > \
"$fixture_dir/alpha/sibling/marker.txt"
index=0
while [ "$index" -lt 320 ]; do
name=$(printf 'entry-%03d-abcdefghijklmnopqrstuvwxyz.txt' "$index")
printf 'wide entry %03d\n' "$index" > "$fixture_dir/wide/$name"
index=$((index + 1))
done
find "$fixture_dir" -exec touch -h -t 197001010000.00 {} +
build_image()
{
image=$1
mkfs.erofs -d0 -x-1 -T0 --all-time --all-root --workers=1 \
-U 11111111-2222-3333-4444-555555555555 \
"$image" "$fixture_dir"
}
build_image "$artifact_dir/nested.erofs"
build_image "$artifact_dir/nested-repeat.erofs"
cmp "$artifact_dir/nested.erofs" "$artifact_dir/nested-repeat.erofs"
ARTIFACT_DIR="$artifact_dir" python3 <<'PY'
import hashlib
import os
import shutil
import struct
artifact_dir = os.environ["ARTIFACT_DIR"]
base_path = os.path.join(artifact_dir, "nested.erofs")
def u16(image, offset):
return struct.unpack_from("<H", image, offset)[0]
def u32(image, offset):
return struct.unpack_from("<I", image, offset)[0]
def put_u16(image, offset, value):
struct.pack_into("<H", image, offset, value)
def put_u32(image, offset, value):
struct.pack_into("<I", image, offset, value)
def update_superblock_checksum(image):
polynomial = 0x82F63B78
checksum = 0xFFFFFFFF
put_u32(image, 1024 + 4, 0)
for byte in image[1024:4096]:
checksum ^= byte
for _ in range(8):
checksum = (checksum >> 1) ^ (polynomial if checksum & 1 else 0)
put_u32(image, 1024 + 4, checksum & 0xFFFFFFFF)
def inode_offset(nid):
return nid << 5
def compact_inline_dir(image, nid):
offset = inode_offset(nid)
inode_format = u16(image, offset)
layout = (inode_format >> 1) & 0x7
assert (inode_format & 0x1) == 0
assert layout == 2
assert u16(image, offset + 2) == 0
return offset, offset + 32, u32(image, offset + 8)
def dir_entries(image, data_offset, size):
first_nameoff = u16(image, data_offset + 8)
assert first_nameoff >= 12
assert first_nameoff % 12 == 0
assert first_nameoff < size
count = first_nameoff // 12
entries = []
for index in range(count):
entry_offset = data_offset + index * 12
nid, nameoff, file_type, reserved = struct.unpack_from(
"<QHBB", image, entry_offset
)
endoff = (
u16(image, data_offset + (index + 1) * 12 + 8)
if index + 1 < count
else size
)
assert first_nameoff <= nameoff < endoff <= size
slot = bytes(image[data_offset + nameoff : data_offset + endoff])
name = slot.split(b"\0", 1)[0]
assert name
entries.append((nid, nameoff, file_type, reserved, name))
return entries
with open(base_path, "rb") as source:
base = bytearray(source.read())
assert u32(base, 1024) == 0xE0F5E1E2
assert base[1024 + 12] == 12
assert u16(base, 1024 + 14) == 36
root_nid = 36
root_inode, root_data, root_size = compact_inline_dir(base, root_nid)
root_entries = dir_entries(base, root_data, root_size)
assert [entry[4] for entry in root_entries] == [b".", b"..", b"alpha", b"wide"]
dot_omitted = bytearray(base)
removed = 13
new_size = root_size - removed
new_entries = root_entries[1:]
new_first_nameoff = len(new_entries) * 12
for index, entry in enumerate(new_entries):
nid, nameoff, file_type, reserved, _ = entry
struct.pack_into(
"<QHBB",
dot_omitted,
root_data + index * 12,
nid,
nameoff - removed,
file_type,
reserved,
)
names = b"".join(entry[4] for entry in new_entries)
dot_omitted[root_data + new_first_nameoff : root_data + new_size] = names
dot_omitted[root_data + new_size : root_data + root_size] = b"\0" * (
root_size - new_size
)
put_u16(dot_omitted, root_inode, u16(dot_omitted, root_inode) | (1 << 4))
put_u32(dot_omitted, root_inode + 8, new_size)
assert [entry[4] for entry in dir_entries(dot_omitted, root_data, new_size)] == [
b"..",
b"alpha",
b"wide",
]
short_block = bytearray(base)
put_u32(short_block, root_inode + 8, 8)
nonmonotonic = bytearray(base)
first_nameoff = u16(nonmonotonic, root_data + 8)
put_u16(nonmonotonic, root_data + 12 + 8, first_nameoff)
wide_nid = next(entry[0] for entry in root_entries if entry[4] == b"wide")
wide_inode = inode_offset(wide_nid)
assert ((u16(base, wide_inode) >> 1) & 0x7) == 2
wide_startblk = u32(base, wide_inode + 16)
wide_block = wide_startblk << 12
wide_first_nameoff = u16(base, wide_block + 8)
wide_count = wide_first_nameoff // 12
wide_last_nameoff = u16(base, wide_block + (wide_count - 1) * 12 + 8)
padding_nul = base.index(0, wide_block + wide_last_nameoff, wide_block + 4096)
assert padding_nul + 1 < wide_block + 4096
assert base[padding_nul + 1] == 0
bad_padding = bytearray(base)
bad_padding[padding_nul + 1] = ord("X")
outputs = {
"dot-omitted.erofs": dot_omitted,
"corrupt-short-block.erofs": short_block,
"corrupt-nameoff.erofs": nonmonotonic,
"corrupt-padding.erofs": bad_padding,
}
for filename, image in outputs.items():
update_superblock_checksum(image)
path = os.path.join(artifact_dir, filename)
with open(path, "wb") as output:
output.write(image)
with open(os.path.join(artifact_dir, "SHA256SUMS"), "w", encoding="ascii") as sums:
for filename in sorted(["nested.erofs", "nested-repeat.erofs", *outputs]):
path = os.path.join(artifact_dir, filename)
with open(path, "rb") as image_file:
digest = hashlib.sha256(image_file.read()).hexdigest()
sums.write(f"{digest} {filename}\n")
PY
cat "$artifact_dir/SHA256SUMS"
@@ -0,0 +1,71 @@
#include <sys/types.h>
#include <sys/dirent.h>
#include <dirent.h>
#include <err.h>
#include <fcntl.h>
#include <inttypes.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int
main(int argc, char **argv)
{
struct dirent *entry;
off_t base, before, start;
char *buffer;
char *end;
size_t buffer_size;
ssize_t bytes;
int calls, fd;
if (argc < 2 || argc > 5)
errx(2, "usage: %s directory [offset [buffer-size [calls]]]",
argv[0]);
start = argc >= 3 ? strtoll(argv[2], NULL, 0) : 0;
buffer_size = argc >= 4 ? strtoul(argv[3], NULL, 0) : 128;
calls = argc >= 5 ? strtol(argv[4], NULL, 0) : 32;
if (buffer_size < 32 || calls < 1)
errx(2, "invalid buffer size or call count");
fd = open(argv[1], O_RDONLY | O_DIRECTORY);
if (fd < 0)
err(1, "open %s", argv[1]);
if (lseek(fd, start, SEEK_SET) < 0)
err(1, "lseek %jd", (intmax_t)start);
buffer = malloc(buffer_size);
if (buffer == NULL)
err(1, "malloc");
for (int call = 0; call < calls; call++) {
before = lseek(fd, 0, SEEK_CUR);
if (before < 0)
err(1, "lseek current");
base = -1;
bytes = getdirentries(fd, buffer, buffer_size, &base);
if (bytes < 0)
err(1, "getdirentries");
printf("call=%d before=%jd after=%jd base=%jd bytes=%zd\n",
call, (intmax_t)before,
(intmax_t)lseek(fd, 0, SEEK_CUR), (intmax_t)base, bytes);
if (bytes == 0)
break;
end = buffer + bytes;
for (entry = (struct dirent *)buffer;
(char *)entry < end;
entry = (struct dirent *)((char *)entry + entry->d_reclen)) {
if (entry->d_reclen == 0 ||
(char *)entry + entry->d_reclen > end)
errx(1, "invalid dirent record");
printf(" off=%jd ino=%ju reclen=%u type=%u name=%.*s\n",
(intmax_t)entry->d_off, (uintmax_t)entry->d_fileno,
entry->d_reclen, entry->d_type, entry->d_namlen,
entry->d_name);
}
}
free(buffer);
close(fd);
return (0);
}
@@ -0,0 +1,201 @@
#!/bin/sh
set -u
test_dir=/root/repo22-namei
mount_dir=/mnt/repo22-namei
module_id=
md_device=
fail()
{
echo "FAIL: $*" >&2
exit 1
}
cleanup()
{
set +e
if mount | grep -q " on $mount_dir "; then
umount "$mount_dir"
fi
if [ -n "$md_device" ]; then
mdconfig -d -u "${md_device#md}"
fi
if [ -n "$module_id" ] && kldstat -q -i "$module_id"; then
kldunload -i "$module_id"
fi
}
trap cleanup EXIT INT TERM
mount_image()
{
image=$1
md_device=$(mdconfig -a -t vnode -f "$test_dir/$image") ||
fail "mdconfig $image"
mount -t erofs "/dev/$md_device" "$mount_dir" ||
fail "mount $image"
echo "mounted image=$image device=$md_device"
}
unmount_image()
{
umount "$mount_dir" || fail "umount $md_device"
mdconfig -d -u "${md_device#md}" || fail "detach $md_device"
md_device=
}
expect_integrity_failure()
{
image=$1
lookup_path=$2
readdir_path=$3
label=${image%.erofs}
mount_image "$image"
attempt=1
while [ "$attempt" -le 2 ]; do
output="$test_dir/$label-lookup-$attempt.txt"
if stat "$mount_dir/$lookup_path" >"$output" 2>&1; then
fail "$image lookup attempt $attempt unexpectedly succeeded"
fi
if grep -qi "No such file" "$output"; then
fail "$image lookup attempt $attempt became ENOENT"
fi
attempt=$((attempt + 1))
done
output="$test_dir/$label-readdir.txt"
if ./readdir_probe "$mount_dir/$readdir_path" 0 512 2 \
>"$output" 2>&1; then
fail "$image readdir unexpectedly succeeded"
fi
unmount_image
echo "integrity image=$image repeated-lookup=error readdir=error"
}
cd "$test_dir" || exit 1
mkdir -p "$mount_dir"
if mount | grep -qi erofs; then
fail "pre-existing EROFS mount"
fi
if kldstat | grep -qi erofs; then
fail "pre-existing EROFS module"
fi
if [ -n "$(mdconfig -l)" ]; then
fail "pre-existing md device"
fi
echo "== guest =="
uname -a
date -u
echo "== module load =="
kldload "$test_dir/erofs.ko" || fail "kldload"
module_id=$(kldstat | awk '$NF == "erofs.ko" { print $1 }')
[ -n "$module_id" ] || fail "loaded module not found"
kldstat -v -i "$module_id"
echo "== cold nested lookup =="
mount_image nested.erofs
payload=$(cat "$mount_dir/alpha/bravo/charlie/payload.txt") ||
fail "cold nested lookup"
[ "$payload" = "cold nested lookup payload" ] || fail "cold payload mismatch"
payload=$(cat "$mount_dir/alpha/bravo/charlie/payload.txt") ||
fail "repeated nested lookup"
[ "$payload" = "cold nested lookup payload" ] || fail "repeat payload mismatch"
payload=$(cat "$mount_dir/alpha/bravo/repeat.txt") ||
fail "sibling nested lookup"
[ "$payload" = "repeat lookup payload" ] || fail "sibling payload mismatch"
attempt=1
while [ "$attempt" -le 2 ]; do
output="$test_dir/missing-$attempt.txt"
if stat "$mount_dir/alpha/bravo/missing" >"$output" 2>&1; then
fail "missing lookup attempt $attempt unexpectedly succeeded"
fi
grep -qi "No such file" "$output" || fail "missing lookup was not ENOENT"
attempt=$((attempt + 1))
done
payload=$(cat "$mount_dir/alpha/bravo/charlie/payload.txt") ||
fail "existing lookup after negative cache"
[ "$payload" = "cold nested lookup payload" ] || fail "post-negative payload mismatch"
echo "cold lookup=pass repeat=pass negative-cache=pass"
echo "== large readdir and cookies =="
wide_count=$(ls -A1 "$mount_dir/wide" | wc -l | tr -d ' ')
[ "$wide_count" = 320 ] || fail "wide entry count $wide_count"
./readdir_probe "$mount_dir/wide" 0 128 400 > wide-probe.txt ||
fail "wide readdir probe"
awk '/^ off=/ {
off = $1; sub(/^off=/, "", off);
name = $5; sub(/^name=/, "", name);
print off, name;
}' wide-probe.txt > wide-cookies.txt
wide_dirents=$(wc -l < wide-cookies.txt | tr -d ' ')
[ "$wide_dirents" = 322 ] || fail "wide dirent count $wide_dirents"
awk '{ cookie[NR] = $1; name[NR] = $2 }
END {
for (i = 1; i <= NR; i++)
print cookie[i], (i < NR ? name[i + 1] : "<EOF>");
}' wide-cookies.txt > wide-resume-cases.txt
while read -r cookie expected; do
./readdir_probe "$mount_dir/wide" "$cookie" 128 1 > wide-resume.txt ||
fail "resume cookie $cookie"
if [ "$expected" = "<EOF>" ]; then
grep -q 'bytes=0$' wide-resume.txt || fail "cookie $cookie not EOF"
else
actual=$(awk '/^ off=/ {
name = $5; sub(/^name=/, "", name); print name; exit;
}' wide-resume.txt)
[ "$actual" = "$expected" ] ||
fail "cookie $cookie expected $expected got $actual"
fi
done < wide-resume-cases.txt
echo "wide entries=$wide_dirents all-resume-cookies=pass"
unmount_image
echo "== dot omitted cookies =="
mount_image dot-omitted.erofs
./readdir_probe "$mount_dir" 0 512 8 > dot-probe.txt || fail "dot probe"
awk '/^ off=/ {
off = $1; sub(/^off=/, "", off);
name = $5; sub(/^name=/, "", name);
print off, name;
}' dot-probe.txt > dot-cookies.txt
cat > dot-expected.txt <<'EOF'
12 ..
24 alpha
47 wide
48 .
EOF
cmp dot-cookies.txt dot-expected.txt || fail "dot cookie sequence"
./readdir_probe "$mount_dir" 47 128 1 > dot-resume-47.txt || fail "dot resume 47"
grep -q '^ off=48 .* name=\.$' dot-resume-47.txt || fail "dot resume 47 result"
./readdir_probe "$mount_dir" 48 128 1 > dot-resume-48.txt || fail "dot resume 48"
grep -q 'bytes=0$' dot-resume-48.txt || fail "dot resume 48 not EOF"
echo "dot cookies=12,24,47,48 resume=pass"
unmount_image
echo "== corrupted directories =="
expect_integrity_failure corrupt-short-block.erofs alpha .
expect_integrity_failure corrupt-nameoff.erofs alpha .
expect_integrity_failure \
corrupt-padding.erofs \
wide/entry-000-abcdefghijklmnopqrstuvwxyz.txt \
wide
echo "== module unload =="
kldunload -i "$module_id" || fail "kldunload"
module_id=
if kldstat | grep -qi erofs; then
fail "module remains loaded"
fi
if mount | grep -qi erofs; then
fail "mount remains"
fi
if [ -n "$(mdconfig -l)" ]; then
fail "md remains"
fi
dmesg | tail -120 > dmesg-tail.txt
echo "PASS: all VM regressions"