erofs-freebsd-out-tree/tests/results/manual/2026-08-08T2037Z-nfs/manual-test-report.md

repo22 FreeBSD 15 NFS export manual test report

Date: 2026-08-08 20:08-20:37 UTC

Baseline: 4ef680df6dcbfad329c0b2b1c0af8fcb21cbed4a plus this NFS export batch

Guest: FreeBSD 15.0-RELEASE-p8 amd64, releng/15.0-n281036-53054229dcb3, QEMU TCG, 4 vCPUs, 6144 MB RAM

Host tools: erofs-utils 1.8.6, clang cross-target build

Result

PASS for the implementation and all reasonably executable NFS tests.

FreeBSD mountd installed real EROFS exports through export-only MNT_UPDATE; NFSv3 clients resolved 16-byte EROFS handles for regular files, directories, symlinks, FIFOs, metabox bit-63 inodes, and multidevice files. Handle mutation, same-md remount stability, nfsd restart, 12,050-entry READDIRPLUS pagination, cold remounts, concurrent traversal/read/stat load, read-only enforcement, and device-boundary errors all behaved as required.

The only non-code issue was a cleanup-order mistake after the completed test matrix: one hard loopback NFS mount remained while nfsd and the direct EROFS mount were stopped. This blocked mount -p and later produced five client fileid changed messages when nfsd was temporarily restarted without the underlying export. The functional-test dmesg snapshot taken before that cleanup mistake was byte-identical to its baseline. The VM was rebooted to clear the blocked cleanup process, and a final correctly ordered NFS rerun completed with byte-identical pre/post dmesg and zero remaining resources.

Repository and build

Initial verification:

git -C /work/repo-community/repo22 fetch xdm main
git -C /work/repo-community/repo22 rev-parse HEAD
git -C /work/repo-community/repo22 rev-parse xdm/main
git -C /work/repo-community/repo22 diff --quiet
git -C /work/repo-community/repo22 diff --cached --quiet

Both revisions were:

4ef680df6dcbfad329c0b2b1c0af8fcb21cbed4a

Tracked files were clean. Existing untracked repo22/build, tests/results/manual/2026-08-08T1800Z-namei/artifacts, and files outside repo22 were not deleted, modified, staged, or committed.

To execute the required ./build.sh without overwriting the pre-existing untracked build products, the current source was copied to an isolated build directory:

BUILD_ROOT=$(mktemp -d /work/build/repo22-nfs-build.XXXXXX)
mkdir -p "$BUILD_ROOT/src"
cp repo-community/repo22/build.sh "$BUILD_ROOT/build.sh"
cp repo-community/repo22/src/* "$BUILD_ROOT/src/"
chmod +x "$BUILD_ROOT/build.sh"
cd "$BUILD_ROOT"
./build.sh
nm -u build/erofs.ko | \
  awk '$NF == "bcmp" { found = 1 } END { if (found) exit 1 }'
sha256sum build/erofs.ko

Result:

==> SUCCESS: /work/build/repo22-nfs-build.eFulVf/build/erofs.ko
48776485d1679f00100bc7b2be5f891c8f03a3e8b53ac7c00da8c2fd5f74cc70  erofs.ko

There was no unresolved bcmp reference.

The FreeBSD helper compiled without warnings:

cc -O2 -Wall -Wextra -std=c17 nfs_fh_tool.c -o nfs_fh_tool

Fixtures

The primary fixture was generated with:

mkdir -p plain-src/basic/subdir plain-src/bigdir plain-src/concurrent
printf 'repo22-nfs-regular\nline-two\n' > plain-src/basic/regular.txt
printf 'nested-directory-file\n' > plain-src/basic/subdir/nested.txt
ln -s regular.txt plain-src/basic/link-to-regular
mkfifo plain-src/basic/test.fifo
for i in $(seq -w 0 12049); do
  printf 'entry-%s\n' "$i" > plain-src/bigdir/file-$i
done
for i in $(seq -w 0 255); do
  awk -v n="$i" 'BEGIN {
    for (j = 0; j < 256; j++)
      printf "worker-%s-line-%03d-abcdefghijklmnopqrstuvwxyz0123456789\\n", n, j
  }' > plain-src/concurrent/file-$i.dat
done
awk 'BEGIN {
  for (i = 0; i < 262144; i++)
    printf "throughput-%08d-abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ\\n", i
}' > plain-src/throughput.dat
mkfs.erofs -T0 --all-root --ignore-mtime -x-1 -Uclear \
  plain-nfs.erofs plain-src

Fixture and manifest hashes:

Object	SHA256
plain-nfs.erofs	6928f05b58ce462596f9c7bf1d3442ad50d555dc5c31ed01d28e1a921a448bf2
metabox-nfs.erofs	8f493ed70d4be8096ab1adf8208622734d61a64903cf96a1ca1a1cc1854b1985
multidevice primary	fa2f35f59f63cc4a56d230ba960a6c1d27748506fccce081447cab0dc0d540fd
multidevice blob 1	3aeb19c519636b61dc3ebd47c6286098deaa0990ff618ce3facbff4fd65907cd
source manifest	0eda6f88c3b6c4b7c8d81d940a237d1f63c958a7f6fe7bdd4f77d6b293736fa9
basic/regular.txt	93e7f327676f582356e9de2fcca7b714910227008fa1c5e7820e942c471edb6f
basic/subdir/nested.txt	6da159d7ea19e9d1253ccb0f2c67586e6e075459954e740ff8405546683d87c3
concurrent/file-000.dat	8435866622642b6dadf502b09f95af32cb01907ba3376e4f1fe96278e5ada692
throughput.dat (21,757,952 bytes)	0841effed82d1adf394b6834ce30d4d9eb5fc9427527e854ec1cb6bb4b86c119
expected 12,050-name sorted list	4fbf1b6103e8884223629c3fd03f4ec36061e65cd2a1becfc96670daa96f9d71

The metabox image was the previously qualified compressed-metabox fixture. The multidevice fixture was the previously qualified primary-plus-one-blob image.

FreeBSD 15 setup commands

The artifacts were copied to the existing VM through SSH port 9222. Password material and askpass script contents were neither printed nor recorded.

scp -P 9222 erofs.ko plain-nfs.erofs metabox-nfs.erofs \
  tests/nfs_fh_tool.c [email protected]:/tmp/repo22-nfs-test/
ssh -p 9222 [email protected]
cd /tmp/repo22-nfs-test
sha256 erofs.ko plain-nfs.erofs metabox-nfs.erofs
cc -O2 -Wall -Wextra -std=c17 nfs_fh_tool.c -o nfs_fh_tool

The guest hashes matched the host hashes.

EROFS and NFS setup:

kldload /tmp/repo22-nfs-test/erofs.ko
mdconfig -a -t vnode -f /tmp/repo22-nfs-test/plain-nfs.erofs -u 42
mkdir -p /mnt/repo22-erofs
mount -t erofs -o ro /dev/md42 /mnt/repo22-erofs
mount -v | grep /mnt/repo22-erofs

printf '%s\n' \
  '/mnt/repo22-erofs -ro -maproot=root -network 127.0.0.0 -mask 255.0.0.0' \
  > /etc/exports
service rpcbind onestart
service mountd onestart
service nfsd onestart
service mountd onereload
rpcinfo -p 127.0.0.1
showmount -e 127.0.0.1
mount -v | grep /mnt/repo22-erofs

Before mountd reload the EROFS line did not contain NFS exported. Afterwards:

/dev/md42 on /mnt/repo22-erofs (erofs, NFS exported, local, read-only, acls, ...)

NFSv2/v3 and mountd were registered over TCP and UDP. NFS components are built into this GENERIC kernel: kldload nfsd and kldload nfscl reported already loaded or in kernel, while rpcbind/mountd/nfsd operated normally.

File-handle validation commands

Normal vnode types:

./nfs_fh_tool capture /mnt/repo22-erofs/basic/regular.txt regular.before.fh
./nfs_fh_tool capture /mnt/repo22-erofs/basic/subdir directory.fh
./nfs_fh_tool lcapture /mnt/repo22-erofs/basic/link-to-regular symlink.fh
./nfs_fh_tool capture /mnt/repo22-erofs/basic/test.fifo fifo.fh
./nfs_fh_tool describe regular.before.fh
./nfs_fh_tool describe directory.fh
./nfs_fh_tool describe symlink.fh
./nfs_fh_tool describe fifo.fh
./nfs_fh_tool stat regular.before.fh
./nfs_fh_tool stat directory.fh
./nfs_fh_tool stat symlink.fh
./nfs_fh_tool stat fifo.fh
./nfs_fh_tool cat regular.before.fh regular.fhopen.out

Observed handles:

Type	NID	Generation	Result
regular	0x74	1	fhstat and fhopen PASS
directory	0x76	1	fhstat PASS
symlink (lgetfh)	0x72	1	fhstat PASS
FIFO	0x79	1	fhstat PASS

All had len=16 and pad=0. The regular full-handle SHA256 was:

fb1778fc6bdc979e10d003dd6131a961e7532baa16273c5465935a665bf990f6

Malformed and stale classification:

./nfs_fh_tool mutate regular.before.fh bad-len.fh len 15
./nfs_fh_tool mutate regular.before.fh bad-pad.fh pad 1
./nfs_fh_tool mutate regular.before.fh bad-gen.fh gen 2
./nfs_fh_tool mutate regular.before.fh bad-nid.fh nid_hi 0xffffffff
./nfs_fh_tool expect-stat bad-len.fh EINVAL
./nfs_fh_tool expect-open bad-len.fh EINVAL
./nfs_fh_tool expect-stat bad-pad.fh EINVAL
./nfs_fh_tool expect-open bad-pad.fh EINVAL
./nfs_fh_tool expect-stat bad-gen.fh ESTALE
./nfs_fh_tool expect-open bad-gen.fh ESTALE
./nfs_fh_tool expect-stat bad-nid.fh ESTALE
./nfs_fh_tool expect-open bad-nid.fh ESTALE

All eight checks passed. fhstat exercised the shared-lock VFS_FHTOVP path; fhopen exercised the exclusive-lock path.

Same explicit md-unit remount:

umount /mnt/repo22-erofs
mdconfig -d -u 42
mdconfig -a -t vnode -f plain-nfs.erofs -u 42
mount -t erofs -o ro /dev/md42 /mnt/repo22-erofs
./nfs_fh_tool capture /mnt/repo22-erofs/basic/regular.txt regular.after.fh
./nfs_fh_tool compare regular.before.fh regular.after.fh
./nfs_fh_tool cat regular.before.fh regular.remount.out

The complete handles were byte-identical and the pre-remount handle still read the correct file after remount.

Non-export mount update rejection:

mount -u -o noexec /mnt/repo22-erofs

Result: Operation not supported; the mount remained exported, local, read-only, and did not gain noexec.

NFSv3 basic and restart tests

Four NFSv3 TCP READDIRPLUS clients were mounted:

mount_nfs -o nfsv3,tcp,rdirplus,readdirsize=512 \
  127.0.0.1:/mnt/repo22-erofs /mnt/repo22-nfs-512
mount_nfs -o nfsv3,tcp,rdirplus,readdirsize=1024 \
  127.0.0.1:/mnt/repo22-erofs /mnt/repo22-nfs-1024
mount_nfs -o nfsv3,tcp,rdirplus,readdirsize=4096 \
  127.0.0.1:/mnt/repo22-erofs /mnt/repo22-nfs-4096
mount_nfs -o nfsv3,tcp,rdirplus \
  127.0.0.1:/mnt/repo22-erofs /mnt/repo22-nfs-default
nfsstat -m

FreeBSD clamped the requested 512/1024/4096 values to an effective readdirsize=8192; the default client used 65536. This was recorded as a client environment limit, not a failure.

For each client:

cmp NFS/basic/regular.txt DIRECT/basic/regular.txt
test -d NFS/basic/subdir
test "$(readlink NFS/basic/link-to-regular)" = regular.txt
test -p NFS/basic/test.fifo
test "$(stat -f %i NFS/basic/regular.txt)" = \
  "$(stat -f %i DIRECT/basic/regular.txt)"
! touch NFS/write-must-fail

All passed. Writes failed with Read-only file system.

nfsd restart with an open descriptor:

exec 3< /mnt/repo22-nfs-default/basic/regular.txt
service nfsd onerestart
cat <&3 > open-fd-after-nfsd-restart.out
exec 3<&-
cmp open-fd-after-nfsd-restart.out /mnt/repo22-erofs/basic/regular.txt

The open descriptor and subsequent path reopen both passed. Direct file handles captured before and after restart were byte-identical.

Metabox bit-63 test

Commands:

mdconfig -a -t vnode -f metabox-nfs.erofs -u 43
mount -t erofs -o ro /dev/md43 /mnt/repo22-metabox
./nfs_fh_tool capture /mnt/repo22-metabox/long/file1.txt metabox.before.fh
./nfs_fh_tool describe metabox.before.fh
./nfs_fh_tool stat metabox.before.fh
./nfs_fh_tool cat metabox.before.fh metabox.fhopen.out
./nfs_fh_tool mutate metabox.before.fh metabox.bad-nid.fh \
  nid_lo 0xffffffff
./nfs_fh_tool expect-stat metabox.bad-nid.fh ESTALE
./nfs_fh_tool expect-open metabox.bad-nid.fh ESTALE

Observed handle:

len=16 pad=0 nid=800000000000010a gen=1
ino=9223372036854776074

The metabox mount was exported with mountd and mounted over NFSv3. Direct, fhopen, and NFS contents had SHA256:

661b22d2a7bd94a7da35834d4cc3647eb527d48eb08acc75bde2e3a7691ed45e

The NFS and direct inode numbers matched. The out-of-range metabox mutation returned ESTALE through both shared and exclusive paths.

Multidevice boundary test

Normal NFS commands:

mdconfig -a -t vnode -f primary.img -u 44
mdconfig -a -t vnode -f blob1.img -u 45
mount -t erofs -o ro -o device.1=/dev/md45 /dev/md44 \
  /mnt/repo22-multidev
mount_nfs -o nfsv3,tcp,rdirplus,readdirsize=512 \
  127.0.0.1:/mnt/repo22-multidev /mnt/repo22-nfs-multidev
cmp /mnt/repo22-nfs-multidev/alpha.bin /mnt/repo22-multidev/alpha.bin
cmp /mnt/repo22-nfs-multidev/small.txt /mnt/repo22-multidev/small.txt

Normal multidevice NFS reads passed. Representative hashes:

f779dbe3aeaa609beb32212579f3aea441f244621d13c977cb8b52d7ec3016d9  alpha.bin
5990b055a3c6d27681185a25e15ff21ba3cbdca7632faf4332233238e5f45aad  small.txt
001bcf4626e7d52f1bd76dee9a6c6e7d3a718c03602c5f9b07840a6c98b8fdc3  tree/beta.bin

Cold detached-provider test:

umount /mnt/repo22-multidev
mdconfig -d -u 44
mdconfig -a -t vnode -f primary.img -u 44
mdconfig -a -t vnode -f blob1.img -u 45
mount -t erofs -o ro -o device.1=/dev/md45 /dev/md44 \
  /mnt/repo22-multidev
./nfs_fh_tool capture /mnt/repo22-multidev/tree/beta.bin multidev-beta.fh
./nfs_fh_tool stat multidev-beta.fh
mdconfig -d -o force -u 45
! ./nfs_fh_tool cat multidev-beta.fh beta.after-orphan

The handle resolved to NID 0x36, then the cold read returned:

Device not configured

This preserved the multidevice ENXIO boundary error instead of converting it to ESTALE, reading past the provider, or panicking.

READDIRPLUS and stress

Every one of the four initial listings returned exactly 12,050 unique expected names. The expected list and all four sorted outputs had SHA256:

4fbf1b6103e8884223629c3fd03f4ec36061e65cd2a1becfc96670daa96f9d71

Dot handling:

ls -a1 /mnt/repo22-nfs-default/bigdir > nfs-dot-list
test "$(wc -l < nfs-dot-list)" -eq 12052
test "$(grep -cx '\.' nfs-dot-list)" -eq 1
test "$(grep -cx '\.\.' nfs-dot-list)" -eq 1

All passed.

Cold pagination used five unmount/remount/list/compare cycles on the small client. Concurrency then used eight workers, three complete 12,050-entry rounds each, distributed across the four mounts. All 29 cold/concurrent sorted outputs had one unique SHA256 value, matching the expected list.

Parallel data/metadata load:

for M in 512 1024 4096 default; do
  find /mnt/repo22-nfs-$M/concurrent -type f -maxdepth 1 -print0 | \
    xargs -0 -n 1 -P 8 cat > /dev/null &
  find /mnt/repo22-nfs-$M/concurrent -type f -maxdepth 1 -print0 | \
    xargs -0 -n 1 -P 8 stat -f '%i %z' > /dev/null &
done
wait

All jobs completed successfully.

Final stress counters:

Counter	Client	Server
requests / cache misses	270,854	271,174
lookup	264,989	265,109
read	1,360	1,360
READDIRPLUS	2,512	2,512
write	0	0
timed out	0	n/a
retries	0	n/a

There were no READDIR RPCs because the clients negotiated READDIRPLUS.

Informational throughput:

21757952 bytes transferred in 5.843937 seconds
3723167 bytes/second

No fixed throughput threshold was used.

Test matrix

Test	Actual	Status
isolated ./build.sh	success; module hash recorded	PASS
unresolved bcmp	none	PASS
FreeBSD helper compile	-Wall -Wextra, no warnings	PASS
final kldload / kldunload	load, unload, reload, unload succeeded	PASS
initial mount export flag	absent before mountd	PASS
export-only MNT_UPDATE	mountd installed export	PASS
non-export MNT_UPDATE	EOPNOTSUPP, flags unchanged	PASS
regular file	direct, fhopen, NFS content match	PASS
directory	handle and NFS traversal	PASS
symlink	lgetfh, target and NFS lookup	PASS
FIFO	handle metadata and NFS type	PASS
read-only behavior	direct and all NFS mounts rejected writes	PASS
16-byte handle ABI	len/pad/NID hi/NID lo/gen observed	PASS
generation	1, matching va_gen	PASS
shared/exclusive lock contract	fhstat and fhopen passed	PASS
malformed len/pad	EINVAL	PASS
stale generation/NID	ESTALE	PASS
same md42 remount	complete handle byte-identical	PASS
nfsd restart	open descriptor and reopen passed	PASS
metabox bit 63	real 0x800000000000010a round-trip	PASS
metabox boundary mutation	ESTALE	PASS
multidevice NFS	external-data reads matched	PASS
detached external device	cold handle read preserved ENXIO	PASS
12,050-entry listings	exact count and names on four clients	PASS
cookie pagination	five cold remounts, no duplicate/omission	PASS
dot omitted	one . and one .., 12,052 total	PASS
concurrent traversal	8 workers x 3 rounds, all hashes equal	PASS
concurrent read/stat	all jobs completed	PASS
READDIRPLUS	client/server 2,512	PASS
NFS timeouts/retries	0 / 0	PASS
successful NFS writes	0	PASS
throughput	3,723,167 B/s, informational	INFO

dmesg and cleanup

The functional matrix captured dmesg before loading the test module and after all functional tests plus a load/unload cycle. Both files contained the same two pre-existing historical lines and compared byte-for-byte equal:

interface erofs.1 already present in the KLD 'erofs-repo22-xattrfix.ko'!
md0: truncating fractional last sector by 14 bytes

No EROFS error, stale-handle message, trap, panic, or new kernel line occurred during the functional matrix.

The first cleanup attempt used the wrong order: it stopped nfsd and unmounted the direct EROFS mounts before one hard loopback client was fully gone. This left a mount -p process in uninterruptible NFS wait. When rpcbind/nfsd were temporarily restarted without the underlying EROFS export to release those clients, the NFS client logged five fileid changed messages. These messages occurred after the byte-identical functional dmesg snapshot and are classified as environment cleanup artifact, not an EROFS functional failure.

The VM was rebooted to remove the blocked cleanup process. Post-reboot audit:

erofs_mounts=0
nfs_mounts=0
md_units=0
erofs_modules=0
nfsd_pids=0 mountd_pids=0 rpcbind_pids=0
exports_exists=no

A final correctly ordered clean rerun then performed:

kldload erofs.ko
mdconfig -a -t vnode -f plain-nfs.erofs -u 42
mount -t erofs -o ro /dev/md42 /mnt/repo22-erofs
service rpcbind onestart
service mountd onestart
service nfsd onestart
mount_nfs -o nfsv3,tcp,rdirplus,readdirsize=512 \
  127.0.0.1:/mnt/repo22-erofs /mnt/repo22-nfs-clean
cmp /mnt/repo22-nfs-clean/basic/regular.txt \
  /mnt/repo22-erofs/basic/regular.txt
test "$(find /mnt/repo22-nfs-clean/bigdir -type f -maxdepth 1 | wc -l)" \
  -eq 12050
umount /mnt/repo22-nfs-clean
: > /etc/exports
service mountd onereload
service nfsd onestop
service mountd onestop
service rpcbind onestop
umount /mnt/repo22-erofs
mdconfig -d -u 42
kldunload erofs

The clean rerun's pre/post dmesg files compared byte-for-byte equal. Final state after that rerun:

clean_rerun_erofs=0 nfs=0 md=0 module=0 services=0 exports=absent

Limitations

• ENVIRONMENT-LIMIT: FreeBSD clamped requested readdir sizes 512, 1024, and 4096 to an effective minimum of 8192. The test still exercised a much smaller value than the 65536 default and produced 2,512 real READDIRPLUS calls.
• ENVIRONMENT-CLEANUP-ARTIFACT: the five post-test fileid changed lines were caused by the explicitly documented wrong cleanup order. A reboot and a correctly ordered rerun left no new dmesg lines or resources.
• KERNEL-FAIL: none.
• NOT RUN: none of the required functional cases were omitted.