# repo22 FreeBSD 15 NFS export manual test report

Date: 2026-08-08 20:08-20:37 UTC

Baseline: `4ef680df6dcbfad329c0b2b1c0af8fcb21cbed4a` plus this NFS export batch

Guest: FreeBSD 15.0-RELEASE-p8 amd64,
`releng/15.0-n281036-53054229dcb3`, QEMU TCG, 4 vCPUs, 6144 MB RAM

Host tools: erofs-utils 1.8.6, clang cross-target build

## Result

**PASS** for the implementation and all reasonably executable NFS tests.

FreeBSD mountd installed real EROFS exports through export-only `MNT_UPDATE`;
NFSv3 clients resolved 16-byte EROFS handles for regular files, directories,
symlinks, FIFOs, metabox bit-63 inodes, and multidevice files. Handle mutation,
same-md remount stability, nfsd restart, 12,050-entry READDIRPLUS pagination,
cold remounts, concurrent traversal/read/stat load, read-only enforcement, and
device-boundary errors all behaved as required.

The only non-code issue was a cleanup-order mistake after the completed test
matrix: one hard loopback NFS mount remained while nfsd and the direct EROFS
mount were stopped. This blocked `mount -p` and later produced five client
`fileid changed` messages when nfsd was temporarily restarted without the
underlying export. The functional-test dmesg snapshot taken before that cleanup
mistake was byte-identical to its baseline. The VM was rebooted to clear the
blocked cleanup process, and a final correctly ordered NFS rerun completed with
byte-identical pre/post dmesg and zero remaining resources.

## Repository and build

Initial verification:

```sh
git -C /work/repo-community/repo22 fetch xdm main
git -C /work/repo-community/repo22 rev-parse HEAD
git -C /work/repo-community/repo22 rev-parse xdm/main
git -C /work/repo-community/repo22 diff --quiet
git -C /work/repo-community/repo22 diff --cached --quiet
```

Both revisions were:

```text
4ef680df6dcbfad329c0b2b1c0af8fcb21cbed4a
```

Tracked files were clean. Existing untracked `repo22/build`,
`tests/results/manual/2026-08-08T1800Z-namei/artifacts`, and files outside
repo22 were not deleted, modified, staged, or committed.

To execute the required `./build.sh` without overwriting the pre-existing
untracked build products, the current source was copied to an isolated build
directory:

```sh
BUILD_ROOT=$(mktemp -d /work/build/repo22-nfs-build.XXXXXX)
mkdir -p "$BUILD_ROOT/src"
cp repo-community/repo22/build.sh "$BUILD_ROOT/build.sh"
cp repo-community/repo22/src/* "$BUILD_ROOT/src/"
chmod +x "$BUILD_ROOT/build.sh"
cd "$BUILD_ROOT"
./build.sh
nm -u build/erofs.ko | \
  awk '$NF == "bcmp" { found = 1 } END { if (found) exit 1 }'
sha256sum build/erofs.ko
```

Result:

```text
==> SUCCESS: /work/build/repo22-nfs-build.eFulVf/build/erofs.ko
48776485d1679f00100bc7b2be5f891c8f03a3e8b53ac7c00da8c2fd5f74cc70  erofs.ko
```

There was no unresolved `bcmp` reference.

The FreeBSD helper compiled without warnings:

```sh
cc -O2 -Wall -Wextra -std=c17 nfs_fh_tool.c -o nfs_fh_tool
```

## Fixtures

The primary fixture was generated with:

```sh
mkdir -p plain-src/basic/subdir plain-src/bigdir plain-src/concurrent
printf 'repo22-nfs-regular\nline-two\n' > plain-src/basic/regular.txt
printf 'nested-directory-file\n' > plain-src/basic/subdir/nested.txt
ln -s regular.txt plain-src/basic/link-to-regular
mkfifo plain-src/basic/test.fifo
for i in $(seq -w 0 12049); do
  printf 'entry-%s\n' "$i" > plain-src/bigdir/file-$i
done
for i in $(seq -w 0 255); do
  awk -v n="$i" 'BEGIN {
    for (j = 0; j < 256; j++)
      printf "worker-%s-line-%03d-abcdefghijklmnopqrstuvwxyz0123456789\\n", n, j
  }' > plain-src/concurrent/file-$i.dat
done
awk 'BEGIN {
  for (i = 0; i < 262144; i++)
    printf "throughput-%08d-abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ\\n", i
}' > plain-src/throughput.dat
mkfs.erofs -T0 --all-root --ignore-mtime -x-1 -Uclear \
  plain-nfs.erofs plain-src
```

Fixture and manifest hashes:

| Object | SHA256 |
|---|---|
| `plain-nfs.erofs` | `6928f05b58ce462596f9c7bf1d3442ad50d555dc5c31ed01d28e1a921a448bf2` |
| `metabox-nfs.erofs` | `8f493ed70d4be8096ab1adf8208622734d61a64903cf96a1ca1a1cc1854b1985` |
| multidevice primary | `fa2f35f59f63cc4a56d230ba960a6c1d27748506fccce081447cab0dc0d540fd` |
| multidevice blob 1 | `3aeb19c519636b61dc3ebd47c6286098deaa0990ff618ce3facbff4fd65907cd` |
| source manifest | `0eda6f88c3b6c4b7c8d81d940a237d1f63c958a7f6fe7bdd4f77d6b293736fa9` |
| `basic/regular.txt` | `93e7f327676f582356e9de2fcca7b714910227008fa1c5e7820e942c471edb6f` |
| `basic/subdir/nested.txt` | `6da159d7ea19e9d1253ccb0f2c67586e6e075459954e740ff8405546683d87c3` |
| `concurrent/file-000.dat` | `8435866622642b6dadf502b09f95af32cb01907ba3376e4f1fe96278e5ada692` |
| `throughput.dat` (21,757,952 bytes) | `0841effed82d1adf394b6834ce30d4d9eb5fc9427527e854ec1cb6bb4b86c119` |
| expected 12,050-name sorted list | `4fbf1b6103e8884223629c3fd03f4ec36061e65cd2a1becfc96670daa96f9d71` |

The metabox image was the previously qualified compressed-metabox fixture. The
multidevice fixture was the previously qualified primary-plus-one-blob image.

## FreeBSD 15 setup commands

The artifacts were copied to the existing VM through SSH port 9222. Password
material and askpass script contents were neither printed nor recorded.

```sh
scp -P 9222 erofs.ko plain-nfs.erofs metabox-nfs.erofs \
  tests/nfs_fh_tool.c root@127.0.0.1:/tmp/repo22-nfs-test/
ssh -p 9222 root@127.0.0.1
cd /tmp/repo22-nfs-test
sha256 erofs.ko plain-nfs.erofs metabox-nfs.erofs
cc -O2 -Wall -Wextra -std=c17 nfs_fh_tool.c -o nfs_fh_tool
```

The guest hashes matched the host hashes.

EROFS and NFS setup:

```sh
kldload /tmp/repo22-nfs-test/erofs.ko
mdconfig -a -t vnode -f /tmp/repo22-nfs-test/plain-nfs.erofs -u 42
mkdir -p /mnt/repo22-erofs
mount -t erofs -o ro /dev/md42 /mnt/repo22-erofs
mount -v | grep /mnt/repo22-erofs

printf '%s\n' \
  '/mnt/repo22-erofs -ro -maproot=root -network 127.0.0.0 -mask 255.0.0.0' \
  > /etc/exports
service rpcbind onestart
service mountd onestart
service nfsd onestart
service mountd onereload
rpcinfo -p 127.0.0.1
showmount -e 127.0.0.1
mount -v | grep /mnt/repo22-erofs
```

Before mountd reload the EROFS line did not contain `NFS exported`. Afterwards:

```text
/dev/md42 on /mnt/repo22-erofs (erofs, NFS exported, local, read-only, acls, ...)
```

NFSv2/v3 and mountd were registered over TCP and UDP. NFS components are built
into this GENERIC kernel: `kldload nfsd` and `kldload nfscl` reported
`already loaded or in kernel`, while rpcbind/mountd/nfsd operated normally.

## File-handle validation commands

Normal vnode types:

```sh
./nfs_fh_tool capture /mnt/repo22-erofs/basic/regular.txt regular.before.fh
./nfs_fh_tool capture /mnt/repo22-erofs/basic/subdir directory.fh
./nfs_fh_tool lcapture /mnt/repo22-erofs/basic/link-to-regular symlink.fh
./nfs_fh_tool capture /mnt/repo22-erofs/basic/test.fifo fifo.fh
./nfs_fh_tool describe regular.before.fh
./nfs_fh_tool describe directory.fh
./nfs_fh_tool describe symlink.fh
./nfs_fh_tool describe fifo.fh
./nfs_fh_tool stat regular.before.fh
./nfs_fh_tool stat directory.fh
./nfs_fh_tool stat symlink.fh
./nfs_fh_tool stat fifo.fh
./nfs_fh_tool cat regular.before.fh regular.fhopen.out
```

Observed handles:

| Type | NID | Generation | Result |
|---|---:|---:|---|
| regular | `0x74` | 1 | `fhstat` and `fhopen` PASS |
| directory | `0x76` | 1 | `fhstat` PASS |
| symlink (`lgetfh`) | `0x72` | 1 | `fhstat` PASS |
| FIFO | `0x79` | 1 | `fhstat` PASS |

All had `len=16` and `pad=0`. The regular full-handle SHA256 was:

```text
fb1778fc6bdc979e10d003dd6131a961e7532baa16273c5465935a665bf990f6
```

Malformed and stale classification:

```sh
./nfs_fh_tool mutate regular.before.fh bad-len.fh len 15
./nfs_fh_tool mutate regular.before.fh bad-pad.fh pad 1
./nfs_fh_tool mutate regular.before.fh bad-gen.fh gen 2
./nfs_fh_tool mutate regular.before.fh bad-nid.fh nid_hi 0xffffffff
./nfs_fh_tool expect-stat bad-len.fh EINVAL
./nfs_fh_tool expect-open bad-len.fh EINVAL
./nfs_fh_tool expect-stat bad-pad.fh EINVAL
./nfs_fh_tool expect-open bad-pad.fh EINVAL
./nfs_fh_tool expect-stat bad-gen.fh ESTALE
./nfs_fh_tool expect-open bad-gen.fh ESTALE
./nfs_fh_tool expect-stat bad-nid.fh ESTALE
./nfs_fh_tool expect-open bad-nid.fh ESTALE
```

All eight checks passed. `fhstat` exercised the shared-lock `VFS_FHTOVP` path;
`fhopen` exercised the exclusive-lock path.

Same explicit md-unit remount:

```sh
umount /mnt/repo22-erofs
mdconfig -d -u 42
mdconfig -a -t vnode -f plain-nfs.erofs -u 42
mount -t erofs -o ro /dev/md42 /mnt/repo22-erofs
./nfs_fh_tool capture /mnt/repo22-erofs/basic/regular.txt regular.after.fh
./nfs_fh_tool compare regular.before.fh regular.after.fh
./nfs_fh_tool cat regular.before.fh regular.remount.out
```

The complete handles were byte-identical and the pre-remount handle still read
the correct file after remount.

Non-export mount update rejection:

```sh
mount -u -o noexec /mnt/repo22-erofs
```

Result: `Operation not supported`; the mount remained exported, local,
read-only, and did not gain `noexec`.

## NFSv3 basic and restart tests

Four NFSv3 TCP READDIRPLUS clients were mounted:

```sh
mount_nfs -o nfsv3,tcp,rdirplus,readdirsize=512 \
  127.0.0.1:/mnt/repo22-erofs /mnt/repo22-nfs-512
mount_nfs -o nfsv3,tcp,rdirplus,readdirsize=1024 \
  127.0.0.1:/mnt/repo22-erofs /mnt/repo22-nfs-1024
mount_nfs -o nfsv3,tcp,rdirplus,readdirsize=4096 \
  127.0.0.1:/mnt/repo22-erofs /mnt/repo22-nfs-4096
mount_nfs -o nfsv3,tcp,rdirplus \
  127.0.0.1:/mnt/repo22-erofs /mnt/repo22-nfs-default
nfsstat -m
```

FreeBSD clamped the requested 512/1024/4096 values to an effective
`readdirsize=8192`; the default client used 65536. This was recorded as a client
environment limit, not a failure.

For each client:

```sh
cmp NFS/basic/regular.txt DIRECT/basic/regular.txt
test -d NFS/basic/subdir
test "$(readlink NFS/basic/link-to-regular)" = regular.txt
test -p NFS/basic/test.fifo
test "$(stat -f %i NFS/basic/regular.txt)" = \
  "$(stat -f %i DIRECT/basic/regular.txt)"
! touch NFS/write-must-fail
```

All passed. Writes failed with `Read-only file system`.

nfsd restart with an open descriptor:

```sh
exec 3< /mnt/repo22-nfs-default/basic/regular.txt
service nfsd onerestart
cat <&3 > open-fd-after-nfsd-restart.out
exec 3<&-
cmp open-fd-after-nfsd-restart.out /mnt/repo22-erofs/basic/regular.txt
```

The open descriptor and subsequent path reopen both passed. Direct file handles
captured before and after restart were byte-identical.

## Metabox bit-63 test

Commands:

```sh
mdconfig -a -t vnode -f metabox-nfs.erofs -u 43
mount -t erofs -o ro /dev/md43 /mnt/repo22-metabox
./nfs_fh_tool capture /mnt/repo22-metabox/long/file1.txt metabox.before.fh
./nfs_fh_tool describe metabox.before.fh
./nfs_fh_tool stat metabox.before.fh
./nfs_fh_tool cat metabox.before.fh metabox.fhopen.out
./nfs_fh_tool mutate metabox.before.fh metabox.bad-nid.fh \
  nid_lo 0xffffffff
./nfs_fh_tool expect-stat metabox.bad-nid.fh ESTALE
./nfs_fh_tool expect-open metabox.bad-nid.fh ESTALE
```

Observed handle:

```text
len=16 pad=0 nid=800000000000010a gen=1
ino=9223372036854776074
```

The metabox mount was exported with mountd and mounted over NFSv3. Direct,
`fhopen`, and NFS contents had SHA256:

```text
661b22d2a7bd94a7da35834d4cc3647eb527d48eb08acc75bde2e3a7691ed45e
```

The NFS and direct inode numbers matched. The out-of-range metabox mutation
returned `ESTALE` through both shared and exclusive paths.

## Multidevice boundary test

Normal NFS commands:

```sh
mdconfig -a -t vnode -f primary.img -u 44
mdconfig -a -t vnode -f blob1.img -u 45
mount -t erofs -o ro -o device.1=/dev/md45 /dev/md44 \
  /mnt/repo22-multidev
mount_nfs -o nfsv3,tcp,rdirplus,readdirsize=512 \
  127.0.0.1:/mnt/repo22-multidev /mnt/repo22-nfs-multidev
cmp /mnt/repo22-nfs-multidev/alpha.bin /mnt/repo22-multidev/alpha.bin
cmp /mnt/repo22-nfs-multidev/small.txt /mnt/repo22-multidev/small.txt
```

Normal multidevice NFS reads passed. Representative hashes:

```text
f779dbe3aeaa609beb32212579f3aea441f244621d13c977cb8b52d7ec3016d9  alpha.bin
5990b055a3c6d27681185a25e15ff21ba3cbdca7632faf4332233238e5f45aad  small.txt
001bcf4626e7d52f1bd76dee9a6c6e7d3a718c03602c5f9b07840a6c98b8fdc3  tree/beta.bin
```

Cold detached-provider test:

```sh
umount /mnt/repo22-multidev
mdconfig -d -u 44
mdconfig -a -t vnode -f primary.img -u 44
mdconfig -a -t vnode -f blob1.img -u 45
mount -t erofs -o ro -o device.1=/dev/md45 /dev/md44 \
  /mnt/repo22-multidev
./nfs_fh_tool capture /mnt/repo22-multidev/tree/beta.bin multidev-beta.fh
./nfs_fh_tool stat multidev-beta.fh
mdconfig -d -o force -u 45
! ./nfs_fh_tool cat multidev-beta.fh beta.after-orphan
```

The handle resolved to NID `0x36`, then the cold read returned:

```text
Device not configured
```

This preserved the multidevice `ENXIO` boundary error instead of converting it
to `ESTALE`, reading past the provider, or panicking.

## READDIRPLUS and stress

Every one of the four initial listings returned exactly 12,050 unique expected
names. The expected list and all four sorted outputs had SHA256:

```text
4fbf1b6103e8884223629c3fd03f4ec36061e65cd2a1becfc96670daa96f9d71
```

Dot handling:

```sh
ls -a1 /mnt/repo22-nfs-default/bigdir > nfs-dot-list
test "$(wc -l < nfs-dot-list)" -eq 12052
test "$(grep -cx '\.' nfs-dot-list)" -eq 1
test "$(grep -cx '\.\.' nfs-dot-list)" -eq 1
```

All passed.

Cold pagination used five unmount/remount/list/compare cycles on the small
client. Concurrency then used eight workers, three complete 12,050-entry rounds
each, distributed across the four mounts. All 29 cold/concurrent sorted outputs
had one unique SHA256 value, matching the expected list.

Parallel data/metadata load:

```sh
for M in 512 1024 4096 default; do
  find /mnt/repo22-nfs-$M/concurrent -type f -maxdepth 1 -print0 | \
    xargs -0 -n 1 -P 8 cat > /dev/null &
  find /mnt/repo22-nfs-$M/concurrent -type f -maxdepth 1 -print0 | \
    xargs -0 -n 1 -P 8 stat -f '%i %z' > /dev/null &
done
wait
```

All jobs completed successfully.

Final stress counters:

| Counter | Client | Server |
|---|---:|---:|
| requests / cache misses | 270,854 | 271,174 |
| lookup | 264,989 | 265,109 |
| read | 1,360 | 1,360 |
| READDIRPLUS | 2,512 | 2,512 |
| write | 0 | 0 |
| timed out | 0 | n/a |
| retries | 0 | n/a |

There were no READDIR RPCs because the clients negotiated READDIRPLUS.

Informational throughput:

```text
21757952 bytes transferred in 5.843937 seconds
3723167 bytes/second
```

No fixed throughput threshold was used.

## Test matrix

| Test | Actual | Status |
|---|---|---|
| isolated `./build.sh` | success; module hash recorded | PASS |
| unresolved `bcmp` | none | PASS |
| FreeBSD helper compile | `-Wall -Wextra`, no warnings | PASS |
| final `kldload` / `kldunload` | load, unload, reload, unload succeeded | PASS |
| initial mount export flag | absent before mountd | PASS |
| export-only `MNT_UPDATE` | mountd installed export | PASS |
| non-export `MNT_UPDATE` | `EOPNOTSUPP`, flags unchanged | PASS |
| regular file | direct, `fhopen`, NFS content match | PASS |
| directory | handle and NFS traversal | PASS |
| symlink | `lgetfh`, target and NFS lookup | PASS |
| FIFO | handle metadata and NFS type | PASS |
| read-only behavior | direct and all NFS mounts rejected writes | PASS |
| 16-byte handle ABI | len/pad/NID hi/NID lo/gen observed | PASS |
| generation | `1`, matching `va_gen` | PASS |
| shared/exclusive lock contract | `fhstat` and `fhopen` passed | PASS |
| malformed len/pad | `EINVAL` | PASS |
| stale generation/NID | `ESTALE` | PASS |
| same `md42` remount | complete handle byte-identical | PASS |
| nfsd restart | open descriptor and reopen passed | PASS |
| metabox bit 63 | real `0x800000000000010a` round-trip | PASS |
| metabox boundary mutation | `ESTALE` | PASS |
| multidevice NFS | external-data reads matched | PASS |
| detached external device | cold handle read preserved `ENXIO` | PASS |
| 12,050-entry listings | exact count and names on four clients | PASS |
| cookie pagination | five cold remounts, no duplicate/omission | PASS |
| dot omitted | one `.` and one `..`, 12,052 total | PASS |
| concurrent traversal | 8 workers x 3 rounds, all hashes equal | PASS |
| concurrent read/stat | all jobs completed | PASS |
| READDIRPLUS | client/server 2,512 | PASS |
| NFS timeouts/retries | 0 / 0 | PASS |
| successful NFS writes | 0 | PASS |
| throughput | 3,723,167 B/s, informational | INFO |

## dmesg and cleanup

The functional matrix captured dmesg before loading the test module and after
all functional tests plus a load/unload cycle. Both files contained the same
two pre-existing historical lines and compared byte-for-byte equal:

```text
interface erofs.1 already present in the KLD 'erofs-repo22-xattrfix.ko'!
md0: truncating fractional last sector by 14 bytes
```

No EROFS error, stale-handle message, trap, panic, or new kernel line occurred
during the functional matrix.

The first cleanup attempt used the wrong order: it stopped nfsd and unmounted
the direct EROFS mounts before one hard loopback client was fully gone. This
left a `mount -p` process in uninterruptible NFS wait. When rpcbind/nfsd were
temporarily restarted without the underlying EROFS export to release those
clients, the NFS client logged five `fileid changed` messages. These messages
occurred after the byte-identical functional dmesg snapshot and are classified
as **environment cleanup artifact**, not an EROFS functional failure.

The VM was rebooted to remove the blocked cleanup process. Post-reboot audit:

```text
erofs_mounts=0
nfs_mounts=0
md_units=0
erofs_modules=0
nfsd_pids=0 mountd_pids=0 rpcbind_pids=0
exports_exists=no
```

A final correctly ordered clean rerun then performed:

```sh
kldload erofs.ko
mdconfig -a -t vnode -f plain-nfs.erofs -u 42
mount -t erofs -o ro /dev/md42 /mnt/repo22-erofs
service rpcbind onestart
service mountd onestart
service nfsd onestart
mount_nfs -o nfsv3,tcp,rdirplus,readdirsize=512 \
  127.0.0.1:/mnt/repo22-erofs /mnt/repo22-nfs-clean
cmp /mnt/repo22-nfs-clean/basic/regular.txt \
  /mnt/repo22-erofs/basic/regular.txt
test "$(find /mnt/repo22-nfs-clean/bigdir -type f -maxdepth 1 | wc -l)" \
  -eq 12050
umount /mnt/repo22-nfs-clean
: > /etc/exports
service mountd onereload
service nfsd onestop
service mountd onestop
service rpcbind onestop
umount /mnt/repo22-erofs
mdconfig -d -u 42
kldunload erofs
```

The clean rerun's pre/post dmesg files compared byte-for-byte equal. Final
state after that rerun:

```text
clean_rerun_erofs=0 nfs=0 md=0 module=0 services=0 exports=absent
```

## Limitations

- **ENVIRONMENT-LIMIT**: FreeBSD clamped requested readdir sizes 512, 1024, and
  4096 to an effective minimum of 8192. The test still exercised a much smaller
  value than the 65536 default and produced 2,512 real READDIRPLUS calls.
- **ENVIRONMENT-CLEANUP-ARTIFACT**: the five post-test `fileid changed` lines
  were caused by the explicitly documented wrong cleanup order. A reboot and a
  correctly ordered rerun left no new dmesg lines or resources.
- **KERNEL-FAIL**: none.
- **NOT RUN**: none of the required functional cases were omitted.