# repo22 real multi-device manual test report

Date: 2026-08-08 19:37 UTC
Baseline: `96e22cc713cc57180ce3ecb2b98852f090e3868b` plus this multi-device batch
Guest: FreeBSD 15.0-RELEASE-p8 amd64, QEMU TCG
Host tools: erofs-utils 1.8.6

## Result

**PASS**. Real external providers, explicit slot mapping, flatdev, unified
addresses, 48-bit fields, failure rollback, forced GEOM orphaning, and the
single-device regressions all passed. There were no new dmesg lines, traps, or
panics. Final EROFS mount, md-provider, and loaded-module counts were zero.

## Build and module

Commands:

```sh
git diff --check
./build.sh
nm -u build/erofs.ko | awk '$NF == "bcmp" {bad=1} END {exit bad}'
sha256sum build/erofs.ko
```

Actual:

- `git diff --check`: PASS.
- Cross-build against `/work/dev-freebsd-releng`: PASS.
- Unresolved `bcmp`: none, PASS.
- Final `build/erofs.ko` SHA256:
  `aa6708256246b303f4be2a8d516bf503f8a232df1ad3f466f5685573f221c180`.
- Final `kldload` and `kldunload`: PASS.

## Fixture commands

The legal baseline was generated by erofs-utils 1.8.6, not by synthesizing an
EROFS image from scratch:

```sh
WORK=/work/build/repo22-multidev-fixtures-20260808
mkdir -p "$WORK/src/cross"
python3 - <<'PY'
from pathlib import Path
import hashlib
root = Path('/work/build/repo22-multidev-fixtures-20260808/src')
def content(label, blocks, tail=0):
    out = bytearray()
    for block in range(blocks):
        seed = f'{label}:block:{block}'.encode()
        chunk = bytearray()
        counter = 0
        while len(chunk) < 4096:
            chunk += hashlib.sha256(
                seed + counter.to_bytes(4, 'little')).digest()
            counter += 1
        out += chunk[:4096]
    if tail:
        out += hashlib.sha256(f'{label}:tail'.encode()).digest()[:tail]
    return bytes(out)
root.joinpath('cross/striped.bin').write_bytes(content('striped', 24, 173))
root.joinpath('cross/second.bin').write_bytes(content('second', 17, 29))
root.joinpath('control.txt').write_text(
    'repo22 multidev deterministic control\n', encoding='ascii')
PY
truncate -s 0 "$WORK/base.blob"
mkfs.erofs -T0 --all-root --chunksize=4096 \
    --blobdev="$WORK/base.blob" "$WORK/base.primary" "$WORK/src"
```

erofs-utils 1.8.6 accepts one `--blobdev`; direct generation of two or more
external slots is therefore **MKFS-UNAVAILABLE**. Multi-slot fixtures were made
by a structured, assertion-driven patch of the legal baseline:

1. `dump.erofs --path=PATH -e base.primary` supplied each NID and file size.
2. The patch asserted compact chunk layout, zero xattr size, index format, old
   device ID 1, and every old block address.
3. A complete physical chunk, rounded to 4096 bytes for the final partial
   chunk, was copied into a deterministic round-robin blob.
4. Each index was rewritten as little-endian
   `(startblk_hi, device_id, startblk_lo)`.
5. One zeroed primary block was appended for the enlarged device table;
   `blocks_lo`, `extra_devices`, and `devt_slotoff` were updated. This avoids
   overwriting inode metadata merely to add slots.
6. Each 128-byte slot was written at offsets 64/68/72/74 with `blocks_lo`,
   `uniaddr_lo`, `blocks_hi`, and `uniaddr_hi`.
7. The EROFS CRC32C was recomputed over bytes 1024 through 4095 with polynomial
   `0x82f63b78` and initial value `0xffffffff`.

The asserted patch manifest was:

```text
/control.txt nid=44 size=38 chunk=4096 indexes=1 base=1440
/cross/second.bin nid=50 size=69661 chunk=4096 indexes=18 base=1632
/cross/striped.bin nid=56 size=98336 chunk=32768 indexes=4 base=1824
```

Pre-kernel qualification:

```sh
fsck.erofs --device=two.blob1 --device=two.blob2 \
    --extract=two.extract two.primary
fsck.erofs --device=three.blob1 --device=three.blob2 \
    --device=three.blob3 --extract=three.extract three.primary
fsck.erofs --device=four.blob1 --device=four.blob2 \
    --device=four.blob3 --device=four.blob4 \
    --extract=four.extract four.primary
```

All extracted source SHA256 values matched. The 48-bit feature and patched
`device_id=0 + uniaddr` extraction are not understood by erofs-utils 1.8.6;
those layouts were qualified by the FreeBSD kernel SHA256 tests below and the
Linux 7.1 `erofs_map_dev` semantics.

FreeBSD attachment and mount commands used one md provider per image/blob:

```sh
mdconfig -a -t vnode -f primary -u 90
mdconfig -a -t vnode -f blob1 -u 91
mdconfig -a -t vnode -f blob2 -u 92
mount -t erofs -o ro -o device.2=/dev/md92 \
    -o device.1=/dev/md91 /dev/md90 /mnt/repo22-multidev
```

The reversed option order was intentional. Flatdev used only:

```sh
mdconfig -a -t vnode -f two.flat -u 90
mount -t erofs -o ro /dev/md90 /mnt/repo22-multidev
```

## Fixture SHA256

| Fixture | SHA256 |
|---|---|
| one-blob primary | `fa2f35f59f63cc4a56d230ba960a6c1d27748506fccce081447cab0dc0d540fd` |
| one-blob slot 1 | `3aeb19c519636b61dc3ebd47c6286098deaa0990ff618ce3facbff4fd65907cd` |
| two-slot primary | `46d76d5cc311f97f263ea4c20a2510338f5b11da9d71e0c73f69ae3751afd696` |
| two-slot blob 1 | `53853d033adf154a956192d9813726888c3f9575ecd13fa45a26aea5c4f07eaf` |
| two-slot blob 2 | `9ee3e3ad420024ff1945d92469d3d337817977f28c5b745c4ca7b1664f4d6584` |
| two-slot flatdev | `4b561db1eb0048989256f3c51504b4611881069bf283fbfe2051847e8162301e` |
| three-slot primary | `4f2dc4ee6385f31139151e1b04279a8fb90fcd2cbba928ee6949df62f1bb9585` |
| three-slot blobs 1/2/3 | `12c5d73cb12e1bb5fd1a9d9715ec88d2132dbaa8df62e2b5aad3cfb454116a3a`, `3d1574254a97e572fca323f153bb2a46ebdfb060a0765298700768e39a7f91d8`, `ba42a2a3724771975c27a43ac8d64a2568515fb4e8ad14b29cc09a7a865ffba0` |
| four-slot primary | `d3366dfb5f4c98db2db669ead3209a0abbba5df989d9aea7e1217aff46f0154b` |
| four-slot blobs 1/2/3/4 | `ad24e257baf97f855f756cc917878a9616858860ddbfab4ffe3cc362e9233662`, `eb55bd05f873dd8732bcbe35a0818ef232a9eff6fedec618f5f949253d7cd2d1`, `59f9a3aed34cc8598e6a96381b55a18f6a0e6674e3e26921834948ad333cb3c5`, `d5080743a85e47752cd33e86e26690b8978d314e685b27d449d5394c44e54244` |
| device ID 0 + uniaddr | `e4cf71f76f2a2e690e905e4135c21ce29d647be5dd2b6ce567ac19f82d1875d2` |
| 48-bit high uniaddr | `00d49beea1237125886f13912bf062153b992582c1cf3a2494b44679285b192e` |
| overlapping-slot negative | `6ecfc77b688ac1931f3e904f48fa43a81c05a6047c21ccfc4636e9e07c17d5cf` |
| mapped-ID-3 negative | `dd6ddc6943b7edd2009a48999d03a2d5b69a9ce7d820357527dbcdb52adb2afc` |

Source SHA256 values for the cross-device fixture:

```text
6a38f93ca44de4affcef23ad5979505225f21e5b6912b5c886866170c007b16c  ./control.txt
45b7df16a5833013042694c2e8171d782102bf20abbc6c1ec2789518d5a65125  ./cross/second.bin
7ab83ca35cc4bb361bf60b5619d3f55b717fb8e2a7a81a2f069dce859efe8de9  ./cross/striped.bin
```

## Test matrix

| Test | Expected | Actual | Status |
|---|---|---|---|
| TC093 plain single device | full SHA256 match | both files matched | PASS |
| TC093 chunk single device | device ID 0 reads primary | both files matched | PASS |
| compression regression | LZ4 data unchanged | both files matched | PASS |
| TC098 fragments | packed-inode file matches | `65b9a59e...e7cd` | PASS |
| metabox regression | cold metabox-backed files match | `f1` and `f10` matched known hashes | PASS |
| TC006/TC099 primary + one blob | all external chunks use slot 1 | three files matched | PASS |
| TC094 two external slots | reversed option order, cross-slot SHA256 | all three files matched | PASS |
| TC100 four providers total | slots 1-3, order 3/1/2 | all three files matched | PASS |
| additional four external slots | file crosses four blobs | SHA256 matched | PASS |
| statfs combined blocks | sum primary and external blocks | 184 KiB reported for 46 blocks | PASS |
| TC101 flatdev | non-zero IDs add `uniaddr` on primary | SHA256 matched | PASS |
| TC101 device ID 0 + uniaddr | unified range selects external provider | SHA256 matched | PASS |
| 48-bit root/index/uniaddr | high fields are not truncated | source SHA256 matched | PASS |

## Error and rollback matrix

| Case | Expected | Actual | Status |
|---|---|---|---|
| split primary, no external options | `ENXIO` flatdev media bound | `Device not configured` | PASS |
| missing slot 2 | `ENXIO` | `external devices don't match ... Device not configured` | PASS |
| short slot 2 | `ENXIO` | `Device not configured` | PASS |
| swapped short/long providers | fail before root | `Device not configured` | PASS |
| same provider in two slots | `EINVAL` | `Invalid argument` | PASS |
| overlapping unified ranges | `EINTEGRITY` | `Integrity check failed` | PASS |
| explicit writable request | `EROFS` | `Read-only file system` | PASS |
| mapped device ID 3 with two slots | read-time `ENODEV` | `Operation not supported by device` | PASS |
| second concurrent reuse mount | `EBUSY` | `Device busy`; first mount remained readable | PASS |
| normal detach while mounted | `EBUSY` | `mdconfig ... Device busy` | PASS |
| forced orphan then cold slot-2 read | `ENXIO` | `Device not configured` | PASS |
| unmount after forced orphan | clean close of detached consumer | succeeded | PASS |

Every mount failure was followed immediately by successful md detach. Opened
external devices were therefore rolled back, including failures after one or
more earlier slots had opened.

## dmesg and cleanup

The final matrix captured `dmesg` before and after all tests:

```text
before_lines=2
after_lines=2
no-new-lines
```

The two pre-existing lines were unchanged historical messages:

```text
interface erofs.1 already present in the KLD 'erofs-repo22-xattrfix.ko'!
md0: truncating fractional last sector by 14 bytes
```

Final state:

```text
kldstat -n erofs: empty
mdconfig -l: empty
mount -p | grep erofs: empty
```

## Tool limitations

- **MKFS-UNAVAILABLE**: erofs-utils 1.8.6 cannot directly emit more than one
  external blob option in this environment. Multi-slot images were derived
  from a legal mkfs image with deterministic, asserted structural patches.
- **MKFS-UNAVAILABLE**: erofs-utils 1.8.6 rejects the experimental 48-bit
  incompat feature and does not correctly extract the patched device-ID-0
  unified-address control. Both were verified by complete FreeBSD kernel
  SHA256 reads.
- **KERNEL-FAIL**: none.