227 lines
8.5 KiB
Markdown
227 lines
8.5 KiB
Markdown
# G7 Boundary and Stress Manual Setup
|
|
|
|
This setup applies to exactly `TC120` through `TC130`: 11 tests, with no
|
|
additional test IDs. Run each test's Markdown commands directly. Do not use a
|
|
CI job, runner, or test wrapper.
|
|
|
|
## Deterministic fixtures
|
|
|
|
The host requires Python 3, erofs-utils 1.8.6, GNU tar, QEMU, and at least
|
|
2 GiB of free working space. Generate two fresh fixture directories; the
|
|
helper rejects an existing output path.
|
|
|
|
```sh
|
|
REPO=/path/to/worktree/repo-community/repo22
|
|
RUN=/work/build/repo22-g7-$(date -u +%Y%m%dT%H%M%SZ)
|
|
mkdir -p "$RUN"
|
|
cd "$REPO"
|
|
python3 -B tests/g7_fixtures.py create --output "$RUN/fixtures-a"
|
|
python3 -B tests/g7_fixtures.py create --output "$RUN/fixtures-b"
|
|
python3 -B tests/g7_fixtures.py verify --output "$RUN/fixtures-a"
|
|
cmp "$RUN/fixtures-a/SOURCE-INVENTORY.tsv" \
|
|
"$RUN/fixtures-b/SOURCE-INVENTORY.tsv"
|
|
cmp "$RUN/fixtures-a/SOURCE-SHA256SUMS" \
|
|
"$RUN/fixtures-b/SOURCE-SHA256SUMS"
|
|
cmp "$RUN/fixtures-a/SHA256SUMS" "$RUN/fixtures-b/SHA256SUMS"
|
|
sha256sum "$RUN/fixtures-a/SOURCE-INVENTORY.tsv" \
|
|
"$RUN/fixtures-a/SOURCE-SHA256SUMS" \
|
|
"$RUN/fixtures-a/SHA256SUMS" \
|
|
"$RUN/fixtures-a/fixture-manifest.json"
|
|
```
|
|
|
|
`SOURCE-INVENTORY.tsv` records the exact relative path, size, and SHA256 of
|
|
every source file. The helper re-hashes the complete inventory, checks exact
|
|
counts and names, checks all sparse markers and allocated-block usage, runs
|
|
`fsck.erofs`, reopens the 4 GiB boundary inode with `dump.erofs`, and verifies
|
|
the image checksums. `boundaries.erofs` covers TC120-TC125;
|
|
`workloads.erofs` covers TC126-TC130.
|
|
|
|
The practical sparse boundary is 4 GiB + 4097 bytes. It crosses signed 32-bit,
|
|
2 GiB, unsigned 32-bit, 4 GiB, block, hole, and EOF boundaries without
|
|
claiming that a 16 TiB image is practical in this VM. The source remains
|
|
sparse and every selected range is compared to that exact source in TC121.
|
|
|
|
## Dedicated FreeBSD 15 VM
|
|
|
|
Create a new qcow2 overlay and use only SSH port 9227:
|
|
|
|
```sh
|
|
qemu-img create -f qcow2 -F qcow2 \
|
|
-b /work/build/vm-freebsd-dev-base.qcow2 \
|
|
"$RUN/freebsd15-overlay.qcow2"
|
|
qemu-system-x86_64 -accel tcg,thread=multi -cpu qemu64 \
|
|
-m 6144 -smp 4 \
|
|
-drive file="$RUN/freebsd15-overlay.qcow2",if=virtio,format=qcow2 \
|
|
-netdev user,id=net0,hostfwd=tcp:127.0.0.1:9227-:22 \
|
|
-device virtio-net-pci,netdev=net0 -display none \
|
|
-serial file:"$RUN/freebsd15-serial.log" -monitor none \
|
|
-pidfile "$RUN/freebsd15-qemu.pid" \
|
|
-D "$RUN/freebsd15-qemu.log" -daemonize
|
|
```
|
|
|
|
Record the initial guest identity and resource-control state:
|
|
|
|
```sh
|
|
uname -a
|
|
freebsd-version -ku
|
|
sysctl -n kern.osreldate
|
|
sha256 /boot/kernel/kernel
|
|
sysctl kern.racct.enable
|
|
rctl
|
|
vmstat -H 1 3
|
|
mdconfig -l
|
|
mount -p | awk '$3 == "erofs"'
|
|
```
|
|
|
|
If RACCT/RCTL is disabled, enable the FreeBSD loader tunable in this overlay
|
|
and reboot it. Do not use a jail as a memory-limit command.
|
|
|
|
```sh
|
|
grep -q '^kern.racct.enable=' /boot/loader.conf || \
|
|
printf 'kern.racct.enable="1"\n' >> /boot/loader.conf
|
|
grep '^kern.racct.enable=' /boot/loader.conf
|
|
shutdown -r now
|
|
```
|
|
|
|
After reconnecting, require `sysctl -n kern.racct.enable` to print `1` and
|
|
record `rctl` plus `vmstat -H 1 3` again.
|
|
|
|
## Exact-source KLD and guest inputs
|
|
|
|
Commit the helper and corrected Markdown before building. Then archive the
|
|
exact commit and the tracked FreeBSD 15 sys tree. The final report may be a
|
|
later documentation-only commit, but its `repo22/src` tree hash must equal the
|
|
build input tree hash.
|
|
|
|
```sh
|
|
cd /path/to/worktree
|
|
BUILD_COMMIT=$(git rev-parse HEAD)
|
|
git status --short
|
|
printf '%s\n' "$BUILD_COMMIT" > "$RUN/source.commit"
|
|
git rev-parse "$BUILD_COMMIT:repo-community/repo22/src" \
|
|
> "$RUN/repo22-src.tree"
|
|
git archive --format=tar "$BUILD_COMMIT" repo-community/repo22 | \
|
|
gzip -n > "$RUN/repo22-source.tar.gz"
|
|
git archive --format=tar "$BUILD_COMMIT" dev-freebsd-releng/sys | \
|
|
gzip -n > "$RUN/freebsd-sys-source.tar.gz"
|
|
tar --sparse --format=gnu -C "$RUN/fixtures-a" \
|
|
-cf "$RUN/boundaries-source.tar" sources/boundaries
|
|
tar --sparse --format=gnu -C "$RUN/fixtures-a" \
|
|
-cf "$RUN/workloads-source.tar" sources/workloads
|
|
gzip -n "$RUN/boundaries-source.tar"
|
|
gzip -n "$RUN/workloads-source.tar"
|
|
sha256sum "$RUN/repo22-source.tar.gz" \
|
|
"$RUN/freebsd-sys-source.tar.gz" \
|
|
"$RUN/boundaries-source.tar.gz" \
|
|
"$RUN/workloads-source.tar.gz" > "$RUN/source-archives.sha256"
|
|
```
|
|
|
|
Transfer the archives, images, and fixture metadata to the dedicated guest.
|
|
Authentication configuration stays outside the repository.
|
|
|
|
```sh
|
|
ssh -p 9227 [email protected] 'mkdir -p /root/repo22-g7-transfer'
|
|
scp -O -P 9227 "$RUN/repo22-source.tar.gz" \
|
|
"$RUN/freebsd-sys-source.tar.gz" \
|
|
"$RUN/boundaries-source.tar.gz" "$RUN/workloads-source.tar.gz" \
|
|
"$RUN/fixtures-a/images/boundaries.erofs" \
|
|
"$RUN/fixtures-a/images/workloads.erofs" \
|
|
"$RUN/fixtures-a/SOURCE-INVENTORY.tsv" \
|
|
"$RUN/fixtures-a/SOURCE-SHA256SUMS" \
|
|
"$RUN/fixtures-a/SHA256SUMS" \
|
|
"$RUN/fixtures-a/fixture-manifest.json" \
|
|
"$RUN/fixtures-a/DEEP-PATH" "$RUN/fixtures-a/LONG-NAME" \
|
|
"$RUN/fixtures-a/SPARSE-RANGES.tsv" \
|
|
"$RUN/source.commit" "$RUN/repo22-src.tree" \
|
|
"$RUN/source-archives.sha256" \
|
|
[email protected]:/root/repo22-g7-transfer/
|
|
```
|
|
|
|
Build and verify natively in the guest:
|
|
|
|
```sh
|
|
mkdir -p /root/freebsd-src /root/repo22-g7-src \
|
|
/root/repo22-g7/fixtures/images /root/repo22-g7/fixtures/sources \
|
|
/root/repo22-g7/evidence
|
|
tar -xzf /root/repo22-g7-transfer/freebsd-sys-source.tar.gz \
|
|
-C /root/freebsd-src --strip-components 1
|
|
tar -xzf /root/repo22-g7-transfer/repo22-source.tar.gz \
|
|
-C /root/repo22-g7-src --strip-components 2
|
|
tar -xzf /root/repo22-g7-transfer/boundaries-source.tar.gz \
|
|
-C /root/repo22-g7/fixtures
|
|
tar -xzf /root/repo22-g7-transfer/workloads-source.tar.gz \
|
|
-C /root/repo22-g7/fixtures
|
|
cp /root/repo22-g7-transfer/*.erofs /root/repo22-g7/fixtures/images/
|
|
cp /root/repo22-g7-transfer/SOURCE-INVENTORY.tsv \
|
|
/root/repo22-g7-transfer/SOURCE-SHA256SUMS \
|
|
/root/repo22-g7-transfer/SHA256SUMS \
|
|
/root/repo22-g7-transfer/fixture-manifest.json \
|
|
/root/repo22-g7-transfer/DEEP-PATH \
|
|
/root/repo22-g7-transfer/LONG-NAME \
|
|
/root/repo22-g7-transfer/SPARSE-RANGES.tsv \
|
|
/root/repo22-g7/fixtures/
|
|
cd /root/repo22-g7/fixtures
|
|
sha256sum -c SHA256SUMS
|
|
sha256sum -c SOURCE-SHA256SUMS
|
|
stat -f 'size=%z blocks=%b blocksize=%k name=%N' \
|
|
sources/boundaries/maximum/sparse-boundary.bin
|
|
cd /root/repo22-g7-src
|
|
grep -E '^(REVISION|BRANCH)=' /root/freebsd-src/sys/conf/newvers.sh
|
|
env WITH_ZSTDIO=1 FREEBSD_SRC=/root/freebsd-src ./build.sh
|
|
cp build/erofs.ko /root/repo22-g7/erofs.ko
|
|
cc -std=c11 -O2 -Wall -Wextra -Werror \
|
|
-o /root/repo22-g7/g7_probe tests/g7_probe.c
|
|
sha256 /root/repo22-g7/erofs.ko /root/repo22-g7/g7_probe
|
|
file /root/repo22-g7/erofs.ko
|
|
```
|
|
|
|
Record the build commit, source tree hash, `WITH_ZSTDIO=1`, archive hashes,
|
|
FreeBSD source `REVISION`/`BRANCH`, KLD SHA256, probe SHA256, guest identity,
|
|
kernel SHA256, and TCG/QEMU configuration in the report.
|
|
|
|
## Manual lifecycle
|
|
|
|
Load the exact module by full path and record its file ID and pathname:
|
|
|
|
```sh
|
|
kldload /root/repo22-g7/erofs.ko
|
|
kldstat -v | tee /root/repo22-g7/evidence/kldstat-loaded.txt
|
|
KLD_ID=$(kldstat -v | awk '$NF == "(/root/repo22-g7/erofs.ko)" { print $1 }')
|
|
test -n "$KLD_ID"
|
|
printf '%s\n' "$KLD_ID" > /root/repo22-g7/evidence/kld.id
|
|
dmesg > /root/repo22-g7/evidence/dmesg-before.txt
|
|
```
|
|
|
|
Each TC attaches one fresh vnode-backed md provider, mounts read-only, records
|
|
all correctness and metrics evidence, unmounts, detaches that exact provider,
|
|
and verifies both are gone. Never use `killall`; concurrent tests persist
|
|
every child PID, wait every child, and record every exit code.
|
|
|
|
TC128 and TC129 metrics are recording-only under QEMU TCG. Their PASS gate is
|
|
read completion and exact source hash/byte comparison, not a fixed MB/s,
|
|
IOPS, or latency threshold. TC127 uses FreeBSD RCTL `vmemoryuse:deny` with
|
|
cooperating allocation probes, records allocation failure and reclamation,
|
|
and verifies reads while pressure is held.
|
|
|
|
## Final cleanup
|
|
|
|
After TC130, require zero child processes, EROFS mounts, md providers, and
|
|
RCTL rules created by G7. Unload the exact KLD file ID, compare dmesg, and
|
|
power off the dedicated guest.
|
|
|
|
```sh
|
|
mount -p | awk '$3 == "erofs" { print }'
|
|
mdconfig -l
|
|
rctl
|
|
kldstat -v | grep -A2 -B2 '/root/repo22-g7/erofs.ko'
|
|
KLD_ID=$(cat /root/repo22-g7/evidence/kld.id)
|
|
kldunload -i "$KLD_ID"
|
|
test -z "$(kldstat -v | grep '/root/repo22-g7/erofs.ko')"
|
|
dmesg > /root/repo22-g7/evidence/dmesg-after.txt
|
|
shutdown -p now
|
|
```
|
|
|
|
Remove the host overlay, QEMU logs/PID file, source tars, temporary sparse
|
|
probe data, and Python bytecode caches only after evidence has been copied
|
|
into the committed report. Confirm port 9227 is released.
|