update
This commit is contained in:
@@ -0,0 +1,226 @@
|
||||
# G7 Boundary and Stress Manual Setup
|
||||
|
||||
This setup applies to exactly `TC120` through `TC130`: 11 tests, with no
|
||||
additional test IDs. Run each test's Markdown commands directly. Do not use a
|
||||
CI job, runner, or test wrapper.
|
||||
|
||||
## Deterministic fixtures
|
||||
|
||||
The host requires Python 3, erofs-utils 1.8.6, GNU tar, QEMU, and at least
|
||||
2 GiB of free working space. Generate two fresh fixture directories; the
|
||||
helper rejects an existing output path.
|
||||
|
||||
```sh
|
||||
REPO=/path/to/worktree/repo-community/repo22
|
||||
RUN=/work/build/repo22-g7-$(date -u +%Y%m%dT%H%M%SZ)
|
||||
mkdir -p "$RUN"
|
||||
cd "$REPO"
|
||||
python3 -B tests/g7_fixtures.py create --output "$RUN/fixtures-a"
|
||||
python3 -B tests/g7_fixtures.py create --output "$RUN/fixtures-b"
|
||||
python3 -B tests/g7_fixtures.py verify --output "$RUN/fixtures-a"
|
||||
cmp "$RUN/fixtures-a/SOURCE-INVENTORY.tsv" \
|
||||
"$RUN/fixtures-b/SOURCE-INVENTORY.tsv"
|
||||
cmp "$RUN/fixtures-a/SOURCE-SHA256SUMS" \
|
||||
"$RUN/fixtures-b/SOURCE-SHA256SUMS"
|
||||
cmp "$RUN/fixtures-a/SHA256SUMS" "$RUN/fixtures-b/SHA256SUMS"
|
||||
sha256sum "$RUN/fixtures-a/SOURCE-INVENTORY.tsv" \
|
||||
"$RUN/fixtures-a/SOURCE-SHA256SUMS" \
|
||||
"$RUN/fixtures-a/SHA256SUMS" \
|
||||
"$RUN/fixtures-a/fixture-manifest.json"
|
||||
```
|
||||
|
||||
`SOURCE-INVENTORY.tsv` records the exact relative path, size, and SHA256 of
|
||||
every source file. The helper re-hashes the complete inventory, checks exact
|
||||
counts and names, checks all sparse markers and allocated-block usage, runs
|
||||
`fsck.erofs`, reopens the 4 GiB boundary inode with `dump.erofs`, and verifies
|
||||
the image checksums. `boundaries.erofs` covers TC120-TC125;
|
||||
`workloads.erofs` covers TC126-TC130.
|
||||
|
||||
The practical sparse boundary is 4 GiB + 4097 bytes. It crosses signed 32-bit,
|
||||
2 GiB, unsigned 32-bit, 4 GiB, block, hole, and EOF boundaries without
|
||||
claiming that a 16 TiB image is practical in this VM. The source remains
|
||||
sparse and every selected range is compared to that exact source in TC121.
|
||||
|
||||
## Dedicated FreeBSD 15 VM
|
||||
|
||||
Create a new qcow2 overlay and use only SSH port 9227:
|
||||
|
||||
```sh
|
||||
qemu-img create -f qcow2 -F qcow2 \
|
||||
-b /work/build/vm-freebsd-dev-base.qcow2 \
|
||||
"$RUN/freebsd15-overlay.qcow2"
|
||||
qemu-system-x86_64 -accel tcg,thread=multi -cpu qemu64 \
|
||||
-m 6144 -smp 4 \
|
||||
-drive file="$RUN/freebsd15-overlay.qcow2",if=virtio,format=qcow2 \
|
||||
-netdev user,id=net0,hostfwd=tcp:127.0.0.1:9227-:22 \
|
||||
-device virtio-net-pci,netdev=net0 -display none \
|
||||
-serial file:"$RUN/freebsd15-serial.log" -monitor none \
|
||||
-pidfile "$RUN/freebsd15-qemu.pid" \
|
||||
-D "$RUN/freebsd15-qemu.log" -daemonize
|
||||
```
|
||||
|
||||
Record the initial guest identity and resource-control state:
|
||||
|
||||
```sh
|
||||
uname -a
|
||||
freebsd-version -ku
|
||||
sysctl -n kern.osreldate
|
||||
sha256 /boot/kernel/kernel
|
||||
sysctl kern.racct.enable
|
||||
rctl
|
||||
vmstat -H 1 3
|
||||
mdconfig -l
|
||||
mount -p | awk '$3 == "erofs"'
|
||||
```
|
||||
|
||||
If RACCT/RCTL is disabled, enable the FreeBSD loader tunable in this overlay
|
||||
and reboot it. Do not use a jail as a memory-limit command.
|
||||
|
||||
```sh
|
||||
grep -q '^kern.racct.enable=' /boot/loader.conf || \
|
||||
printf 'kern.racct.enable="1"\n' >> /boot/loader.conf
|
||||
grep '^kern.racct.enable=' /boot/loader.conf
|
||||
shutdown -r now
|
||||
```
|
||||
|
||||
After reconnecting, require `sysctl -n kern.racct.enable` to print `1` and
|
||||
record `rctl` plus `vmstat -H 1 3` again.
|
||||
|
||||
## Exact-source KLD and guest inputs
|
||||
|
||||
Commit the helper and corrected Markdown before building. Then archive the
|
||||
exact commit and the tracked FreeBSD 15 sys tree. The final report may be a
|
||||
later documentation-only commit, but its `repo22/src` tree hash must equal the
|
||||
build input tree hash.
|
||||
|
||||
```sh
|
||||
cd /path/to/worktree
|
||||
BUILD_COMMIT=$(git rev-parse HEAD)
|
||||
git status --short
|
||||
printf '%s\n' "$BUILD_COMMIT" > "$RUN/source.commit"
|
||||
git rev-parse "$BUILD_COMMIT:repo-community/repo22/src" \
|
||||
> "$RUN/repo22-src.tree"
|
||||
git archive --format=tar "$BUILD_COMMIT" repo-community/repo22 | \
|
||||
gzip -n > "$RUN/repo22-source.tar.gz"
|
||||
git archive --format=tar "$BUILD_COMMIT" dev-freebsd-releng/sys | \
|
||||
gzip -n > "$RUN/freebsd-sys-source.tar.gz"
|
||||
tar --sparse --format=gnu -C "$RUN/fixtures-a" \
|
||||
-cf "$RUN/boundaries-source.tar" sources/boundaries
|
||||
tar --sparse --format=gnu -C "$RUN/fixtures-a" \
|
||||
-cf "$RUN/workloads-source.tar" sources/workloads
|
||||
gzip -n "$RUN/boundaries-source.tar"
|
||||
gzip -n "$RUN/workloads-source.tar"
|
||||
sha256sum "$RUN/repo22-source.tar.gz" \
|
||||
"$RUN/freebsd-sys-source.tar.gz" \
|
||||
"$RUN/boundaries-source.tar.gz" \
|
||||
"$RUN/workloads-source.tar.gz" > "$RUN/source-archives.sha256"
|
||||
```
|
||||
|
||||
Transfer the archives, images, and fixture metadata to the dedicated guest.
|
||||
Authentication configuration stays outside the repository.
|
||||
|
||||
```sh
|
||||
ssh -p 9227 [email protected] 'mkdir -p /root/repo22-g7-transfer'
|
||||
scp -O -P 9227 "$RUN/repo22-source.tar.gz" \
|
||||
"$RUN/freebsd-sys-source.tar.gz" \
|
||||
"$RUN/boundaries-source.tar.gz" "$RUN/workloads-source.tar.gz" \
|
||||
"$RUN/fixtures-a/images/boundaries.erofs" \
|
||||
"$RUN/fixtures-a/images/workloads.erofs" \
|
||||
"$RUN/fixtures-a/SOURCE-INVENTORY.tsv" \
|
||||
"$RUN/fixtures-a/SOURCE-SHA256SUMS" \
|
||||
"$RUN/fixtures-a/SHA256SUMS" \
|
||||
"$RUN/fixtures-a/fixture-manifest.json" \
|
||||
"$RUN/fixtures-a/DEEP-PATH" "$RUN/fixtures-a/LONG-NAME" \
|
||||
"$RUN/fixtures-a/SPARSE-RANGES.tsv" \
|
||||
"$RUN/source.commit" "$RUN/repo22-src.tree" \
|
||||
"$RUN/source-archives.sha256" \
|
||||
[email protected]:/root/repo22-g7-transfer/
|
||||
```
|
||||
|
||||
Build and verify natively in the guest:
|
||||
|
||||
```sh
|
||||
mkdir -p /root/freebsd-src /root/repo22-g7-src \
|
||||
/root/repo22-g7/fixtures/images /root/repo22-g7/fixtures/sources \
|
||||
/root/repo22-g7/evidence
|
||||
tar -xzf /root/repo22-g7-transfer/freebsd-sys-source.tar.gz \
|
||||
-C /root/freebsd-src --strip-components 1
|
||||
tar -xzf /root/repo22-g7-transfer/repo22-source.tar.gz \
|
||||
-C /root/repo22-g7-src --strip-components 2
|
||||
tar -xzf /root/repo22-g7-transfer/boundaries-source.tar.gz \
|
||||
-C /root/repo22-g7/fixtures
|
||||
tar -xzf /root/repo22-g7-transfer/workloads-source.tar.gz \
|
||||
-C /root/repo22-g7/fixtures
|
||||
cp /root/repo22-g7-transfer/*.erofs /root/repo22-g7/fixtures/images/
|
||||
cp /root/repo22-g7-transfer/SOURCE-INVENTORY.tsv \
|
||||
/root/repo22-g7-transfer/SOURCE-SHA256SUMS \
|
||||
/root/repo22-g7-transfer/SHA256SUMS \
|
||||
/root/repo22-g7-transfer/fixture-manifest.json \
|
||||
/root/repo22-g7-transfer/DEEP-PATH \
|
||||
/root/repo22-g7-transfer/LONG-NAME \
|
||||
/root/repo22-g7-transfer/SPARSE-RANGES.tsv \
|
||||
/root/repo22-g7/fixtures/
|
||||
cd /root/repo22-g7/fixtures
|
||||
sha256sum -c SHA256SUMS
|
||||
sha256sum -c SOURCE-SHA256SUMS
|
||||
stat -f 'size=%z blocks=%b blocksize=%k name=%N' \
|
||||
sources/boundaries/maximum/sparse-boundary.bin
|
||||
cd /root/repo22-g7-src
|
||||
grep -E '^(REVISION|BRANCH)=' /root/freebsd-src/sys/conf/newvers.sh
|
||||
env WITH_ZSTDIO=1 FREEBSD_SRC=/root/freebsd-src ./build.sh
|
||||
cp build/erofs.ko /root/repo22-g7/erofs.ko
|
||||
cc -std=c11 -O2 -Wall -Wextra -Werror \
|
||||
-o /root/repo22-g7/g7_probe tests/g7_probe.c
|
||||
sha256 /root/repo22-g7/erofs.ko /root/repo22-g7/g7_probe
|
||||
file /root/repo22-g7/erofs.ko
|
||||
```
|
||||
|
||||
Record the build commit, source tree hash, `WITH_ZSTDIO=1`, archive hashes,
|
||||
FreeBSD source `REVISION`/`BRANCH`, KLD SHA256, probe SHA256, guest identity,
|
||||
kernel SHA256, and TCG/QEMU configuration in the report.
|
||||
|
||||
## Manual lifecycle
|
||||
|
||||
Load the exact module by full path and record its file ID and pathname:
|
||||
|
||||
```sh
|
||||
kldload /root/repo22-g7/erofs.ko
|
||||
kldstat -v | tee /root/repo22-g7/evidence/kldstat-loaded.txt
|
||||
KLD_ID=$(kldstat -v | awk '$NF == "(/root/repo22-g7/erofs.ko)" { print $1 }')
|
||||
test -n "$KLD_ID"
|
||||
printf '%s\n' "$KLD_ID" > /root/repo22-g7/evidence/kld.id
|
||||
dmesg > /root/repo22-g7/evidence/dmesg-before.txt
|
||||
```
|
||||
|
||||
Each TC attaches one fresh vnode-backed md provider, mounts read-only, records
|
||||
all correctness and metrics evidence, unmounts, detaches that exact provider,
|
||||
and verifies both are gone. Never use `killall`; concurrent tests persist
|
||||
every child PID, wait every child, and record every exit code.
|
||||
|
||||
TC128 and TC129 metrics are recording-only under QEMU TCG. Their PASS gate is
|
||||
read completion and exact source hash/byte comparison, not a fixed MB/s,
|
||||
IOPS, or latency threshold. TC127 uses FreeBSD RCTL `vmemoryuse:deny` with
|
||||
cooperating allocation probes, records allocation failure and reclamation,
|
||||
and verifies reads while pressure is held.
|
||||
|
||||
## Final cleanup
|
||||
|
||||
After TC130, require zero child processes, EROFS mounts, md providers, and
|
||||
RCTL rules created by G7. Unload the exact KLD file ID, compare dmesg, and
|
||||
power off the dedicated guest.
|
||||
|
||||
```sh
|
||||
mount -p | awk '$3 == "erofs" { print }'
|
||||
mdconfig -l
|
||||
rctl
|
||||
kldstat -v | grep -A2 -B2 '/root/repo22-g7/erofs.ko'
|
||||
KLD_ID=$(cat /root/repo22-g7/evidence/kld.id)
|
||||
kldunload -i "$KLD_ID"
|
||||
test -z "$(kldstat -v | grep '/root/repo22-g7/erofs.ko')"
|
||||
dmesg > /root/repo22-g7/evidence/dmesg-after.txt
|
||||
shutdown -p now
|
||||
```
|
||||
|
||||
Remove the host overlay, QEMU logs/PID file, source tars, temporary sparse
|
||||
probe data, and Python bytecode caches only after evidence has been copied
|
||||
into the committed report. Confirm port 9227 is released.
|
||||
Reference in New Issue
Block a user