Verb should be imperative mood matching rest of manual.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Ziaee <[email protected]>
Closes#19184
When zfs_root() or d_make_root() fails, zfs_domount() calls
zfs_umount(), which frees zfsvfs->z_vfs via zfsvfs_free(). After
the vfs_t lifetime was matched to fs_context, that vfs_t is still
owned by the caller in fc->fs_private. zpl_get_tree() then returns
without clearing fs_private, and put_fs_context() frees it again.
Detach z_vfs before zfs_umount() so the caller retains ownership,
matching the other zfs_domount() error paths.
This is a follow-up to the vfs_t lifetime change in #18377.
Reviewed-by: Rob Norris <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Wale Zhang <[email protected]>
Closes#19167
When a '.zfs/snapshot/<name>' entry is looked up by name (stat, open, or
a path walk through it; listing the directory does not do this),
zfsctl_inode_lookup() reads the snapshot's creation time for the
entry's btime through dsl_dataset_hold_obj(). That instantiates the
whole in-core dataset (dsl_dir hold, deadlists, neighbour references,
fsid uniqueness) and tears it down again on release, and it accounts
for most of a dentry-cold lookup. The value is a field of the
snapshot's dsl_dataset_phys_t, so read it from the bonus buffer the way
the fsid is read, through a helper both callers share.
Same value, same locking, same fallback on error. A warm lookup is
unchanged; a cold one loses the dataset instantiation, roughly an order
of magnitude on the systems it was measured on.
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: Rob Norris <[email protected]>
Signed-off-by: Ameer Hamza <[email protected]>
Closes#19174
statfs() on a control directory inode ('.zfs', '.zfs/snapshot', or an
unmounted snapshot entry reached without the automount) fails with EIO
because zfs_statvfs() verifies the znode's SA handle, which these inodes
lack. Handle them explicitly: '.zfs' and '.zfs/snapshot' report the
containing filesystem, and a '.zfs/snapshot/<name>' entry reports the
fsid of its snapshot, read from the dataset's bonus buffer (the objset id
is encoded in the entry's inode number) without instantiating the
dataset, so a probe costs a few microseconds.
This identifies a snapshot without mounting it: open the entry with
O_PATH, which does not trigger the automount, and fstatfs() it. The
value is what the snapshot's superblock reports once mounted, and path
based statfs() still automounts as before. An NFS server resolving a
file handle for an unmounted snapshot (expired, or imported on another
node) can probe entries instead of mounting every snapshot until one
matches. Add the statfs_nomount helper, the snapdir_statfs_fsid test
and a zfsconcepts(7) note.
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: Rob Norris <[email protected]>
Signed-off-by: Ameer Hamza <[email protected]>
Closes#19174
The replication check required a special or dedup vdev to tolerate
exactly as many device failures as the normal vdevs in the pool. A
3-way mirror special vdev on a raidz1 pool, or on a 2-way mirror pool,
was rejected as a mismatch, and the only way past it was -f, which
also overrides unrelated checks.
Accept a special or dedup vdev which tolerates at least as many
failures as the normal vdevs of a redundant pool. Special and dedup
vdevs are now compared with the normal vdevs rather than with whichever
vdev happens to precede them, so normal vdevs are still required to
match each other. Less redundant special or dedup vdevs, and redundant
ones added to a non-redundant pool, are still rejected.
As a side effect, a pool that already has a more redundant special
vdev is now considered consistent, so later additions to it are
checked instead of being skipped.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Andrey Mnatsakanov <[email protected]>
Assisted-by: Claude Code
Closes#12596Closes#19165
Commit 0d5bb1d010 added a check to silence warnings when building
on Debian systems where the /var/lib/rpm/rpmdb.sqlite RPM database
does not exist. However, RHEL8 still uses the older Berkeley DB
RPM database format, not SQLite, and the primary database file is
instead named /var/lib/rpm/Packages. Setting an the alternate,
empty RPM_DBPATH results in the package build failing due to missing
build dependencies. Resolve the build failure by additionally
checking for the legacy primary database name.
Reviewed-by: Tony Hutter <[email protected]>
Reviewed-by: Glenn Washburn <[email protected]>
Signed-off-by: Brian Behlendorf <[email protected]>
Closes#19181
In practice, we've seen limited benefit from always running the
centos-stream9 and centos-stream10 builders as part of the CI.
Remove them from the default set of builders to reduce the CI
load per PR. They may still be optionally requested.
Reviewed-by: Alexander Moch <[email protected]>
Reviewed-by: George Melikov <[email protected]>
Reviewed-by: Tony Hutter <[email protected]>
Signed-off-by: Brian Behlendorf <[email protected]>
Closes#19182
This allows easier package upgrade management for package git builds.
The previous behavior had the release number be the number of patches
on top of a ZFS release or on top of master. This is an issue if
patches are removed, thus making the release version not monotonically
increasing. This won't happen if only using official commits because
the history is treated as immutable. But for those wanting to carry
patches that have not been accepted, this can happen.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Glenn Washburn <[email protected]>
Closes#18953Closes#19159
When building on many debian machines, the RPM database, by default at
/var/lib/rpm/rpmdb.sqlite, does not exist and can not be created when
running as an unprivileged user. This will cause any rpm command that
ties to access the database to print this obnoxious error message:
error: Unable to open sqlite database /var/lib/rpm/rpmdb.sqlite: unable to open database file
error: cannot open Packages index using sqlite - Operation not permitted (1)
error: cannot open Packages database in /var/lib/rpm
It appears as though this doesn't cause any failures and is really a
warning. However, it can be printed more than 20 times when making the
deb-kmod target.
Detect when the database file is not readable at configure time and
setup various rpm and rpmbuild invocations to use the root of the build
directory as the database directory path. The alien program internally
runs various rpm queries, however it provides no good way to tell those
rpm invocations to use an alternate database path. A fake home is setup
in the build directory to create the user rpm macro file with the
database path defined there. Then run is alien with that home directory
so its rpm invocations will pick up the desired database path.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Glenn Washburn <[email protected]>
Closes#19162
An error message is already displayed by dpkg-buildpackage when build
dependencies are not met. However, the new error message shows the
exact command to use to satisfy those dependencies.
Rename the target native-deb-local to deb-local-common because its
rules are common to both deb and native-deb package builds. And make
nateive-deb-local depend on both deb-local-common and the new
native-deb-check-build-dep.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Glenn Washburn <[email protected]>
Closes#19161
On some systems, notaby Debian and many of its derivatives, /bin/sh is
a symlink to /usr/bin/dash. The scripts created for the various
rpmbuild stages are run with /bin/sh. The construct {a,b,c} does not
get expanded in dash, as it does in bash's sh.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Glenn Washburn <[email protected]>
Closes#19158
When FreeBSD switched their ZFS to OpenZFS in 13, a compat layer was
included to allow older userspace to work with newer kernel modules and
vice-versa. Since FreeBSD 12 is now almost 3 years out of support, we
shouldn't need this compatibility support any longer, and can remove it.
The small versioned wrapping struct zfs_iocparm_t is still in use for
current ioctls (and honestly a pretty good idea), so that's retained,
and we're now explicitly enforcing the version and size fields are what
they're supposed to be.
Sponsored-by: TrueNAS
Reviewed-by: Kyle Evans <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Rob Norris <[email protected]>
Closes#19157
Currently, when probe fails on some vdev, it would became
faulted, but the parent's vdev state won't be changed. So we
can have a situation, for example, when several disks in draid
vdev are faulted (domain failure), but draid vdev itself and
pool are online, which doesn't seem right.
Solution: set isopen argument to false when calling
vdev_set_state() from spa_async_fault_vdev(). It's not quite
clear why it was set to true in the first place when the code
was introduced at commit c4f2f1aa2d.
isopen argument to vdev_set_state() is normally set to true
when we are "in the process of opening children depth-first",
in which case we don't want to propagate the change of state
to the parent, but it doesn't seem to be the case here.
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Andriy Tkachuk <[email protected]>
Closes#19155
The mountpoint property does not apply to snapshots, so change_one()
drops every snapshot it is given. Iterating over them while gathering
a mountpoint changelist therefore costs a full stats ioctl per snapshot
and never contributes an entry, which makes "zfs set mountpoint", "zfs
inherit mountpoint" and "zfs set volsize" scale with the number of
snapshots for no reason.
Iterate filesystems only. This also covers canmount, volsize and zoned,
since those are handled through the mountpoint changelist.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Ameer Hamza <[email protected]>
Closes#19137
Reduce replace_resilver_sit_out test time from 1:14 to 30sec.
- Use 4 "lanes" in zinject instead of 1.
- Use a 1M recordsize to speed up replacements.
- Use 1 second zfs_txg_timeout to help with replacement
Also, double the number or reads to more reliably hit the sit-out
condition. This test has been flaky in the past.
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: Alexander Moch <[email protected]>
Reviewed-by: George Melikov <[email protected]>
Signed-off-by: Tony Hutter <[email protected]>
Closes#19156
This PR makes several changes to the user-facing command line syntax and
documentation:
- The `decompress,` `drop_records,` and `recompress` subcommands now
allow naming a stream file on the command line, bringing them into
line with `dump` and `raw.` Reading from standard input remains an
option.
- Tests have been modified to exercise command-line specification of
input files.
- The man page has been updated and copy edited.
- Subcommands are presented in a consistent (alphabetical) order in the
man page, the usage message, and the source code.
- The man page and usage message now use identical names and
capitalization for options.
- The `decompress` and `recompress` subcommands now use
`zfs_prop_string_to_index()` from `libzfs` to parse compression
specifiers. Wherever a compression type is required, users can
specify any value accepted by the `compression` property except
`on`, e.g., `zstd-10`. In previous versions, these subcommands
had their own parsers.
- The `-l` option to `recompress` (for specifying a zstd compression
level) is now described as deprecated in the man page. It no longer
appears in the usage message. For the most part, `-l` still works
as before. However, it now prints a deprecation warning and tells
you what to use instead. It's also more strict about insisting on
being used only in combination with 'zstd'. Previously, the `-l`
option was accepted but silently ignored when used with non-zstd
compression methods. Various permutations of -l and compression
specifiers are tested by ZTS.
- Since it can accept more than one record specifier, the `drop_record`
subcommand has been renamed `drop_records`. `drop_record` remains as
an (undocumented) alias for `drop_records`.
- `decompress` and `drop_records` now share a parser for record
specifiers.
- Since `decompress` and `drop_records` now accept a filename as their
last command line argument, there is a potential ambiguity as to
whether the last argument is a filename or a record specifier.
This will likely be invisible to users because `object,offset`
specifiers are distinctively spelled. Broadly speaking, if an
argument is parsable as a record specifier, it's assumed to be
a record specifier. Error messages may be blurry for the last
command line argument, e.g., "invalid record specifier or
input file '123,'". It is always possibile to take input from
stdin if the command line parser won't read your stream file
named `123,456`.
- Hash table management for `decompress` and `drop_records` is now
shared and moved into `zstream_util.c`. As of now this still
uses hcreate() et al, but I have a hash table update coming
that should obviate the need to convert to string keys.
The full command line grammar is now:
```
zstream decompress [-v] [object,offset[,compress_type]...] [file]
zstream drop_records [-v] [object,offset...] [file]
zstream dump [-Cvd] [file]
zstream raw [-v] [-b max_buffers] [-g fromguid] image|device [file]
zstream recompress [-t num_threads] compress_type [file]
zstream redup [-v] file
zstream token resume_token
```
`decompress` and `drop_records` are the only two subcommands that
operate on individual records. Arguably, they should be brought into
closer alignment either by renaming `decompress` to
`decompress_records` or renaming `drop_records` to `drop`. I rather
like the `_records` suffix to mark record-specific operations, but
`decompress_records` is long, and both renaming options seem more
disruptive than they're worth.
## Does this break things that used to work?
Only `recompress -l` when used with non-zstd compression methods.
That shouldn't have been accepted in the past since it probably
doesn't do what users expect. It now produces an error message.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Garth Snyder <[email protected]>
Closes#19109
When the domain fails and there is no traffic, zed might get
list.suspect event with the disks entering into UNAVAIL state,
instead of zfs.statechange events with the disks entering into
FAULTED state. We should handle that situation similarly and
and don't start resilvering, which would be only a huge waste
of CPU and i/o bandwidth resources.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Andriy Tkachuk <[email protected]>
Related-to #18148Closes#19154
When a user enables configure caching when running configure, eg. by
passing -C, enable configure caching in the corresponding rpm builds.
The configure cache from the initial configure run by the user can not
generally be used because the host and build values will be different.
Normally those values are left unset when invoking configure whereas
the configure from rpmbuild explicitly sets them.
However, the intent of the user to enable configure caching can still
be satisfied by using a cache file solely for rpm builds. This implies
that initially, when building an rpm package with configure caching
enabled, that the first two runs of configure will be uncached, the
user-initiated and rpmbuild initiated ones. The cache file is named
config.rpm.cache and is located in the root of the build directory.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Glenn Washburn <[email protected]>
Closes#19111Closes#19148
Readers took scl_lock on every spa_config_enter()/exit(), which cost
~29% of CPU time in 4KB record write workloads. Fold the writer gate
into the reference count: SCL_COUNT_WRITER in the top bit of scl_count
lets a reader take a reference and test for a writer in one atomic
operation, and lets the last reader detect a waiting writer without a
second load. Writers still serialize on scl_lock, and priority readers
still take it, since they must tell a waiting writer from a holding one.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Motin <[email protected]>
Closes#19131
dsl_pool_dirty_space() and dsl_pool_undirty_space() are called once per
logical block, from open context and from every ZIO completion, all
serializing on the single pool-wide dp_lock. On a 12-thread 32KB
overwrite benchmark they took 33% of CPU, almost all of it spinning.
The lock only protected the counters against each other, so make
dp_dirty_pertxg/dp_dirty_total and dp_sync_reserve_pertxg/_total plain
atomics. The subtractions clamp at zero via a CAS loop. Reading these
counters without the lock was already the norm.
The cv_signal() on every accounting event is only ever needed by
dmu_tx_wait(), which sleeps solely at the hard zfs_dirty_data_max wall,
so gate it on a count of waiters. Both sides need membar_sync(), since
the atomics used here provide no ordering. dmu_tx_wait() in turn only
registers as a waiter if it may really sleep, keeping dp_lock off the
soft throttle path taken by every delayed transaction.
This gives 20.6 -> 26.0 GiB/s on the above benchmark, with the code
almost disappearing from CPU profiles.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Motin <[email protected]>
Closes#19126
zio_batch_arrive() publishes onto zb_arrived and then drops its hold;
whoever drops the last one walks that list with a plain load. Neither
side carries a barrier, so on arm64, where FreeBSD's atomics are
relaxed, a thread can see zb_holds reach zero and still read a stale
zb_arrived. The missed member never leaves ZIO_VDEV_IO_DONE, its parent
never completes, and the txg sync thread waits forever in zio_wait().
Linux and FreeBSD/x86 are unaffected; their atomics are ordered. A
Cortex-A76 litmus test observes the reordering in roughly 4% of the
cases where the reader wins the batch.
Reported by: Dave Cottlehuber <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: Alexander Motin <[email protected]>
Signed-off-by: Nick Price <[email protected]>
Closes#19152
By adding the "-dirty" tag to the release and by extension the full
version string, it becomes more clear that the build contains changes
not checked in to the repo. This could indicated changes that are
included by mistake.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Glenn Washburn <[email protected]>
Closes#19151
An entry is only listed when the recorded block still carries the birth
transaction recorded against it, so a log whose blocks have since been
freed or rewritten prints a header promising files and then nothing at
all. The count shown without -v comes from spa_approx_errlog_size(),
which tallies table entries without resolving them, so the two forms
disagree and neither explains why.
Say that the entries resolved to no file. Reading the log is also what
removes such entries, so no advice to scrub is given: by the next read
they are gone.
Reviewed-by: Alek Pinchuk <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Nick Price <[email protected]>
Closes#19108
dsl_scan_sync() sets spa_scrub_active before it checks whether a scan is
running, behind only a test that dsl_scan_active() is true. That test is
also satisfied by an async destroy, a non-empty free bpobj, or a pending
livelist delete, and nothing clears the flag except dsl_scan_done() or
dsl_errorscrub_done(), neither of which runs when no scan was started.
So after any dataset destroy the flag stays set for the life of the spa,
and spa_log_error() files every subsequent error in the scrub error log
rather than the last error log. An error scrub reads only the last error
log and so never sees them, and zpool clear does not touch the error log
at all. When a later scan ends it rotates the scrub error log into the
last error log and deletes only the log it replaces, laundering those
entries in as though that scan had found them, after which they survive
every subsequent scrub.
This is consistent with a long-standing class of reports, among them
survive repeated scrubs, and do not respond to zpool clear. The
workaround found in #16158, starting a scrub and cancelling it before
scrubbing again, fits the same mechanism: dsl_scan_done() rotates on
cancel as well, moving the stale entries somewhere a later scrub will
delete.
Set the flag only when a scan is actually running. Do not clear it here.
dsl_errorscrub_sync() sets the same flag, and dsl_scan_sync() still runs
for async destroys while an error scrub is in progress.
Also heal pools already damaged, at import. A non-empty scrub error log
with an empty last error log and no scan to own it can only be a stale
remnant, so request the rotation that moves it into the last error log,
where an error scrub can reach it and the status read can drop those
that resolve to no file. A pool with both logs populated is left alone;
the next full scrub folds the rest in.
Reviewed-by: Alek Pinchuk <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Nick Price <[email protected]>
Closes#19108
objtool sizes its alternate signal stack with SIGSTKSZ, which musl fixes
at 8192. A runner whose CPU exposes AMX needs 11952, so sigaltstack() is
refused, objtool exits before doing any work, make deletes the object,
and configure reports
checking whether CONFIG_MODULES is defined... no
on a kernel built with CONFIG_MODULES=y. Two separate samples of forty
Alpine jobs each landed four of them on such a host; all four failed,
every other job passed.
amx-tile is the only AMX feature carrying xsave state, so masking it
alone brings the frame back under SIGSTKSZ. host-model would also avoid
this but takes AVX-512 with it, which the suite exercises. Masking on
every runner keeps the diff to one line and is a no-op on the nine in
ten without AMX; ZFS has no AMX code. This can go once a fixed objtool
reaches the distribution images.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Moch <[email protected]>
Closes#19150
This fixes an issue where the system rpmbuild configuration is setup
to generate debuginfo rpms and will fail in make rpm-kmod if the needed
debuginfo source files are empty. Since by default debuginfo generation
is not enabled, this will fail reliably. Tell rpmbuild to not try to
build debuginfo rpms, when debuginfo generation is not enabled by
configure, by passing it the --nodebuginfo option.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Glenn Washburn <[email protected]>
Closes#19146
AM_V_GEN must be placed at the start of the recipe line because it gets
expanded to "@echo" which the shell doesn't know what to do with. The
@-prefix is interpreted by make, but only at the beginning of a recipe
line.
Fixes: 7adc908d3 (build: Add gitrev as a dependency to specific targets that require it)
Reviewed-by: Rob Norris <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Glenn Washburn <[email protected]>
Closes#19145
Ten of this test's thirty bad_combine entries ("all -r", "all -p", "all
-H", "all -rp", "all -rH", "all -rpH", and four "$TESTPOOL" variants)
place a valid option after the property list. GNU getopt_long(3)
reorders those into a valid invocation, so zfs get accepts them and the
negative test does not fail.
They date from the Solaris original, where getopt(3) never permutes and
they failed naturally. Rather than reassess them when the test was
enabled on Linux, 95401cb6f exported POSIXLY_CORRECT=1 to push glibc
back into non-permuting mode. It borrowed that from
zfs_unallow_008_neg, where f74b821a6 had added it with a comment saying
it is set "so the original test cases will fail".
That only ever worked against glibc. musl's getopt_long(3) selects
permutation solely from optstring[0] and never consults POSIXLY_CORRECT,
so the ten cases fail on musl systems such as Alpine Linux.
The cases do not exercise zfs get regardless: the optstring and
zfs_do_get() behave identically either way, and only the C library's
argument handling changes. Drop them and the export rather than teach
the test a second libc's quirks. The remaining twenty entries fail on
their own merits on any libc -- invalid options, invalid property lists,
getsubopt residue and nonexistent datasets.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Moch <[email protected]>
Closes#19138
The -m option checks for /sys/kernel/debug/kmemleak with
os.path.exists() before accepting the flag. That check fails on a
kernel where the detector is compiled in, booted with kmemleak=on and
running: debugfs is mounted 0700, so an unprivileged caller cannot stat
anything below it, and the suite is normally started as an ordinary
user.
Every later access to that file goes through sudo, so check for it the
same way, with -n to match the passwordless sudo the runner already
requires of the calling user.
Report what sudo said when the check does fail. The old message blamed
the kernel configuration, which is the one thing that was not wrong, and
gave no way to tell the cases apart.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Moch <[email protected]>
Closes#19136
dsl_dir_tempreserve_impl() and dsl_dir_willuse_space() took dd_lock of
every dsl_dir up to the pool root for each write, costing ~10% of all
CPU on a 12-thread 32KB sequential overwrite benchmark.
dd_tempreserved[] and dd_space_towrite[] are per-txg estimates of what
the open context is about to write. They never reach the on-disk
accounting, which is still updated under dd_lock from the sync context,
and all their consumers only need an estimate, as dmu_tx_try_assign()
already notes. So update them with atomics and read them unlocked.
dsl_dir_sync() no longer needs dd_lock either: it only clears the slots
of a txg that is already done, which the open context can not use any
more.
Also add a few fast paths: parent_delta() and dsl_dir_willuse_space()
have nothing to compute when there is no reservation,
dsl_dir_tempreserve_impl() needs no estimates when there is no quota
and no reservation to account for, and dsl_dataset_check_quota() needs
no ds_lock when the dataset has neither quota nor reservation. Not
charging dd_tempreserved in that case needs no matching decrement,
since dsl_dir_tempreserve_clear() only walks the dirs put on tr_list.
Together with the previous commit this takes the benchmark from
12.9 GiB/s to 20.6 GiB/s.
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: George Melikov <[email protected]>
Reviewed-by: Ameer Hamza <[email protected]>
Signed-off-by: Alexander Motin <[email protected]>
Closes#19122
dsl_dataset_block_born() and dsl_dataset_block_kill() applied the space
accounting of every single block under ds_lock, and then rolled it up
under dd_lock of every dsl_dir up to the pool root. On a 12-thread
32KB sequential overwrite benchmark that accounted for ~62% of all CPU
time.
Instead accumulate the deltas in per-CPU wmsums and apply them once per
txg from dsl_dataset_sync_done(). The result is identical rather than
approximate: parent_delta() is a difference of two MAX()es around the
same counter, so it telescopes over the txg, and the rest of the
dsl_dir accounting is linear in its arguments.
Blocks moved to the deadlist are counted in a separate group from the
blocks unique to the dataset, so the hot paths need only three
wmsum_add() calls each, and ds_referenced/compressed/uncompressed_bytes
derive as the difference of the two groups when the deltas are applied.
old_synchronous_dataset_destroy() is not synced by dsl_pool_sync(), so
it applies the deltas itself, right where it already asserted that the
accounting has settled.
The ds_feature_activation[] stores are now conditional, which lets them
leave ds_lock the same way dbuf.c, dnode_sync.c and zil.c already do.
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: George Melikov <[email protected]>
Reviewed-by: Ameer Hamza <[email protected]>
Signed-off-by: Alexander Motin <[email protected]>
Closes#19122
Receive a snapshot into $POOL2/. and $POOL2/.. and check that both
are rejected and that a name starting with a dot is still accepted.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Christos Longros <[email protected]>
Closes#18975Closes#19010
ZTS calls cmp with skip offsets, and zfs_send_sparse does it against a
file whose hole doubles every round. GNU cmp seeks to the offset;
busybox cmp reads its way there, 512 bytes at a time. Timed per command
on the runner, the send stays flat while the compare doubles:
hole zfs send | zfs receive cmp
256MiB 0.07s 25.2s
512MiB 0.08s 51.7s
1GiB 0.07s 102.1s
so the test walks into its ten minute cap, and the group's cleanup fails
behind it and takes the next groups' setup with it.
The GNU cmp comes from diffutils, which is not in this list -- without
it the busybox applet answers to cmp instead. diffutils arrives on 3.24
only because something else happens to pull it in, so the runner works
by accident of the dependency graph rather than by asking for what it
needs.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Moch <[email protected]>
Closes#19139
A receive fails with ZFS_ERR_STREAM_LARGE_BLOCK_MISMATCH when the
destination has large blocks and the stream does not carry them. The
error told the user to add -L, but -L cannot help when the sending
dataset has no large blocks of its own: setup_featureflags() sets
DMU_BACKUP_FEATURE_LARGE_BLOCKS only when the feature is active on the
sending dataset, so -L is silently a no-op there. That is the state a
dataset is left in by an earlier replication in the opposite direction
that ran without -L, which splits the large blocks and leaves the copy
without the feature. The two copies then have different block layouts,
and advising -L sends the user in a circle.
Reword the error to name both causes and the remedy, and document the
trap in zfs-send.8 and zfs-receive.8.
Add a test for the reverse direction, which was not covered: replicate
a dataset that has large blocks without -L, then send an incremental
back with -L and confirm it is rejected, that omitting -L fails the
same way, and that a full receive into a new dataset succeeds.
While here, correct the claim in zfs-send.8 that -L has no effect
unless recordsize has been set above 128 KiB. The per-dataset
large_blocks feature is activated in dsl_dataset_block_born() when a
block larger than 128 KiB is written, so a dataset with recordsize=1M
that has never stored a large block still sends without large blocks.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Martin Minkus <[email protected]>
Issue #18908Closes#19128
mount_loopback attaches an image with "losetup -f" and then looks the
device up again with "losetup --associated". Alpine's udev populates a
/dev/loop directory with the devices the kernel pre-creates at init, and
util-linux resolves every device name to /dev/loop/N whenever that
directory exists. A device allocated beyond those, through
/dev/loop-control once the first eight are busy, has no node there and
is never found. mkfs.xfs then runs with an empty argument and the test
aborts.
That is also why it is intermittent: it depends on how many loop devices
earlier tests left attached.
Aborting leaves the loop device holding the image open, so the pool
underneath cannot be unmounted and every later test that destroys a pool
fails too.
Ask losetup to print the device it attached instead, and fail loudly if
it prints nothing.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Moch <[email protected]>
Closes#19135
The Alpine runner takes over five minutes to answer after the build step
reboots it, and all of it is one service:
* cloud-init local ...Cloud-init v. 26.1 running 'init-local'
DataSourceLXD.py[WARNING]: /dev/lxd/sock does not exist.
<301 seconds>
[ ok ]
Everything after it takes two seconds. It is datasource detection
timing out on an LXD socket a libvirt guest will never have.
Three cloud-init services are already removed here, but the one that
stalls is cloud-init-local, which lives in the boot runlevel rather than
default and so was never touched. Remove it too. Cloud-init is still
needed on the first boot, where the ssh key arrives through it; this
runs after that.
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Alexander Moch <[email protected]>
Closes#19132
Change zpool_initialize_multiple_pools test vdev sizes from
5GB -> 1GB. This size reduction is important since the test creates
four vdevs of this size (20GB) and initializes them. This
space reduction is need for running 3 CI VMs in parallel.
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: George Melikov <[email protected]>
Signed-off-by: Tony Hutter <[email protected]>
Closes#19134
zpool_prefetch_001 creates 16 thousand files, and makes copies of a
fraction of them. This takes a long time, especially when run on the
slow CI VMs.
Make it faster by only creating 8k files, and do some of the creations
in parallel. Both optimizations help to bring down the test time
from 40sec -> 14sec, as measured on a local, 10-core, VM.
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: George Melikov <[email protected]>
Signed-off-by: Tony Hutter <[email protected]>
Closes#19133
Add akmod (Automatic Kernel Module) support to ZFS, providing a third
packaging option alongside the existing DKMS and kmod packages.
Akmod packages automatically build binary kmod RPMs when new kernels
are installed, enabling:
- Automatic kernel module rebuilds on kernel updates
- Distribution of binary kmod RPMs without recompilation
- Seamless minor version upgrades on RHEL/Fedora and clones
Key changes:
- rpm/generic/zfs-kmod.spec.in: Add conditional akmod support via
rpmbuild --define '_enable_akmod 1'. When enabled, sets
buildforkernels=akmod and repo=rpmfusion so kmodtool generates
an akmod-zfs package. Add -common subpackage for shared
license/doc files. Guard chmod and %files sections for empty
kernel_versions. Make kmod-devel packages opt-in via
%bcond_with devel to prevent akmods from installing unnecessary
devel packages on every kernel update. When devel is not
enabled, devel header files are cleaned from the buildroot to
avoid unpackaged-files errors.
- config/rpm.am: Add 'make rpm-akmod' and 'make srpm-akmod'
targets. These targets use the generic spec with _enable_akmod
and intentionally omit kernel-specific defines (kernels, ksrc,
kobj) so kmodtool enters its akmod-only code path. The existing
rpm-kmod/srpm-kmod targets pass --with devel to preserve the
current behavior of producing devel packages.
- config/zfs-build.m4: Update comments to document the akmod vs
regular kmod distinction for RPM_DEFINE_KMOD.
- scripts/kmodtool: Fix a pre-existing bug where
print_rpmtemplate_kmoddevelpkg (the common kmod-NAME-devel
package) was called inside the per-kernel loop in
print_customrpmtemplate and print_rpmtemplate, causing duplicate
%package definitions when multiple kernel-devel packages are
installed. Move it outside the loop so it is emitted once.
The akmod infrastructure requires the 'akmods' package on the
target system. When the akmod-zfs RPM is installed, it places a
zfs-kmod src.rpm in /usr/src/akmods/. The akmods service then
automatically rebuilds it for the running kernel via rpmbuild
--define 'kernels <uname-r>' --rebuild.
Usage:
make rpm-akmod # Build akmod-zfs, kmod-zfs, zfs-kmod-common
make rpm-kmod # Build kmod with devel packages (unchanged)
Tested on Fedora 43 (kernel 7.1), RHEL 9.8 (kernel 5.14), and
Alma Linux 10 (kernel 6.12). Verified akmods rebuilds produce
only kmod-zfs (no devel packages) after kernel updates.
Reviewed-by: Tony Hutter <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Vincent S. Cojot <[email protected]>
Closes#19099
Alexander Ziaee