The initial port of the CTF tools had a FreeBSD-specific patch to print
the termination message but exit with a 0 status, with a goal of getting
as much to build as possible and silently ignoring any issues.
We're now past the point where silently ignoring failures makes sense.
Any future issues need to be found and addressed.
PR: 276826
PR: 276930 [exp-run]
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43743
Notable upstream pull request merges:
#17864e903655c5 zpool: Add zpool status -vv error ranges
#18820 -multiple zdb: account pending DDT-log frees in leak detection
#18884 -multiple zio_crypt: establish platform interface; rework common
code to use it
#19010 -multiple zfs_namecheck: reject '.' and '..' before a snapshot or
bookmark
#19031994fb1703 Fix metaslab count assertion in metaslab_group_alloc()
for small vdevs
#19093f5b2fc8e2 zfs_ctldir: make .zfs/snapshot/<name> btime the snapshot
creation time
#190972dece2a34 zstream: report invalid record context without assertions
#1910278f49e1dd vdev_disk: simplify alignment checks for linear ABDs
#19108 -multiple Fix permanent errors misfiled into the scrub error log
#19110fa4bc4dec spa_errlog: don't let one unresolvable entry hide the
whole error log
#19116b6dde8a17 zio_crypt: free the key unwrap uios when decryption fails
#19122 -multiple Fix dd_lock contention on space accounting
#19126c792317f2 Avoid dp_lock in the dirty space accounting
#191317fd87803e Make the spa_config_lock reader fast path lockless
#19137a83d96835 libzfs: do not iterate snapshots for a mountpoint
changelist
#19152ccce78729 ZIO: Order a batch arrival against the hold it releases
#191559d7260c9b Propagate vdev state change on probe failure
#19157946be3a33 FreeBSD: remove legacy ioctl support
#191651d9c32c16 zpool: accept more redundant special and dedup vdevs
#19167638abd22a zfs_domount: fix vfs_t double-free on root setup failure
Obtained from: OpenZFS
OpenZFS commit: 1f380a4f34
Notable upstream pull request merges:
#188086d4ff3b4f Pool split leaks DTL spacemap objects
#188440833cfd64 Decline Direct I/O reads on a file handle after a benign
verify failure
#1885339764eb06 Fix double free when cloning a block with a pending free
#1890151fe1fb84 dmu_send: a meta-dnode range does not start where an
object does
#18903904d4327f dmu_objset_open_impl: unregister prop callbacks on error
#18906e45640415 zdb: add detailed diagnostics for MOS leaks and spacemap
refs
#189091b7143557 Keep a grown vdev from adopting an older pool's labels
#18912 -multiple Defer destruction of a snapshot that a mount is holding
open
#1892133ae06bb6 ZIO: Batch vdev children completions
#189407853c27eb ZIO: Batch lightweight ZIOs
#18967 -multiple Speed up pool rewind and allow it to tolerate minor damage
#18968b0fff48dc arc: harness buf_hdr's anon state invariant checks
#18969b0da5c4ce dsl_crypt: validate every key change-key will rewrap
#18974b19fa7c97 log_spacemap: make zfs_log_sm_blksz tunable
#18982c878ee452 zstream: track and limit memory use
#18991a1261384a zstream: self-tuning queues
#18992fd639b075 zcp: support bookmarks in zfs.sync.destroy
#189930498c554d vdev_disk_open: fix ENOENT retry & cleanup error handling
#190035280a08f6 Write uberblocks only to vdevs used by the txg
#19008c47591d5d arc: apply arc_min tunables set before arc_init() has run
#19016cd578e518 dnode: fix heap-use-after-free in dnode_rele_and_unlock()
#19017b824f3116 zstd: declare __asan_*_memory_region() for user space
builds
#190296860d1c2f DDT: Use proper size when calling kmem_free on
ddt_prune_entry_t
#19041aa26ca67b Do not complete a device removal that hit IO errors
#190428e5bb1d4b Wait for the initialize and trim threads to exit
#19051ba524e0d7 zpool: things get messy with 0 columns
#1905369ef525a4 dsl_deadlist_space_range: use correct tree when summing
up space
#190590ee8ea01b FreeBSD: use sys/abi_compat.h for time32_t
#19062e08bc2973 Skip interior dnode slots when receiving DRR_FREEOBJECTS
#1906692a3904af Wait for the txg that finished a scan to sync
#19071db07a5e0a dbuf: account dirty data for embedded writes
#190721a8e51491 Fix self-deadlock when cloning a range within the same
file
#1907575bd31c28 L2ARC: do not feed a device while its rebuild is pending
#1908356e8d234b Stop the TRIM and initialize threads when a vdev goes
offline
#19090ab994acdd dmu: remove dmu_prealloc()
#1909261fbdf1a2 zfs: spa_sync_upgrades() should only take lock when needed
Obtained from: OpenZFS
OpenZFS commit: 02b5baf13c
pkgbase splits programs and their tests into separate packages (e.g.
unifdef/yacc/indent/file2c -> toolchain, jail -> jail, etc), while their
tests always land in the generic tests package. Installing just the
tests package therefore leaves these suites unable to find the binary
they exercise, and they fail confusingly instead of skipping.
Rather than force all packages to be installed for the tests package,
make the tests cope with the missing packages. Add require.progs (or,
for TAP/PLAIN-style suites with no atf_set hook: TEST_METADATA
required_programs) for the binary under test in: bectl, certctl,
ctfconvert, dhclient, file2c, indent, ipfw, jail, lastcomm, mixer,
newsyslog, nvmecontrol, pfctl, praudit, sa, syslogd, tunefs, unifdef,
yacc, ztest.
I skipped adding data for all the binaries in the base runtime
package for simplicity.
Assisted-by: Claude Code (Sonnet 5)
Sponsored by: Netflix
Reviewed by: ngie
Differential Revision: https://reviews.freebsd.org/D58262
Previously, we would precompile D test dependencies using the
host's dtrace, which unconditionally outputs ELF files in the
host's format. This breaks the cross-compile build with errors
like the following:
dtrace: failed to link script: incorrect ELF machine type for
object file: tst.usdt.pieo
--- usdt.o ---
*** Failed target: usdt.o
This patch moves compilation to runtime for all C-based testcases
that have a dependent D source file.
Reviewed by: markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D59030
Notable upstream pull request merges:
#16761eb1738bcb FreeBSD: Enable Direct IO by default
#16747 -multiple On-demand log-spacemap flush
#184740d0eae2ab implement thorough scrub support (zpool scrub -t)
#185626721ab981 Calling thread IO
#186577ed1268d3 Fix race between device removal completion and pool export
#187070d1f3b1c6 RAIDZ: Fix parity regeneration/check condition
#18713 -multiple libzfs: fix key unload failure when unmounting an
encryption root
#18714d902eec64 Disconnect metaslab tracing from default builds
#18716e78fa488a zstream: multithreading
#187186acb99cb1 Do not return ESTALE for open-unlinked files
#18720f607ef7e7 Fix insufficient locking in dedup verify
#18722 -multiple snapdir: misc cleanups
#187249bf75b4b1 Fix reads for blocks freed after being cloned
#187259b7642df9 Harden recv record validation
#1873237f066e3c FreeBSD: Wire sha512 offload to the build
#18736 -multiple Remove idmap/userns concept from core & FreeBSD platform
code
#18742d63f14057 libzutil: keep valid spare and l2cache paths on import
#18749f217627d4 Fix receive of split large blocks with a short trailing
chunk
#1875483bf40784 Use a single creation time for a recursive snapshot
#18755ebd9e2765 zstream: add "drop records" chain module
#187637d565cdfa zfs bookmark: add recursive (-r) bookmark creation
#1876550557cc80 zpool initialize: add -z to write zeroes
#187660467ba01c Add SECURITY.md policy filE
#18773ffde77051 Fix deduce_nblkptr for large dnodes on receive
#18784 -multiple libzfs: clarify the raw incremental "IV set guid mismatch"
error
#187939a02e53ad libzfs: don't abort receiving a raw encrypted send that
carries holds
#1879530426217b Rate limit Direct I/O verify zevents
#1879873b202451 Bound explicit user prefetch to a fraction of the ARC
#188012a5331f8e Fix dmu_zfetch_prime() assuming a stream was created
#18802fb5fdefd8 libzfs: fallback VDEV_UPATH to VDEV_PATH for non-DM
devices
#18809bd2d87399 zdb: output refcounts from verify_spacemap_refcounts()
#188115d530b8be ABD: Validate borrowed buffer length
#188125d687a975 dbuf: use dirty record size for overridden writes
#18819 -multiple ddt: fix refcount bypass and gang member leak for dedup
gang blocks
#1882187e2e4047 BRT: Implement partial bv_entcount writing
#18822 -multiple dnode_sync: Relax constraint on indirect freeing - #18822#18827d98fa72ca L2ARC: bound the rebuild by the write hand on a first
sweep
#1883190cdd9aef Batch object reallocation syncs in zfs receive
#18833466d90e15 zed: let autoexpand see capacity changes on partitioned
disks
#188351ac3f2786 mmp: skip non-writeable vdevs during activity check
#18838a04c40138 DDT: Fix several bugs in pruning
#188410bb41751f zpool export: return EBUSY when zvol minors are in use
#188483020c18ca dmu_recv: Avoid potential null deref
#188558cdd9b2b7 mmp: do not require writes to mirror legs the config
marks absent
#18858596c7a0ec Reduce dp_lock scope for MOS writes
#188593ebf3ffc4 Predict and throttle buffers dirtied by the sync context
#18860 -multiple Further parallelize block cloning
#18865412b17a29 Fix DMU bonus hold leak on I/O error
#18867b5bb1f816 Add missing checks to zfs_clone_range_replay()
#188683df5b2e7f libzfs: String trimming should not operate out of bounds
#18869be62c5385 libzfs: Do not call munmap() when mmap() fails
#1887174e76f2d8 libzfs: don't truncate a resolved vdev path in
zpool_vdev_name()
#188747312322b5 nvpair: Fix operator precedence
#188833bd8cefdc libzfs: don't read a dataset handle after closing it in
resume send
#18886b4f70cb9e Fix negative time overflows in DDT pruning
#18888674b1ae5c DDT: Make ddt_zap_walk() use cursor _by_dnode functions
#18889fd1ae59c7 DDT: Skip DDT log lookups in ddt_prune_walk()
#18892 -multiple zhack: add "mmp reclaim" to recover a pool stranded by MMP
#18897216de07d8 arc: fix race between arc_release() and arc_read_done()
#1889928afe8b08 CodeQL: Flag implicit compare-then-assign in branch
conditions
#189170a79c039f arc: save on buf_hash_find() call at arc_read_done()
#189207026d3335 Fix race condition in raidz expansion startup
#18927be55e01bc FreeBSD: Do not leak TSD zfs_geom_probe_vdev_key on errors
#189285bd7d4466 scan: count skipped blocks as examined
#18937a5d678864 Allow pool import with corrupted spare/l2arc configs
#18960 -multiple vdev_open: use calling credential to check for device
access
#18961acbcdd92c dmu_recv_begin_check: dsl_dataset_rele() should be called
on ds
#18962df3ff37fc dmu_redact_snap: Do proper cleanup on ENAMETOOLONG
#1896484aa7e7e0 libspl: consult ZFS_HOSTID on FreeBSD as well
Obtained from: OpenZFS
OpenZFS commit: 84aa7e7e09
Grow the module collection to answer, each with a single command, the
first questions asked when diagnosing a sick system: why is my
application stalling, where is the kernel fighting over locks, what
file could it not find, why is this process getting EPERM, what killed
my process, will that fatal signal actually leave a core behind, what
was my process stuck on, where is my kernel memory going, who is
creating or entering jails, is the network slow because TCP is
resending, how long did my thread wait to run, and is the disk itself
slow. Every module keeps to the house style: invocation-name
overloading through hard links, predicate-only D with inline lookup
tables (no if-statements), and stable providers only (syscall, proc,
sched, io, dtmalloc, and the lockstat, vfs, priv, and mib SDT
providers), so the modules remain drop-in compatible with older
releases (the one documented exception is noted below). No kernel
changes: new and extended profiles under cddl/usr.sbin/dwatch/libexec
plus one libdtrace inline table (priv.d).
slow (slow-fsync, slow-open, slow-read, slow-syscall, slow-write, or
any slow-NAME by new link) records syscall entry timestamps in
thread-local storage and prints, at return, any call whose latency
meets a threshold (DWATCH_SLOW_MS, default 100), naming the syscall,
the elapsed time to the microsecond, and any errno returned. The bare
profile watches a curated set of filesystem-related calls expected to
be fast; slow-syscall watches everything; unrecognized invocation
names fall through to syscall::NAME:return with the matching entry
probe derived mechanically from the return probe list.
lock (lock-adaptive, lock-block, lock-lockmgr, lock-rw, lock-spin,
lock-sx, lock-thread) rides the dtrace_lockstat(4) block and spin
probes, printing the held-off thread (free from the standard event
tag), the holdoff duration, the lock class, the lo_name of the lock
through a single cast of arg0 to struct lock_object (the first member
of every kernel lock), and reader/writer intent on the probes that
report it. Holdoffs shorter than DWATCH_LOCK_MS (default 1; 0 shows
everything) are suppressed.
namei (namei-enoent, namei-entry, namei-failure) records the pathname
at vfs:namei:lookup:entry and reports it with the result at return.
Unlike the vop_lookup profile, which reconstructs paths from the name
cache one component at a time, this sees the whole path exactly as the
process requested it. namei-enoent hunts file-not-found storms -- the
single most common use of truss(1) -- without stopping the victim.
priv (priv-err, priv-ok) watches priv_check(9) verdicts, naming the
exact privilege denied -- something no syscall tracer can see, because
by the time EPERM surfaces the priv(9) value is gone. The number is
decoded by priv_string[], a new libdtrace inline table in the errno.d
and signal.d tradition, mechanically generated from sys/priv.h (247
entries) and installed to /usr/lib/dtrace where dtrace(1) auto-loads
it; on older releases it is a drop-in file like the module itself.
coredump (coredump-top) watches for delivery of signals whose default
action produces a core, per the SIGPROP_CORE entries of the sigproptbl
in kern_sig.c, and renders a verdict the same way and in the same
order the kernel will decide it: ignored or caught per the target's
struct sigacts, then the coredump() gauntlet of kern.coredump,
kern.sugid_coredump vs P_SUGID, procctl(2) PROC_TRACE_CTL, and
RLIMIT_CORE -- the sysctl knobs read live through kernel globals.
Where a coredump-worthy signal will produce no core, the verdict says
precisely which policy ate it. coredump-top maintains a cumulative
catalog of coredump-worthy signals by process and signal, refreshed
every 3 seconds in the style of systop; combine the event profile with
`-O cmd' to capture state as each event occurs.
hang (hang-top) pairs sched:::sleep with sched:::wakeup through a
tid-keyed timestamp array and prints, as each thread wakes, any sleep
that meets a threshold (DWATCH_HANG_MS, default 1000), naming the
sleeper in the details and the waker in the standard event tag. This
is the blocking the slow module structurally cannot see: a syscall
that never returns never reports its latency, while hang reports the
moment the wait ends, with the full duration. hang-top maintains a
cumulative catalog of long sleeps by process (count and maximum) in
the style of coredump-top.
jail (jail-attach, jail-get, jail-remove, jail-set) watches the jail
management plane -- jail(2), jail_set(2), jail_get(2), jail_attach(2),
and jail_remove(2) -- naming the operation, the jail id (taken from
the entry argument for attach/remove, from the return value for the
others), and any errno. Complements the dwatch `-j jail' filter,
which scopes any profile to processes inside one jail; this watches
who manipulates jails, from any jail or none.
dtmalloc (dtmalloc-top, or any dtmalloc-NAME by new link) rides the
dtmalloc provider (one malloc and one free probe per malloc(9) type).
The event profile prints allocations and frees meeting a size
threshold (DWATCH_MALLOC_MIN, default 65536) -- who is allocating huge
kernel buffers. dtmalloc-top maintains a running catalog of net bytes
and outstanding allocation balance by type, sorted by net bytes so
leak suspects rise: a type that climbs without bound while the system
is in steady state is the suspect. The catalog reflects activity
since the watch began, and is honest about caches holding what they
allocate.
mib (tcp-retransmit, or any mib-NAME by new link) rides the
per-counter mib SDT probes of the network stack. The tcp-retransmit
profile curates the counters that signal send-path congestion or loss
-- data packet retransmissions, unnecessary retransmissions,
retransmit timer expirations, and connections dropped by retransmit
exhaustion -- decoded through an inline description table, answering
"is this network slow because TCP is resending?" as events with
process context rather than netstat(1) deltas. NB: the mib probes
exist only in kernels built with options KDTRACE_MIB_SDT (default in
-CURRENT via std.debug); the module documents this and dtrace(1)
refuses the script elsewhere, making the dependency self-announcing.
Four existing modules gain personalities. proc grows
proc-signal-fatal, filtering signal-send to signals whose default
disposition terminates the receiver, most-notably including
kernel-generated SIGSEGV/SIGBUS/SIGILL/SIGFPE that no kill(2) watcher
will ever see. errno now reads its invocation name: errno-NAME shows
only syscalls returning that errno, where NAME is a symbolic name from
errno.d or a number; links are installed for errno-EACCES,
errno-ECAPMODE, errno-ENOENT, errno-ENOTCAPABLE, and errno-EPERM (the
latter pairs covering capsicum(4) capability-mode violations), and any
other errno needs only a new link. sched grows sched-latency,
recording a timestamp at sched:::enqueue keyed by tid and printing at
sched:::on-cpu any run-queue wait meeting a threshold (DWATCH_SCHED_MS,
default 10) -- the literal measurement of scheduler delay on a system
with idle CPU that still feels sluggish. io grows io-slow, pairing
io:::start with io:::done through a bio-keyed timestamp array and
printing any request that meets a threshold (DWATCH_IO_MS, default
100), naming the device, command, size, and elapsed time; watched
against zvols and a pool's leaf vdevs this brackets where in a ZFS
stack the time is going, without touching unstable providers.
Document all of the above plus the DWATCH_HANG_MS, DWATCH_IO_MS,
DWATCH_LOCK_MS, DWATCH_MALLOC_MIN, DWATCH_SCHED_MS, and DWATCH_SLOW_MS
knobs in dwatch(1).
All 46 new invocation names were exercised through `dwatch -d' with a
profile-path sandbox emulating the installed hard links: every one
sources cleanly and emits the intended D -- probe selection per alias,
entry/return and sleep/wakeup pairing through thread-local and global
associative arrays, threshold and mask predicates picking up their
knobs, aggregation clauses and printa column layout in the -top
profiles, multi-line predicate rendering, and `-t' correctly
displacing each module's default test were verified by inspection of
the generated scripts. Invocations untouched by this pass generate D
identical to their previous output. Modules pass sh -n, fit 80
columns, and dwatch.1 passes mandoc -Tlint with no new warnings. A
validation harness performing a `dwatch -e' compile per profile
against the live kernel globs every staged profile for runs wherever
the dtrace device is present.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D58093
When looking up self process we can use `ps -p $$` directly rather
than grep which may find other processes ending in the expected PID.
Sponsored by: Dell Inc.
Reviewed by: markj, vangyzen
Differential Revision: https://reviews.freebsd.org/D58019
The OpenZFS merge 80aae8a3f8 introduced HAVE_SIMD() which checks for
HAVE_TOOLCHAIN_* defines via simd_config.h. The kernel module Makefile
was updated, but kern.pre.mk (static kernel build) and the libzpool/libzfs
Makefiles were missed, still using the old HAVE_SSE2 etc. names. This
caused all vectorized raidz, fletcher, and blake3 implementations to be
compiled out.
Import the ENVIRONMENT section from NetBSD, minus the variable that our
ctfmerge does not have. Alphabetize them, polish grammar and alignment,
and add the variables to the man database. While here, remove whitespace
from the end of some lines to quiet linter.
MFC after: 3 days
PR: 291186
Co-authored-by: Alexander Ziaee <[email protected]>
Obtained from: NetBSD (christos <[email protected]>, 8a0c0d8)
Differential Revision: https://reviews.freebsd.org/D54054
Notable upstream pull request merges:
#18509f16b3744d zstream: refactor common functions
#18573 -multiple Persist z_seq across znode eviction
s18611 eb0c674c2 zfs_ioctl: fix EBUSY race between quota queries and mount
#1863777e64d86e Fix self-deadlock when setting the "allocating"/"path"
vdev property
#18645e3082b923 freebsd: set mnt_time on the rootfs at mountroot time
#1865250d012b2a zbookmark_compare: handle "marker" bookmarks with negative
levels
#18664520eeeaa6 Improve performance of "zpool offline" for log devices
#186686b8f79877 Avoid more abd_t allocations in RAIDZ/dRAID
#1866999ab859c3 Optimize metaslab_set_selected_txg()
#1867397b9ba7a9 delegate: add 'send:encrypted' permission
#186872ea519c2a Avoid lookup overhead for nonexistent xattr directories
#1868887593ea2b Fix handling of _PC_HAS_HIDDENSYSTEM for FreeBSD
#186930483a8e0c Clean up embedded slog metaslab across txgs
#1869541311c665 RAIDZ: Optimize single data column writes
#1870637af89948 ddt_log: Fix refcount tagging for begin/commit
Obtained from: OpenZFS
OpenZFS commit: 37af899488
dt_print_sym() fills the symbol string via snprintf() in non-oformat
mode but the guarding `dtp->dt_oformat != 0 &&` for the dt_printf()
call causes the symbol is computed but never emitted.
This fixes tests:
- common.profile-n.t_dtrace_contrib.tst_sym_ksh
- common.profile-n.t_dtrace_contrib.tst_func_ksh
Reviewed by: markj
Fixes: 93f27766a7 ("dtrace: Add the 'oformat' libdtrace option")
MFC after: 3 days
Event: Halifax Hackathon 202606
Location: Room 208, Computer Science Building, Dalhousie University
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D57895
Port OpenZFS PRs #18597 and #18578 from zed to zfsd.
When activating a spare, sort candidates before trying them:
1. Distributed dRAID spare matching the failed vdev's group (fastest
rebuild via sequential resilver)
2. Regular spares
3. Non-matching distributed spares (kernel will reject anyway)
4. Within each tier: prefer rotational match, then smallest
sufficient size
Also try all healthy spares in order rather than stopping at the
first one, and use sequential rebuild for distributed spares.
libzpool had a number of undefined symbols related to xxhash after
xxhash.c was removed from the build.
Fixes: 8a62a2a565 ("zfs: merge openzfs/zfs@f8e5af53e")
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D56770
In some versions of LLVM (at least 21), the <*intrin.h> headers contain
unguarded duplicate typedefs; this isn't permitted prior to C11, and
libzpool is built as C99. FreeBSD's LLVM backported LLVM PR #153820
to fix this, but other versions of LLVM (e.g., upstream, or on Linux)
don't have the patch, so this breaks the build.
Add -Wno-error=typedef-redefinition to downgrade this from an error
to a warning.
MFC after: 2 weeks
Reviewed by: dim, emaste
Sponsored by: https://www.patreon.com/bsdivy
Differential Revision: https://reviews.freebsd.org/D56653
During a discussion about using -fms-extensions jhb pointed out that
we have them enabled in the kernel for gcc by default (even multiple
times in one part). I had missed all that and clang still failed on
my use case (needing another option).
The original cause for enabling them for our tree back then was that
we needed to support C11 anonymous struct/unions.
Our in-tree gcc 4.2.1, despite later patches, needed the
-fms-extensions to support these even though this was not the expected
use case for that option ( cc4a90c445 enabled it globally for the
kernel).
clang at that time (or at least when it became default for 10.0)
already was fine (with C11).
Any later gcc (4.6.0 onwards) did not need that option anymore, even
when compiled for -std=iso9899:1990 (which does not support anonymous
structs/unions) unless one would add -pedantic (see gcc git 4bdd0a60b27a).
This is also the reason why userland cddl sources now compile with the
option removed despite CSTD=c99.
The only driver which needed the option recently was ccp, but that was
fixed in 8d3f41dbcb by jhb.
So cleanup all uses cases of -fms-extensions for the moment as they are
no longer needed given all compilers currently supported seem to be
fine without them and gcc-4.2.1 was removed from the tree in stable/13
in 2020 (a9854bc381).
Reported by: jhb (all this but possibly the world CDDL parts)
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Reviewed by: emaste (earlier), imp, jhb, glebius
Differential Revision: https://reviews.freebsd.org/D55072
Notable upstream pull request merges:
#17000 -multiple cmd/zfs: support creating parents with canmount=no
#17926b481a8bbb Make zpool status dedup table support raw bytes -p output
#18238 -multiple Report space metrics per allocation class
#182405dad9459d Add --no-preserve-encryption flag
#182571eace5906 libzfs: use mount_setattr for selective remount including
legacy mounts
#182591e2c94a04 More consistent use of TREE_* macros in AVL comparators
#18262c58b8b7dc Fix check for .cfi_negate_ra_state on aarch64
#18264f109c7bb9 Add the --file-layout (-f) option to zdb(8)
#18268304de7f19 libzfs: handle EDOM error in zpool_create
#18275753f1e1e2 zstream: add a drop_record subcommand
#182760f90a797d Fix vdev_rebuild_range() tx commit
#182771e1d64d66 Fix log vdev removal issues
#1827806b0abfe6 Fix the send --exclude option to work with encryption
#18281e73ada771 libzpool: lift zfs_file ops out to separate source file
#18284d97945776 zstream: consolidate shared code
#18285 -multiple Separate SIMD compile checks for userspace and kernel
#18288c5905b2cb Implement lzc_send_progress
#18289 -multiple L2ARC: Add depth cap and write budget fairness for
persistent markers
#18290d45c8d648 FreeBSD: Improve dmesg kernel message prefix
#18294b403040c4 draid: fix data corruption after disk clear
#18296 -multiple libzfs: mnttab cleanup
#18300d35951b18 zpool clear: remove undocumented rewind flags
#18301ae7fcd5f9 fix libzfs diff mem leak in an error path
#1830365165df12 zilstat: add man page
#183147f65e04ab libzfs: scrub: only include start and end nv pairs if
needed for scrub
#18315 -multiple Sync ZVOLs block cloning conditions with file systems
#18330f80338fac zarcsummary: add man page
#18337f259a47c7 zpool-iostat.8: clarify first report shows per-second
averages
Obtained from: OpenZFS
OpenZFS commit: 3ee08abd2f
Notable upstream pull request merges:
#173584975430cf Add vdev property to disable vdev scheduler
#18031c77f17b75 Add snapshots_changed_nsecs dataset property
#18080dbb3f247e cmd/zfs: clone: accept `-u` to not mount newly created
datasets
#18089 -multiple Zstd: Update bundled library to version 1.5.7
#180912301755df Fix zfs_open() to skip zil_async_to_sync() for the
snapshot
#18093 -multiple L2ARC: Rework write throttling with DWPD rate limiting
and parallel writes
#180952dbd6af5e Rename several printf attributes declarations to
__printf__
#180968605bdfdd FreeBSD: unbreak compilation on i386
#18105794f1587d When receiving a stream with the large block flag,
activate feature
#18115765929cb4 DDT: Add locking for table ZAP destruction
#1811809e4e01e9 Fix history logging for `zpool create -t`
#181192f1f25217 icp: emit .note.GNU-stack section for all ELF targets
#181313fffe4e70 Fix --enable-invariants on FreeBSD
#18133d2f5cb3a5 Move range_tree, btree, highbit64 to common code
#1813654b141fab FreeBSD: Remove references to DEBUG_VFS_LOCKS
#18138cdf89f413 Flush RRD only when TXGs contain data
#18139a157ef62a Make sure we can still write data to txg
#18140cd895f0e5 remove thread unsafe debug code causing FreeBSD double
free panic
#181444f180e095 Fix activating large_microzap on receive
#1814635b2d3970 Lock db_mtx around arc_release() in couple places
#18154b36472052 nvpair: chase FreeBSD xdrproc_t definition
#1816021bbe7cb6 Improve caching for dbuf prefetches
#18177 -multiple Multihost Improvements
#181792646bd558 Allow rewrite skip cloned and snapshotted blocks
#18180aa29455dd Restrict cloning with different properties
#18184040ba7a7c libzfs: improve error message for zpool create with
ENXIO
#181881412bdc6c zfs_vnops_os.c: Move a vput() to after
zfs_setattr_dir()
#18198cc184fe98 Fix `send:raw` permission for send `-w -I`
#18208ba970eb20 Cleanup allocation class selection
#182120f9564e85 Simplify dnode_level_is_l2cacheable()
#18214370570890 Remove parent ZIO from dbuf_prefetch()
#18218bfb276e55 freebsd: Fix TIMESPEC_OVERFLOW for PowerPC
#18222d06a1d9ac Fix available space accounting for special/dedup
#18225d48967728 ICP: AES-GCM VAES-AVX2: fix typos and document
source files
#18226c8a72a27e ICP: AES-GCM assembly: remove unused Gmul functions
#18230 -multiple Fix zdb --key crash for unencrypted datasets, and
teach tests to understand this better
#18233 -multiple icp: add SHA-512 implementation using Intel SHA512
extension
#18245991fc56fa Introduce dedupused/dedupsaved pool properties
#182516a717f31e Improve misleading error messages for
ZPOOL_STATUS_CORRUPT_POOL
#182547744f0496 SIMD: libspl: test the correct CPUID bit for AVX512VL
#182556495dafd5 range_tree: use zfs_panic_recover() for
partial-overlap remov
#182563408332d7 zhack: Fix importing large allocation profiles on
small pools
#18258f8457fbdc Fix deadlock on dmu_tx_assign() from vdev_rebuild()
#18263f8e5af53e Fix redundant declaration of dsl_pool_t
Obtained from: OpenZFS
OpenZFS commit: f8e5af53e9
The barrier code was using semaphores which have been deprecated in
macOS and not working at all, causing a race condition. Since macOS
does not have pthread_barrier_*(), this change uses a condition
variable instead.
PR: 290958
Reported by: wosch
MFC after: 2 weeks
Reviewed by: imp, markj
Differential Revision: https://reviews.freebsd.org/D54018
Define this helper variable in one place and make it available while
building the entire base system.
Reviewed by: sjg
Obtained from: CheriBSD (mostly)
Differential Revision: https://reviews.freebsd.org/D53790
Ed Maste