Incorrect ELF might have PT_NOTE slightly larger than the needed to
contain all notes, and the PT_NOTE size could be larger than one page.
Then rtld mmaps just the notes bytes to parse. After the last note,
we iterate past the mapped region trying to read the Elf_Note header.
This was found in wild.
Require full elf note to fit into the [start_note, end_note) region to
continue the parsing. Check it in stages, first verifying the Elf_Note
header structure fits, to be able to read the name and data length.
After that, check the whole note against limit.
Reported and tested by: makc
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D59635
Implement buf_ring/drbr deferred transmit in iflib. This is intended
to allow the new simpler code path to replace mp_ring. This patch
makes the simple_tx outperform mp_ring by a wide margin when CPU is
the bottleneck (eg, cannot fill the NIC). See graphs at:
https://people.freebsd.org/~gallatin/mpring_vs_simple_tx
Note that the buf ring is used for contention, not capacity. Eg,
it is used as a place for contending threads to put packets without
waiting for a mutex. It is not designed to act as a software ring
on top of the hardware descriptors provided by the underlying NIC
driver. "stranded packets" are exceedingly rare due to the fact that
if there is enough load to use the buf_ring, there will probably be
more load coming that can be a drainer. Not scheduling a gtask to
drain is intentional, and we really on the timer as a fallback.
One thing I noticed while developing this patch is that a simple mutex
with no deferral generally outperformed both mp_ring and drbr at high
levels of contention for the same queue. This is inherent in a
bounded MPSC queue where multiple producers are contending on claiming
ring entries. So I came up with the idea of bounding the number of
producers such that the deferral ring would devolve to a mutex when
contention was high.
Identifying the crossover point in a general way was hard. On
different machines, the point between a mutex and a deferral ring was
very different and also depended on the placement of the producers. I
eventually realized that on the large AMD EPYC servers that I was
testing on, the crossover point generally coincided with the work
spilling into another CCX. So I developed an approach where we limit
the number of producers by limiting simultanious producers to the same
L3 AND by bounding the number of simultanious producers.
This limit can be adjusted via the sysctl net.iflib.max_producers.
The number of packets drained from the deferral ring is limited
by net.iflib.simple_drain_quota. The intent is to process just
enough packets in the gtaskq context so as to make space to
allow threads to make progress. The task also uses a trylock
so as to avoid blocking, waiting for a thread to drain.
This change also enables ALTQ support for simple tx.
Reviewed by: kbowling
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D58901
Always offer the VIRTIO_NET_F_GUEST_CSUM feature to the host, and not
only if RXCSUM is enabled on vtnet. Instead of using RXCSUM to control
whether this feature will be negotiated with the host, just use it to
control whether the VIRTIO_NET_HDR_F_DATA_VALID flag on an incoming
packet is processed (i.e., translated to the corresponding mbuf flag
only if RXCSUM is enabled on the vtnet interface).
This has two benefits:
1. Enabling/disabling RXCSUM on vtnet does not require feature
renegotiation.
2. The host is always allowed to send locally generated TCP or UDP
packets to the guest without computing a full checksum (by setting
the VIRTIO_NET_HDR_F_NEEDS_CSUM flag) and not only if the guest has
enabled RXCSUM on vtnet. Since locally generated packets do not
require a checksum, this saves otherwise unnecessarily wasted
computing power.
If a user of a FreeBSD guest really does not want to negotiate the
VIRTIO_NET_F_GUEST_CSUM feature with the host, it still can disable
the loader tunable hw.vtnet.csum_disable.
Reviewed by: kfv, tuexen
MFC after: 1 week
MFC to: stable/15
Differential Revision: https://reviews.freebsd.org/D59052
There are two parts that go together:
* `X86_MATCH_VFM()` to declare a matching pattern
* `x86_match_cpu()` to check if the current CPU matches one of the
patterns in an array.
The i915 DRM driver started to use this in Linux 6.14.
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D57700
The size of multiple embedded structs have changed and may lead to
problems (pci_error_handlers in pci_driver, dev_pm_info in struct device).
Allow these changes to be detected by bumping __FreeBSD_version.
MFC after: 3 days
This reverts commit fc0a5ae094.
This changed the sysctl node path and broke IB user space tools (ibv_*).
PR: 298485
Reported by: Ariel Ehrenberg (aehrenberg nvidia.com)
Add the missing locking to another two MO driver downcalls, as drivers
always expect it (e.g. rtw89 by assertion).
Add the lock and might_sleep assertions to the respective lkpi_80211_mo_*
downcalls.
PR: 298417
Reviewed by: bz
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D59608
We can implement device_set_wakeup_capable() in the !CONFIG_PM_SLEEP
case; we do not have the infrastructure in place for the CONFIG_PM_SLEEP
case so leave a pr_debug TODO.
Needed by an upcoming wireless driver.
MFC after: 3 days
Reviewed by: dumbbell
Differential Revision: https://reviews.freebsd.org/D58883
Add the #define for PCI_IRQ_AFFINITY and leave a pr_debug note in
pci_alloc_irq_vectors() that it is unimplemented.
The flag is needed by an upcoming WiFi driver.
MFC after: 3 days
Reviewed by: dumbbell
Differential Revision: https://reviews.freebsd.org/D58881
We use one to implement the other given direction stays unused.
This is needed by an upcoming wifi driver.
MFC after: 3 days
Reviewed by: dumbbell
Differential Revision: https://reviews.freebsd.org/D58880
lkpi_pci_msi_desc_alloc() takes an unsigned int, not an int.
While here make sure the prototype is visibile in interrupt.h as
well before use to avoid -Wimplicit-function-declaration errors.
Discovered while working on a wireless driver.
MFC after: 3 days
Reviewed by: dumbbell
Differential Revision: https://reviews.freebsd.org/D58879
For pci_select_bars() we use the LinuxKPI internal information, while
for pci_select_bars() we fall back to the native PCI stack.
Needed by a wifi driver.
MFC after: 3 days
Reviewed by: dumbbell
Differential Revision: https://reviews.freebsd.org/D58878
Needed by a wireless driver (not really using it) to compile.
MFC after: 3 days
Reviewed by: dumbbell
Differential Revision: https://reviews.freebsd.org/D58877
Add #include rculist.h to ethtool.h to fullfill expectations of
Linux drivers without having to modify them.
MFC after: 3 days
Reviewed by: dumbbell, emaste
Differential Revision: https://reviews.freebsd.org/D58874
list_for_each_entry_rcu() can take an optional condition. Add the
macro argument so code remains compiling but do not do anything with
it just yet.
Leave comments as list_for_each_entry_rcu() likely should have a
different implementation.
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Reviewed by: dumbbell
Differential Revision: https://reviews.freebsd.org/D59292
Common code between router_prepare_read() & router_prepare_write().
Eventually will be used by other commands (e.g. hotplugging) aswell.
Reviewed by: ngie
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D59668
This fixes the build for kernels with INVARIANT_SUPPORT but without
INVARIANTS.
Fixes: 2e376cca37 ("rangelock: Reimplement _rangelock_cookie_assert()")
tpmcrb_transmit() read CRB_CTRL_STS before requesting locality 0.
An AMD Pluton fTPM (like FrameWork Desktop) using the plain CRB start method
reads the control area as all-ones until locality is assigned, so bit 0 looks
like a stuck tpmSts and every command failed with EIO.
With RANDOM_ENABLE_TPM the harvester retries every 10 seconds, so this printed
"Device has Error bit set" forever.
Reviewed by: kbowling
Approved by: kbowling
MFC after: 2 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D59661
mmu_radix_extract() walks the page tables without holding the pmap lock,
unlike its hash MMU counterpart moea64_extract(). A concurrent unmap can
free and recycle the page table page being walked, so the read returns
whatever now occupies that memory and the caller gets a physical address
that never existed.
That is how mmu_radix_sync_icache() came to hand a bogus address to
__syncicache() and panic the machine. Commit 1574ca1955 worked around
it by taking the pmap lock in mmu_radix_sync_icache(), but the machine
independent callers of pmap_extract() - vm_sync_icache(), proc_rwmem()
and the vslock() paths - remain exposed to the same failure.
Rename the existing body to mmu_radix_extract_locked(), which asserts the
lock, and make mmu_radix_extract() a thin wrapper that acquires it.
mmu_radix_sync_icache() already holds the pmap lock, so it calls the
locked variant directly and neither recurses nor reacquires the lock once
per page.
Suggested by: alc
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D59320
Reviewed by: markj, jhibbits
Notable upstream pull request merges:
#188086d4ff3b4f Pool split leaks DTL spacemap objects
#188440833cfd64 Decline Direct I/O reads on a file handle after a benign
verify failure
#1885339764eb06 Fix double free when cloning a block with a pending free
#1890151fe1fb84 dmu_send: a meta-dnode range does not start where an
object does
#18903904d4327f dmu_objset_open_impl: unregister prop callbacks on error
#18906e45640415 zdb: add detailed diagnostics for MOS leaks and spacemap
refs
#189091b7143557 Keep a grown vdev from adopting an older pool's labels
#18912 -multiple Defer destruction of a snapshot that a mount is holding
open
#1892133ae06bb6 ZIO: Batch vdev children completions
#189407853c27eb ZIO: Batch lightweight ZIOs
#18967 -multiple Speed up pool rewind and allow it to tolerate minor damage
#18968b0fff48dc arc: harness buf_hdr's anon state invariant checks
#18969b0da5c4ce dsl_crypt: validate every key change-key will rewrap
#18974b19fa7c97 log_spacemap: make zfs_log_sm_blksz tunable
#18982c878ee452 zstream: track and limit memory use
#18991a1261384a zstream: self-tuning queues
#18992fd639b075 zcp: support bookmarks in zfs.sync.destroy
#189930498c554d vdev_disk_open: fix ENOENT retry & cleanup error handling
#190035280a08f6 Write uberblocks only to vdevs used by the txg
#19008c47591d5d arc: apply arc_min tunables set before arc_init() has run
#19016cd578e518 dnode: fix heap-use-after-free in dnode_rele_and_unlock()
#19017b824f3116 zstd: declare __asan_*_memory_region() for user space
builds
#190296860d1c2f DDT: Use proper size when calling kmem_free on
ddt_prune_entry_t
#19041aa26ca67b Do not complete a device removal that hit IO errors
#190428e5bb1d4b Wait for the initialize and trim threads to exit
#19051ba524e0d7 zpool: things get messy with 0 columns
#1905369ef525a4 dsl_deadlist_space_range: use correct tree when summing
up space
#190590ee8ea01b FreeBSD: use sys/abi_compat.h for time32_t
#19062e08bc2973 Skip interior dnode slots when receiving DRR_FREEOBJECTS
#1906692a3904af Wait for the txg that finished a scan to sync
#19071db07a5e0a dbuf: account dirty data for embedded writes
#190721a8e51491 Fix self-deadlock when cloning a range within the same
file
#1907575bd31c28 L2ARC: do not feed a device while its rebuild is pending
#1908356e8d234b Stop the TRIM and initialize threads when a vdev goes
offline
#19090ab994acdd dmu: remove dmu_prealloc()
#1909261fbdf1a2 zfs: spa_sync_upgrades() should only take lock when needed
Obtained from: OpenZFS
OpenZFS commit: 02b5baf13c
selwakeup() only wakes select/poll waiters; it does not notify the kqueue
knote registered on the device, so EVFILT_READ never fired when a frame
became available.
- sysutils/ataidle is no more: it was superseded by camcontrol(8).
- Sort the models table alphabetically.
- Use the SPDX License ID instead of the longhand licensing tort in the
manpage header.
- Note that the driver has been heavily modified in 15.1 and later to
support additional platforms and functionality.
- Trim down SYNOPSIS.
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D59470
This new manpage describes what the apple_bce driver services, how the
driver is configured, and what Apple models the driver claims to
support.
Reviewed by: seuros
Differential Revision: https://reviews.freebsd.org/D59467
Yet again. I was trying to make the svc_vc_backchannel()
operations do double duty and be used by the clnt_rdma.c code
as well. It got too messy, so this reverts svc_vc.c back to
its pre-glue form and adds the small changes needed to support
a separate set of svc_rdma_backchannel_xxx() functions.
This commit should not affect non-RDMA behaviour.
MFC after: 3 months
Fixes: 884ee8d6c9 ("nfscl: Add some glue for client side NFS over RDMA")
Firmware expose a DSDT sized for the largest SKU of the platform,
so a verbose boot prints an "ignored" line for every vacant
processor slot. A vacant slot has no enabled MADT entry; a CPU that
failed to come online does.
Reviewed by: olce, adrian
Differential Revision: https://reviews.freebsd.org/D59551
Add driver for Intel Power Management Controller (PMC) found on Sunrise Point PCH chipsets.
This device exposes S0ix sleep state residency counters and power management status.
Sysctls provided:
dev.intelpmc.0.slp_s0_residency_us - Time in deepest sleep (us)
dev.intelpmc.0.ltr_ignore - LTR ignore mask
dev.intelpmc.0.pm_cfg - PM configuration register
dev.intelpmc.0.pm_sts - PM status register
dev.intelpmc.0.access_denied - Firmware lock status
Supported devices for now:
- Sunrise Point-LP (0x9D21)
- Sunrise Point-H (0xA121)
Note: Later PCH generations (Cannon Lake, Tiger Lake, ect.) have different PMC register layouts according to the datasheet and would need per-generation tables.
I avoided adding untested hardware in case they have a quirk.
Reviewed by: olce, adrian
Differential Revision: https://reviews.freebsd.org/D54881
The active LBA format's MS field was never examined. I/O to a
metadata-formatted namespace carries neither interleaved metadata
nor MPTR, so every command is malformed, yet the namespace attaches
as a disk with the wrong sector size.
Reviewed by: imp, adrian
Differential Revision: https://reviews.freebsd.org/D59625
The current interrupt path uses the IRQ resource value directly as the
LPC SIRQ selector in TPM_INT_VECTOR and already restricts it to 1 through
15. This is a driver limitation: a parent interrupt number need not equal
an LPC SIRQ channel, and SPI TPMs can use a separate parallel interrupt.
On the reported system with ACPI IRQ 45, the existing range check runs
after handler registration and returns before disabling firmware interrupt
delivery. This can leave a polling device with a handler on an asserted
source.
Disable and verify interrupt delivery before registering a handler or
starting common TPM services. Preserve the existing range policy, using
polling without registering a handler for routes rejected by that check,
and release their IRQ resources. Keep a failed setup's potentially stale
output cookie out of the device state; the interrupt framework may
already have removed that handler.
Cancel timed-out locality requests in the shared request helper, covering
both interrupt programming and commands. Permit attach after a locality
timeout if the enable register proves delivery is already off, allowing
later command recovery. If delivery remains enabled and cannot be
quiesced, fail attach instead of exposing a device node for possible later
recovery. Report this as a quiesce failure since locality acquisition as
well as register programming can fail.
On detach, publish dying and serialize with commands before quiescing TPM
interrupt delivery. Do this for polling devices too, since firmware may
have re-enabled delivery and resume-time quiescing may have failed. Quiesce
before common release destroys the command lock and before removing any
handler. Report hardware quiesce failure while completing software cleanup.
Reported by: adrian
Reviewed by: adrian, imp
MFC after: 2 weeks
Sponsored by: BBOX.io
Differential Revision: https://reviews.freebsd.org/D59645
On Linux `dma_length` field of `struct scatterlist` is present on the
arches where DMA mapping code is able to coalesce adjacent segments
of physical address space. It contains total length of coalesced
segments while `length` field contains non-coalesced length of each
segment. On other arches `dma_length` is aliased to `length` field with
`sg_dma_len` macro. As FreeBSD does not merge scatterlist segments it
do not have `dma_length` field. It is appered that at least i915kms
driver depends on existence of `dma_length` field.
Add the field and disable it by default. To enable add to Makefile
.if ${MACHINE_CPUARCH} == "i386" || ${MACHINE_CPUARCH} == "amd64" || \\
${MACHINE_CPUARCH} == "aarch64" || ${MACHINE_CPUARCH} == "powerpc"
CFLAGS+= -DCONFIG_NEED_SG_DMA_LENGTH
.endif
Reported by: Ryan Fahy
Reviewed by: bz
MFC after: 1 month
GHI: https://github.com/freebsd/drm-kmod/issues/315
Differential Revision: https://reviews.freebsd.org/D59630
Konstantin Belousov