Pull arm64 fixes from Will Deacon:
- Fix the recently merged user ABI for the Arm CMN PMU driver filtering
logic so that the ordering matches the hardware spec
- Fix spurious warning when attempting to read PROT_NONE mappings of
/dev/mem
- Allow WFxT to be disabled on the command line, which is necessary for
some configurations of recent Apple SoCs
- Fix EL2 fine-grained trap configuration for the CPU PMU
- Fix MIDR matching when applying CPU errata workarounds in a VM
* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
arm64/boot: Disable trapping of PMZR_EL0 writes to EL2
arm64: errata: match the target implementation CPU's own MIDR
arm64: Add override for WFxT
arm64: io: Reject non-user protection in ioremap_prot()
perf/arm-cmn: Fix multi-filter encoding
perf_clear_branch_entry_bitfields() clears the bitfields of struct
perf_branch_entry one by one and leaves from/to alone, since callers
overwrite those straight away. The list has to be kept in sync with the
struct by hand and has already fallen behind: new_type and priv were
added to perf_branch_entry and never added here.
Only BRBE writes those two, and neither for every record.
brbe_set_perf_entry_type() leaves new_type alone for a branch type it
does not recognise, and priv is not set for source-only records.
arm_pmuv3.c allocates the per-CPU branch stack with kmalloc(), so such a
record reaches userspace with whatever the slot held: uninitialised
kmalloc() data on the first pass over the buffer, the previous record's
values after that. Nothing under arch/x86/events/ writes either field,
so only arm64 is affected.
Assign the whole entry at each site instead. Everything not named is
then zero, and there is no list to keep in sync. The bitfields add up to
exactly 64 bits, so the struct has no padding to leave undefined.
perf_clear_branch_entry_bitfields() has no callers left, so remove it.
perf_entry_from_brbe_regset() assigns an empty literal instead, since it
fills from/to conditionally. PERF_BR_SPEC_NA is 0, so dropping the
explicit spec assignment changes nothing.
Fixes: b190bc4ac9 ("perf: Extend branch type classification")
Fixes: 5402d25aa5 ("perf: Capture branch privilege information")
Suggested-by: Peter Zijlstra <[email protected]>
Signed-off-by: Puranjay Mohan <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Tested-by: Yifan Wu <[email protected]>
Link: https://patch.msgid.link/[email protected]
Pull arm64 fixes from Will Deacon:
"In this batch we've got a couple of hibernation fixes, a couple of
minor MTE fixes, some per-cpu codegen fixes (which were found as part
of Mark's series adding preemptible this_cpu_*() operations) and a fix
for the Arm CMN PMU driver.
Summary:
- Fix hypercall arguments when resetting EL2 vectors during
hibernation
- Fix hibernation with 52-bit capable kernels on machines without
52-bit addressing, similarly to the recent kexec fix
- Fix a bunch of clumsy codegen issues with our per-cpu accessors
- Fix MTE ptrace documentation to reflect the de-facto ABI behaviour
- Fix pthread_join() usage in MTE selftest
- Fix port selection in the Arm CMN PMU driver"
* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
arm64: mte: Fix PTRACE_{PEEK,POKE}MTETAGS error documentation
kselftest/arm64: Fix size of thread_data values for pthread_join()
arm64: percpu: Fix LSE operations on {8,16}-bit types
arm64: percpu: Fix this_cpu_and() mask generation
arm64: percpu: Fix this_cpu_write() casting
arm64: hibernate: clone only the linear map that exists at runtime
perf/arm-cmn: Fix wp_dev_sel2 setting for multi-DTM configurations
arm64: hibernate: pass HVC_SET_VECTORS args to the resume hvc
The current special-case for EVICT_STATE_SEL filtering effectively
assigns the "filter" and "filter2" controls in the opposite order from
how the CMN S3 r2 TRM states "Filtering is programmed in pmu_hbt_lbt_sel
and pmu_evict_state_sel". On reflection, not only does this seem
unnecessarily non-obvious to users, but it's also likely to be a problem
for scaling to a full multi-filter abstraction in future. There is a
logical order to filters based on their bitfield positions in the
pmu_event_sel register, which the TRM descriptions allude to, and the
cmn_filter_select enum already (almost) follows, so let's fix the UABI
to follow suit while it's still unreleased.
Fixes: 09178f536b ("perf/arm-cmn: Plumb in new filter types")
Signed-off-by: Robin Murphy <[email protected]>
Reviewed-by: Ilkka Koskinen <[email protected]>
Reviewed-by: Leo Yan <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
When MXP_MULTIPLE_DTM_EN is TRUE, each DTM will monitor at most
two device ports. In this case, {wp_dev_sel2, wp_dev_sel} will
only use values 2'b00 and 2'b01 per DTM.
Previously the setting allowed values beyond the supported range
per DTM, which could cause each DTM to select invalid ports when
MXP_MULTIPLE_DTM_EN is TRUE.
Fix this by only setting CMN_DTM_WPn_CONFIG_WP_DEV_SEL2 when
!multi_dtm.
Fixes: 60d1504070 ("perf/arm-cmn: Support new IP features")
Signed-off-by: Shouping Wang <[email protected]>
Reviewed-by: Robin Murphy <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
The overflow IRQ handler dereferences cpu_hw_evt before the null check.
Move the check first. Defensive only; the cookie is valid on the normal
path today.
Fixes: a8625217a0 ("drivers/perf: riscv: Implement SBI PMU snapshot function")
Assisted-by: DeepSeek:deepseek-v3
Signed-off-by: Xixin Liu <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Paul Walmsley <[email protected]>
The available-counter mask was a single unsigned long, but iteration
uses RISCV_MAX_COUNTERS, which is 64. On RV32 that reads past the object.
Filling with an unsigned-long bit at index 32 and above is also wrong.
Use DECLARE_BITMAP and set_bit/bitmap helpers. Walk each bitmap word
into CFG_MATCH when checking events, when allocating an index, and when
stopping all counters. Set the counter base to i times BITS_PER_LONG.
Share the CFG_MATCH ecall through a small helper so the 32-bit argument
split is not duplicated. On qemu-system-riscv32 the probe bitmap has bits
above XLEN set, so the first word alone is not enough.
Fixes: e999143459 ("RISC-V: Add perf platform driver based on SBI PMU extension")
Assisted-by: DeepSeek:deepseek-v3
Signed-off-by: Xixin Liu <[email protected]>
Link: https://patch.msgid.link/[email protected]
Cc: [email protected]
[[email protected]: updated to apply; fixed checkpatch.pl issues]
Signed-off-by: Paul Walmsley <[email protected]>
Overflow status and restart masks are u64, but bits were built with
BIT(). On RV32 that is an unsigned long shift, so indices >= 32 truncate
or wrap and corrupt the mask.
Use BIT_ULL() for those u64 bitops.
Fixes: a8625217a0 ("drivers/perf: riscv: Implement SBI PMU snapshot function")
Assisted-by: DeepSeek:deepseek-v3
Signed-off-by: Xixin Liu <[email protected]>
Link: https://patch.msgid.link/[email protected]
Cc: [email protected]
[[email protected]: updated to apply]
Signed-off-by: Paul Walmsley <[email protected]>
Pull RISC-V updates from Paul Walmsley
- Add initial definitions and discovery for the Smcsrind, Sscsrind,
Smcntrpmf, Ssccfg, Smcdeleg, Zicclsm, Ziccamoa, Ziccif, Ziccrse,
Za64rs, and Ssqosid RISC-V ISA extensions
- Improve the RISC-V update_mmu_cache_range() implementation by using
flush-by-ASID, enabling performance improvements on
microarchitectures that support related optimizations; and by taking
advantage of the Svinval RISC-V ISA extension on microarchitectures
that support it
- Shrink CFI shadow stack allocation further (to 512MB) to save virtual
address space (and physical memory on systems with strict overcommit
policies)
- Add initial CPU context switch support for QoS tagging (Ssqosid)
- Change our vector misaligned access speed test code to be
synchronous, avoiding a bunch of bugs related to unnecessary
asynchronicity
- Enable ARCH_HAS_ACPI_TABLE_UPGRADE for RISC-V to help with ACPI table
debugging (similar to ARM64 and x86)
- Extract vDSO section offsets at build time, rather than run time, to
avoid the boot time overhead
- Use assembler directives to control the use of instructions from the
RISC-V ISA extensions Zacas and Zabha, rather than compiler -march
flags that could affect the whole kernel (similar to ARM64)
- Add a kselftest for kprobes support for the c.jal instruction on
RISC-V
- When UEFI runtime services are available, use them to restart and
power off
- Fix ARCH_HAS_ACPI_TABLE_UPGRADE for RISC-V by adding a missing
include file to the ACPI code
- Move contiguous DMA memory reservation later in the boot, to ensure
that the correct global and per-node pools are allocated when
CONFIG_CMA_SIZE_PERNUMA is enabled
- Disallow probes on breakpoint handlers to avoid recursing
indefinitely into do_trap_break()
- Patch the compat vDSO during runtime alternatives processing, not
only the standard vDSO
- Remove some leftover XIP support elements missed by commit
9b3a2be848 ("riscv: Remove support for XIP kernel")
- Fix broadcast timer switching in ACPI LPI power states by
implementing a RISC-V version of arch_get_idle_state_flags()
- Miscellaneous fixes, including: vDSO makefile simplification; marking
our default_power_off() as __noreturn; error path fixes in the PMU
SBI perf driver; RISC-V extension capitalization consistency; the use
of BIT() macros in one of our include files; and some documentation
fixes
* tag 'riscv-for-linus-7.3-mw1' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux: (39 commits)
RISC-V: hwprobe: Use BIT macro to avoid warnings
riscv: alternative: Also patch the compat vDSO
riscv: alternative: Use the statically extracted vDSO section offsets
riscv: vdso: Add symbols for the alternative section boundaries
riscv: Add Ziccamoa, Ziccif, Ziccrse, and Za64rs to cpufeature and hwprobe
riscv: Add Zicclsm to cpufeature and hwprobe
riscv: cpufeature: Clarify ISA spec version for canonical order
riscv: Standardize extension capitalization
riscv: kprobes: Prevent probes in breakpoint handlers
riscv: cmpxchg: Use .option arch for Zacas and Zabha
riscv: acpi: Handle LPI architectural context loss flags
riscv: further remove XIP
riscv: Add support for srmcfg CSR from Ssqosid extension
riscv: Detect the Ssqosid extension
dt-bindings: riscv: Add Ssqosid extension description
dt-bindings: riscv: add Counter delegation ISA extensions description
RISC-V: Add Ssccfg/Smcdeleg ISA extension definition and parsing
RISC-V: Add Ssccfg extension CSR definition
dt-bindings: riscv: add Smcntrpmf ISA extension description
RISC-V: Add Smcntrpmf extension parsing
...
Pull bitmap updates from Yury Norov:
"The usual set of fixes, cleanups and performance improvements together
with a couple of new tests:
- bitmap_find_next_zero_area_off() optimization (Sunyi)
- bitmap_find_next_zero_area_off(): return size when no zero area is
found (Yury)
- bitmap vs IDA vs Maple Tree performance test (Yury)
- get rid of cpumap_print_to_pagebuf() (Yury)
- use nr_node_ids in __nodemask_pr_numnodes() (Li RongQing)
- bitops: make the *_bit_le functions use unsigned long (Benjamin)
- bitmap scatter & gather test fix (Christophe)
- use __ASSEMBLER__ in bitmap header files (Thomas)"
* tag 'bitmap-for-7.3' of https://github.com/norov/linux: (25 commits)
lib: test bitmap vs IDA vs Maple Tree performance for region allocations
bitmap: Return size when no zero area is found
media: s5p-mfc: Treat bitmap size as allocation failure
crypto: ccp: Treat bitmap size as allocation failure
powerpc/msi: Treat bitmap size as allocation failure
ARM: dma-mapping: Treat bitmap size as allocation failure
bitmap: drop bitmap_next_set_region()
nodemask: reduce bitmap width to nr_node_ids in __nodemask_pr_numnodes()
bitmap: Properly initialise destination bitmap for scatter & gather test
lib/bitmap-str: get rid of cpumap_print_to_pagebuf()
perf: Use sysfs_emit() for cpumask show callbacks
PCI/sysfs: Use sysfs_emit() for cpumask show callbacks
RDMA/hfi1: Use sysfs_emit() for cpumask show helper
hwtracing: hisi_ptt: Use sysfs_emit() for cpumask show
fpga: dfl-fme-perf: Use sysfs_emit() for cpumask show
devfreq: Use sysfs_emit() for cpumask show callbacks
cpu: Use sysfs_emit() for cpumask show callback
x86/events: Use sysfs_emit() for cpumask show callbacks
powerpc: Use sysfs_emit() for cpumask show callbacks
arm: Use sysfs_emit() for cpumask show callbacks
...
Sashiko pointed out various UAF and memory leak issues around
pmu_sbi_device_probe() error paths.
If the probe fails, here are list of cleanups needed.
a. Already registered pmu must be freed
b. per cpu IRQ must be released
c. pmu_ctr_list data structure must be freed
d. cpu hotplug state must be cleaned up only if added.
Fix the resource cleanup by reorganizing the code around probe failure.
Reported-by: Sashiko AI <[email protected]>
Reviewed-by: Charlie Jenkins <[email protected]>
Signed-off-by: Atish Patra <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Paul Walmsley <[email protected]>
PERF_SAMPLE_BRANCH_HW_INDEX is supported by BRBE so hw_id is passed to
userspace, but it's never set by the BRBE driver. Zero initialize it as
it should be according to the docs:
* For the architectures whose raw branch records are
* already stored in age order, the hw_idx should be 0.
It's probably too risky to remove PERF_SAMPLE_BRANCH_HW_INDEX from BRBE
now in case anyone is setting it and reading the value, but zero
initializing the whole struct also protects against the same issue with
new fields that are added in the future.
Fixes: 58074a0fce ("perf: arm_pmuv3: Add support for the Branch Record Buffer Extension (BRBE)")
Signed-off-by: James Clark <[email protected]>
Reviewed-by: Anshuman Khandual <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
If you were disappointed at how minimal the initial CMN S3 support
looked compared to previous versions, then oh boy is it time to put your
party hats on... The biggest batch of incompatible changes yet comes not
with a new CMN product, but a point release of an existing one. We've
got new filters, loads of changes to existing events, register fields
moved around for no good reason, and much, much more! On the upside, we
do at least gain a means of working around the isolation feature.
As such, for the sake of sanity in the driver it is easiest to split it
into a distict "model" for our internal abstractions despite it bearing
the same part number as r0/r1.
Signed-off-by: Robin Murphy <[email protected]>
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Add the logic to handle events with the upcoming new filter controls.
Since for now we will have the sole invariant of all EVICT_STATE_SEL
events having HBT_LBT_SEL as a secondary filter, for the sake of
simplicity we can just special-case that, and save the complication
of a full multi-filter abstraction until unavoidably necessary.
Reviewed-by: Leo Yan <[email protected]>
Signed-off-by: Robin Murphy <[email protected]>
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
The ABI hole I have dug myself into requires the driver to know which
event encodings are associated with which particular filter control.
Since we will soon have a notion of multiple filters per event, refactor
the event data to encapsulate filters in an explicit structure, which
can then more easily scale as an array in future.
Signed-off-by: Robin Murphy <[email protected]>
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
We're soon going to need to cope with events having multiple filters,
plus the filter fields themselves moving around, wherein any more inline
if/else logic will struggle to scale. Add a more general abstraction for
the node-specific filter controls, and rejig the pmu_event_sel filter
programming around it in a more extensible manner.
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Robin Murphy <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
CMN has already grown many more event-specific filters than the original
Occupancy ID, but since they are all independent of each other we've
just overloaded them onto the same name. Before we add yet more, and
they begin to overlap, rename all our "occupid" variables to "filter" so
that things can be a bit clearer and more consistent (but leaving the
format attribute itself, to avoid UAPI concerns).
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Robin Murphy <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
The 64-bit Programmers' model extension, now named FEAT_CSPMU_EXT64,
makes all the non-counter registers 64-bit as well, of which we most
need to care PMEVFILTR<n>, PMEVTYPER<n>, and PMCR since it changes
location. Our event config fields are ready for this internally, but
we need a few more tweaks to propagate 64-bit values properly from
end to end.
Since 64-bit events leave no room for our existing "cycles" encoding,
and we don't really expect to see implementations of it anyway, we
deliberately omit generic support for the architectural fixed cycle
counter in this case to save some bother. At worst, vendor modules
can still implement their own encoding if they really want to.
Signed-off-by: Robin Murphy <[email protected]>
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Since commit 55b48e23f5 ("genirq/devres: Add error handling in
devm_request_*_irq()"), devm_request_irq() automatically logs
detailed error messages on failure. Remove the now-redundant
driver-specific dev_err() and dev_err_probe() calls.
Signed-off-by: Pan Chuang <[email protected]>
Acked-by: Yushan Wang <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Since commit 55b48e23f5 ("genirq/devres: Add error handling in
devm_request_*_irq()"), devm_request_irq() automatically logs
detailed error messages on failure. Remove the now-redundant
driver-specific dev_err() calls.
Signed-off-by: Pan Chuang <[email protected]>
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Since commit 55b48e23f5 ("genirq/devres: Add error handling in
devm_request_*_irq()"), devm_request_irq() automatically logs
detailed error messages on failure. Remove the now-redundant
driver-specific dev_err() and dev_err_probe() calls.
Signed-off-by: Pan Chuang <[email protected]>
Reviewed-by: Shuai Xue <[email protected]>
Acked-by: Frank Li <[email protected]>
Reviewed-by: Xu Yang <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
The ACPI spec bizarrely indicates that the PMU interrupt can be
edge-triggered, which contradicts the very basics of the PMU
architecture (SW is required to clear the interrupt condition for
the level to drop).
Remove the code parsing this flag and always flag the interrupt
as level triggered, no matter what firmware says.
Signed-off-by: Marc Zyngier <[email protected]>
Reviewed-by: Zenghui Yu <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
If we have 64-bit counters, we can reasonably assume we'll never have to
handle an overflow before the end of the universe (since we're a system
PMU with no sampling). Thus even if firmware does specify an IRQ, we can
still continue in the event of being unable to request it. This can help
systems where IRQs cannot be claimed exclusively, or are broken in other
ways.
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Robin Murphy <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
When waiting for a sub-module to register, we return a bare
-EPROBE_DEFER that ends up showing the end user:
platform arm-cs-arch-pmu.1: deferred probe pending (no reason)
wherein it's not necessarily clear that they might need to take some
action to ensure the appropriate module is available to load. Let's use
dev_err_probe() here so we can show exactly what we're waiting for.
Similarly, in the case where something's gone horribly wrong with an
already-registered module, we can use dev_WARN() to standardise the
device/driver attribution rather than just open-coding "arm_cspmu".
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Robin Murphy <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
On ACPI systems, it has not actually been possible for userspace to
reliably tell which PMU corresponds to which APMT entry for types
other than "ACPI device" - the evidence trail only leads from the
arbitrarily-numbered PMU device to its arbitrarily-numbered parent
platform device that has no distinguishing features either.
While we've now improved the platform device creation to associate the
actual APMT unique ID, we may as well also tweak the PMU devices to
substitute the arbitrary number with a different arbitrary number that
might be more directly meaningful based on the APMT definitions.
We don't have an equivalent for Devicetree, but in that case the
platform devices are at least identifiable via their sysfs-visible
of_node.
Signed-off-by: Robin Murphy <[email protected]>
Reviewed-by: Ilkka Koskinen <[email protected]>
[will: Remove unneeded semicolon reported by coccinelle]
Signed-off-by: Will Deacon <[email protected]>
Implement CRB filtering per CXL 4.0 8.2.7.2.2, and extend the
current filtering support beyond HDM. CRB filtering is only
permitted for the DDR Interface, Queue Occupancy, Queue Residency
and Retry event groups (CXL 4.0 Table 13-5), and only when counting
a single event (a single mask bit). Because these group IDs are
scoped by the CXL vendor ID, events from other vendors are also
rejected. For example, to count DDR activates on channel 2 only:
perf stat -a -e cxl_pmu_mem0.0/ddr_act,crb_filter_en=1,crb=0x02FFFFFF/
Placing the 32-bit CRB value at config2:32-63 leaves the
existing HDM value at config2:0-15 untouched and avoids needing
a new config3.
Signed-off-by: Harshal Thakkar <[email protected]>
[davidlohr: multiple fixes]
Signed-off-by: Davidlohr Bueso <[email protected]>
Reviewed-by: Richard Cheng <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Add support for CXL 4.0 events that are exposed by the CPMU hardware
but not present in the driver. Such events are defined in Table 13-5
of the spec.
Signed-off-by: Harshal Thakkar <[email protected]>
[davidlohr: add missing throttle and queue occupancy events]
Signed-off-by: Davidlohr Bueso <[email protected]>
Reviewed-by: Richard Cheng <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
The HDM decoder filter configuration register is 32 bits wide, but the
driver programs it with a 64-bit writeq(). The filter value never
exceeds 32 bits, so the upper half of the write is always zero and
lands in the adjacent Filter ID 1 (Channel/Rank/Bank) configuration
register at offset+4.
Fixes: 5d7107c727 ("perf: CXL Performance Monitoring Unit driver")
Signed-off-by: Davidlohr Bueso <[email protected]>
Reviewed-by: Richard Cheng <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
The DWC PCIe Time-Based Analysis Data Register (the counter for time-based
events) is architected as 64-bit, but some hardware implementations do not
implement the full width. On these implementations the counter stops after
reaching its implemented width. This will limit the usage for short time
monitoring only. The counter will only cover ~15s for monitoring RX TLP
payloads on our platform.
Add an optional hrtimer that fires every 2 seconds. It'll take the role
as the counter overflow interrupt to read-update-reset the counter and
event counts to break the limits of the narrow counters. It'll only
apply on timer-based counter. The 2 seconds update period is the half
of the maximum counting period (4s) of the time-based counter under
period counting mode of the hardware.
Because fully-implemented 64-bit counters do not need this workaround,
enable this hrtimer on the platforms known to have narrowed counter.
Before this patch, when counting fio for 10m the counts is incorrect:
root@localhost:/tmp# perf stat -e dwc_rootport_20000/rx_pcie_tlp_data_payload/ -- fio --runtime=10m fio_job.config
[...]
Run status group 0 (all jobs):
READ: bw=5594MiB/s (5865MB/s), 5594MiB/s-5594MiB/s (5865MB/s-5865MB/s), io=3278GiB (3519GB), run=600010-600010msec
[...]
Performance counter stats for 'system wide':
137,438,953,456 dwc_rootport_20000/rx_pcie_tlp_data_payload/
After this patch the counts is as expected:
root@localhost:/tmp# perf stat -e dwc_rootport_20000/rx_pcie_tlp_data_payload/ -- fio --runtime=10m fio_job.config
[...]
Run status group 0 (all jobs):
READ: bw=5632MiB/s (5905MB/s), 5632MiB/s-5632MiB/s (5905MB/s-5905MB/s), io=3300GiB (3543GB), run=600013-600013msec
[...]
Performance counter stats for 'system wide':
3,543,850,268,576 dwc_rootport_20000/rx_pcie_tlp_data_payload/
Signed-off-by: Yufan Dou <[email protected]>
Signed-off-by: Yicong Yang <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
cn10k_ddr_perf_remove() did not cancel the poll hrtimer before returning.
If the device was unbound while perf events were still active the timer
callback could run post-free.
To fix the issue by adding hrtimer_cancel() in remove().
Signed-off-by: Geetha sowjanya <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
The CN20K DRAM Subsystem exposes eight programmable
performance counters and two fixed counters for DDR
read and write traffic. Software selects events for
the programmable counters from traffic at the DDR PHY
interface, the CHI interconnect, or inside the DDR controller.
Add CN20K register offsets, event maps, and sysfs attributes;
match the device via OF (marvell,cn20k-ddr-pmu) and ACPI (MRVL000B).
Represent the SoC variant in platform data with bit flags so
CN20K can reuse the CN10K PMU code path where appropriate.
Signed-off-by: Geetha sowjanya <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
The current code checks whether the wakeup point is in the current
writable range by comparing it with handle->head + handle->size.
The perf AUX head is a monotonically increasing index, so that addition
can overflow when head is close to ULONG_MAX. In that case, a wakeup
point which is still inside the free space range can be missed.
Use unsigned subtraction to compare the distance from head to wakeup
against the handle->size. This can dismiss the issue when addition
overflow.
This is unlikely to happen in practice, but the change makes the
watermark check logically correct.
Fixes: d5d9696b03 ("drivers/perf: Add support for ARMv8.2 Statistical Profiling Extension")
Signed-off-by: Leo Yan <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
The PMCCNTR_EL0 in NVIDIA Olympus CPU may increment while
in WFI/WFE, which does not align with counting CPU_CYCLES
on a programmable counter. Add a MIDR range entry and refuse
PMCCNTR_EL0 for cycle events on affected parts so perf does
not mix the two behaviors.
Also keep PMCCNTR_EL0 unavailable to EL0 direct counter reads
on affected CPUs. When userspace counter access is enabled,
avoid setting PMUSERENR_EL0.CR for PMUs that must avoid
PMCCNTR_EL0, while still allowing direct reads from programmable
event counters. For 64-bit userspace CPU_CYCLES events on PMUs
without native long event counters, reject the event if the only
valid direct-read path would be PMCCNTR_EL0.
Signed-off-by: Besar Wicaksono <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Graviton5 uses a customised CMN-S3 implementation where certain
discovery registers report zeroed fields. Add the following workarounds:
- Introduce a dedicated ACPI HID to identify the Graviton5 CMN variant.
- Derive the DTC domain from the XP node ID, since the unit info
register reports it as zero.
- Set the DTC logical ID from the XP's logical ID, since the node info
register's logical ID field is also zeroed.
Signed-off-by: Aviv Bakal <[email protected]>
Reviewed-by: Robin Murphy <[email protected]>
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Robin Murphy <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
The amount of data we need to store all the per-DTM counter and
watchpoint allocations is already testing the limits of hw_perf_event,
and future CMNs are only likely to keep growing larger, so move these
arrays out to separate memory allocations. As part of that we can use
an explicit union for allocating cycle counters to dtc_cycles events,
which is arguably nicer anyway.
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Robin Murphy <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
The new DVM node events added in CMN-700 also apply to CMN S3; fix
the model encoding so that we can expose the aliases and handle
occupancy filtering on newer CMNs too.
Cc: [email protected]
Fixes: 0dc2f4963f ("perf/arm-cmn: Support CMN S3")
Signed-off-by: Robin Murphy <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Various names for Qualcomm as a company are used in user-visible config
options: QCOM, Qualcomm and Qualcomm Technologies. Switch to unified
"Qualcomm" so it will be easier for users to identify the options when
for example running menuconfig.
Signed-off-by: Krzysztof Kozlowski <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Pull arm64 updates from Catalin Marinas:
"The biggest changes are MPAM enablement in drivers/resctrl and new PMU
support under drivers/perf.
On the core side, FEAT_LSUI lets futex atomic operations with EL0
permissions, avoiding PAN toggling.
The rest is mostly TLB invalidation refactoring, further generic entry
work, sysreg updates and a few fixes.
Core features:
- Add support for FEAT_LSUI, allowing futex atomic operations without
toggling Privileged Access Never (PAN)
- Further refactor the arm64 exception handling code towards the
generic entry infrastructure
- Optimise __READ_ONCE() with CONFIG_LTO=y and allow alias analysis
through it
Memory management:
- Refactor the arm64 TLB invalidation API and implementation for
better control over barrier placement and level-hinted invalidation
- Enable batched TLB flushes during memory hot-unplug
- Fix rodata=full block mapping support for realm guests (when
BBML2_NOABORT is available)
Perf and PMU:
- Add support for a whole bunch of system PMUs featured in NVIDIA's
Tegra410 SoC (cspmu extensions for the fabric and PCIe, new drivers
for CPU/C2C memory latency PMUs)
- Clean up iomem resource handling in the Arm CMN driver
- Fix signedness handling of AA64DFR0.{PMUVer,PerfMon}
MPAM (Memory Partitioning And Monitoring):
- Add architecture context-switch and hiding of the feature from KVM
- Add interface to allow MPAM to be exposed to user-space using
resctrl
- Add errata workaround for some existing platforms
- Add documentation for using MPAM and what shape of platforms can
use resctrl
Miscellaneous:
- Check DAIF (and PMR, where relevant) at task-switch time
- Skip TFSR_EL1 checks and barriers in synchronous MTE tag check mode
(only relevant to asynchronous or asymmetric tag check modes)
- Remove a duplicate allocation in the kexec code
- Remove redundant save/restore of SCS SP on entry to/from EL0
- Generate the KERNEL_HWCAP_ definitions from the arm64 hwcap
descriptions
- Add kselftest coverage for cmpbr_sigill()
- Update sysreg definitions"
* tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (109 commits)
arm64: rsi: use linear-map alias for realm config buffer
arm64: Kconfig: fix duplicate word in CMDLINE help text
arm64: mte: Skip TFSR_EL1 checks and barriers in synchronous tag check mode
arm64/sysreg: Update ID_AA64SMFR0_EL1 description to DDI0601 2025-12
arm64/sysreg: Update ID_AA64ZFR0_EL1 description to DDI0601 2025-12
arm64/sysreg: Update ID_AA64FPFR0_EL1 description to DDI0601 2025-12
arm64/sysreg: Update ID_AA64ISAR2_EL1 description to DDI0601 2025-12
arm64/sysreg: Update ID_AA64ISAR0_EL1 description to DDI0601 2025-12
arm64/hwcap: Generate the KERNEL_HWCAP_ definitions for the hwcaps
arm64: kexec: Remove duplicate allocation for trans_pgd
ACPI: AGDI: fix missing newline in error message
arm64: Check DAIF (and PMR) at task-switch time
arm64: entry: Use split preemption logic
arm64: entry: Use irqentry_{enter_from,exit_to}_kernel_mode()
arm64: entry: Consistently prefix arm64-specific wrappers
arm64: entry: Don't preempt with SError or Debug masked
entry: Split preemption from irqentry_exit_to_kernel_mode()
entry: Split kernel mode logic from irqentry_{enter,exit}()
entry: Move irqentry_enter() prototype later
entry: Remove local_irq_{enable,disable}_exit_to_user()
...
Update arm_cspmu to use acpi_get_cpu_uid() instead of
get_acpi_id_for_cpu(), aligning with unified ACPI CPU UID interface.
No functional changes are introduced by this switch (valid inputs retain
original behavior).
Signed-off-by: Chengwen Feng <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Rafael J. Wysocki <[email protected]>
When building for 32-bit ARM, there is a warning when using the %llx
specifier to print a resource_size_t variable:
drivers/perf/arm-cmn.c: In function 'arm_cmn_init_dtc':
drivers/perf/arm-cmn.c:2149:73: error: format '%llx' expects argument of type 'long long unsigned int', but argument 4 has type 'resource_size_t' {aka 'unsigned int'} [-Werror=format=]
2149 | "Failed to request DTC region 0x%llx\n", base);
| ~~~^ ~~~~
| | |
| | resource_size_t {aka unsigned int}
| long long unsigned int
| %x
Use the %pa specifier to handle the possible sizes of phys_addr_t
properly. This requires passing the variable by reference.
Fixes: 5394396ff5 ("perf/arm-cmn: Stop claiming entire iomem region")
Signed-off-by: Nathan Chancellor <[email protected]>
Reviewed-by: Robin murphy <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Check devm_ioremap() return value for NULL instead of ERR_PTR and return
-ENOMEM on failure. devm_ioremap() never returns ERR_PTR, using IS_ERR()
skips the error path and may cause a NULL pointer dereference.
Fixes: 5394396ff5 ("perf/arm-cmn: Stop claiming entire iomem region")
Signed-off-by: Chen Ni <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Adds NVIDIA C2C PMU support in Tegra410 SOC. This PMU is
used to measure memory latency between the SOC and device
memory, e.g GPU Memory (GMEM), CXL Memory, or memory on
remote Tegra410 SOC.
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Besar Wicaksono <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Adds CPU Memory (CMEM) Latency PMU support in Tegra410 SOC.
The PMU is used to measure latency between the edge of the
Unified Coherence Fabric to the local system DRAM.
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Besar Wicaksono <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Adds PCIE-TGT PMU support in Tegra410 SOC. This PMU is
instanced in each root complex in the SOC and it captures
traffic originating from any source towards PCIE BAR and CXL
HDM range. The traffic can be filtered based on the
destination root port or target address range.
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Besar Wicaksono <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Adds PCIE PMU support in Tegra410 SOC. This PMU is instanced
in each root complex in the SOC and can capture traffic from
PCIE device to various memory types. This PMU can filter traffic
based on the originating root port or BDF and the target memory
types (CPU DRAM, GPU Memory, CXL Memory, or remote Memory).
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Besar Wicaksono <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Add interface to get ACPI device associated with the
PMU. This ACPI device may contain additional properties
not covered by the standard properties.
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Besar Wicaksono <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
The Unified Coherence Fabric (UCF) contains last level cache
and cache coherent interconnect in Tegra410 SOC. The PMU in
this device can be used to capture events related to access
to the last level cache and memory from different sources.
Reviewed-by: Ilkka Koskinen <[email protected]>
Signed-off-by: Besar Wicaksono <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Linus Torvalds