ABMC, mbm_event mode, has a helper resctrl_arch_config_cntr() for changing
the mapping between 'cntr_id' and a CLOSID/RMID pair.
Add the helper.
For MPAM this is done by updating the mon->mbwu_idx_to_mon[] array, and as
usual CDP means it needs doing in three different ways.
Signed-off-by: James Morse <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Fenghua Yu <[email protected]>
Tested-by: Gavin Shan <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Shaopeng Tan <[email protected]>
Reviewed-by: Fenghua Yu <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
MPAM is able to emulate ABMC, i.e. mbm_event mode, by making memory
bandwidth monitors assignable. Rather than supporting the 'default'
mbm_assign_mode always use 'mbm_event' mode even if there are sufficient
memory bandwidth monitors. The per monitor event configuration is only
provided by resctrl when in 'mbm_event' mode and so only allowing
'mbm_event' mode will make it easier to support per-monitor event
configuration for MPAM. For the moment, the only event supported is
mbm_total_event with no bandwidth type configuration. The 'mbm_assign_mode'
file will still show 'default' when there is no support for memory
bandwidth monitoring.
The monitors need to be allocated from the driver, and mapped to whichever
control/monitor group resctrl wants to use them with.
Add a second array to hold the monitor values indexed by resctrl's cntr_id.
When CDP is in use, two monitors are needed so the available number of
counters halves. Platforms with one monitor will have zero monitors when
CDP is in use.
Co-developed-by: James Morse <[email protected]>
Signed-off-by: James Morse <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Fenghua Yu <[email protected]>
Tested-by: Gavin Shan <[email protected]>
Reviewed-by: Shaopeng Tan <[email protected]>
Reviewed-by: Fenghua Yu <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
resctrl has two types of bandwidth counters, NUMA-local and global. MPAM
can only count globally; either using MSC at the L3 cache or in the memory
controllers. When global and local equate to the same thing continue just
to call it global.
Pick the corresponding MPAM classes to back the MBM counters. As resctrl
requires all monitors to be at the L3 cache, we can only use the counters
at the memory controllers when they have the same topology as the L3 cache
and the traffic they see if the same. In particular, for the bandwidth
counters at the memory controllers to be exposed to resctrl it is required
there is a single L3 cache and a single NUMA node as otherwise cross NUMA
traffic will be counted at the wrong instance.
Signed-off-by: James Morse <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Zeng Heng <[email protected]>
Tested-by: Fenghua Yu <[email protected]>
Tested-by: Gavin Shan <[email protected]>
Reviewed-by: Shaopeng Tan <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Fenghua Yu <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
resctrl_mon_ctx_waiters is not used outside of this file, so make it
static. This fixes the sparse warning:
drivers/resctrl/mpam_resctrl.c:25:1: warning: symbol 'resctrl_mon_ctx_waiters' was not declared. Should it be static?
Reported-by: kernel test robot <[email protected]>
Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
Fixes: 2a3c79c615 ("arm_mpam: resctrl: Allow resctrl to allocate monitors")
Signed-off-by: Ben Horgan <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: James Morse <[email protected]>
Dan Carpenter reports that, in mpam_resctrl_alloc_domain(), any_mon_comp is
used in an 'if' condition when it may be uninitialized. Initialize it to
NULL so that the check behaves correctly when no monitor components are
found.
Reported-by: Dan Carpenter <[email protected]>
Fixes: 264c285999 ("arm_mpam: resctrl: Add monitor initialisation and domain boilerplate")
Signed-off-by: Ben Horgan <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: James Morse <[email protected]>
The code to set MBA's alloc_capable to true appears to be trying to
restore alloc_capable on unmount. This can never work because
resctrl_arch_set_cdp_enabled() is never invoked with RDT_RESOURCE_MBA
as the rid parameter. Consequently,
mpam_resctrl_controls[RDT_RESOURCE_MBA].cdp_enabled always remains false.
The alloc_capable setting in resctrl_arch_set_cdp_enabled() is to
re-enable MBA if the caller opts in to separate control values using
CDP for this resource. This doesn't happen today.
Add a comment to describe this.
However a bug remains where MBA allocation is permanently disabled after
the mount with CDP option. Remounting without CDP cannot restore the MBA
partition capability.
Add a check to re-enable MBA when CDP is disabled, which happens on
unmount.
Fixes: 6789fb9928 ("arm_mpam: resctrl: Add CDP emulation")
Signed-off-by: Zeng Heng <[email protected]>
[ morse: Added comment for existing code, added hunk to fix this bug from
Ben H ]
Reviewed-by: James Morse <[email protected]>
Signed-off-by: James Morse <[email protected]>
A few resctrl features and hooks need to be provided, but aren't needed or
supported on MPAM platforms.
resctrl has individual hooks to separately enable and disable the
closid/partid and rmid/pmg context switching code. For MPAM this is all the
same thing, as the value in struct task_struct is used to cache the value
that should be written to hardware. arm64's context switching code is
enabled once MPAM is usable, but doesn't touch the hardware unless the
value has changed.
For now event configuration is not supported, and can be turned off by
returning 'false' from resctrl_arch_is_evt_configurable().
The new io_alloc feature is not supported either, always return false from
the enable helper to indicate and fail the enable.
Add this, and empty definitions for the other hooks.
Tested-by: Gavin Shan <[email protected]>
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Peter Newman <[email protected]>
Tested-by: Zeng Heng <[email protected]>
Tested-by: Punit Agrawal <[email protected]>
Tested-by: Jesse Chick <[email protected]>
Reviewed-by: Zeng Heng <[email protected]>
Reviewed-by: Shaopeng Tan <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Co-developed-by: Ben Horgan <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Signed-off-by: James Morse <[email protected]>
resctrl uses resctrl_arch_rmid_read() to read counters. CDP emulation means
the counter may need reading in three different ways.
The helpers behind the resctrl_arch_ functions will be re-used for the ABMC
equivalent functions.
Add the rounding helper for checking monitor values while we're here.
Tested-by: Gavin Shan <[email protected]>
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Peter Newman <[email protected]>
Tested-by: Zeng Heng <[email protected]>
Tested-by: Jesse Chick <[email protected]>
Reviewed-by: Shaopeng Tan <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Co-developed-by: Ben Horgan <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Signed-off-by: James Morse <[email protected]>
When resctrl wants to read a domain's 'QOS_L3_OCCUP', it needs to allocate
a monitor on the corresponding resource. Monitors are allocated by class
instead of component.
Add helpers to allocate a CSU monitor. These helper return an out of range
value for MBM counters.
Allocating a montitor context is expected to block until hardware resources
become available. This only makes sense for QOS_L3_OCCUP as unallocated MBM
counters are losing data.
Tested-by: Gavin Shan <[email protected]>
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Peter Newman <[email protected]>
Tested-by: Zeng Heng <[email protected]>
Tested-by: Punit Agrawal <[email protected]>
Tested-by: Jesse Chick <[email protected]>
Reviewed-by: Zeng Heng <[email protected]>
Reviewed-by: Shaopeng Tan <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Co-developed-by: Ben Horgan <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Signed-off-by: James Morse <[email protected]>
resctrl exposes a counter via a file named llc_occupancy. This isn't really
a counter as its value goes up and down, this is a snapshot of the cache
storage usage monitor.
Add some picking code which will only find an L3. The resctrl counter
file is called llc_occupancy but we don't check it is the last one as
it is already identified as L3.
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Zeng Heng <[email protected]>
Tested-by: Punit Agrawal <[email protected]>
Tested-by: Gavin Shan <[email protected]>
Tested-by: Jesse Chick <[email protected]>
Reviewed-by: Zeng Heng <[email protected]>
Reviewed-by: Shaopeng Tan <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Co-developed-by: Dave Martin <[email protected]>
Signed-off-by: Dave Martin <[email protected]>
Co-developed-by: Ben Horgan <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Signed-off-by: James Morse <[email protected]>
Add the boilerplate that tells resctrl about the mpam monitors that are
available. resctrl expects all (non-telemetry) monitors to be on the L3 and
so advertise them there and invent an L3 resctrl resource if required. The
L3 cache itself has to exist as the cache ids are used as the domain
ids.
Bring the resctrl monitor domains online and offline based on the cpus
they contain.
Support for specific monitor types is left to later.
Tested-by: Punit Agrawal <[email protected]>
Reviewed-by: Zeng Heng <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Tested-by: Gavin Shan <[email protected]>
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Jesse Chick <[email protected]>
Signed-off-by: James Morse <[email protected]>
resctrl supports 'MB', as a percentage throttling of traffic from the
L3. This is the control that mba_sc uses, so ideally the class chosen
should be as close as possible to the counters used for mbm_total. If there
is a single L3, it's the last cache, and the topology of the memory matches
then the traffic at the memory controller will be equivalent to that at
egress of the L3. If these conditions are met allow the memory class to
back MB.
MB's percentage control should be backed either with the fixed point
fraction MBW_MAX or bandwidth portion bitmaps. The bandwidth portion
bitmaps is not used as its tricky to pick which bits to use to avoid
contention, and may be possible to expose this as something other than a
percentage in the future.
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Zeng Heng <[email protected]>
Tested-by: Punit Agrawal <[email protected]>
Tested-by: Gavin Shan <[email protected]>
Tested-by: Jesse Chick <[email protected]>
Reviewed-by: Zeng Heng <[email protected]>
Reviewed-by: Shaopeng Tan <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Co-developed-by: Dave Martin <[email protected]>
Signed-off-by: Dave Martin <[email protected]>
Co-developed-by: Ben Horgan <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Signed-off-by: James Morse <[email protected]>
In order to calculate the rmid realloc threshold the size of the cache
needs to be known. Cache domains will also be named after the cache id. So
that this information can be extracted from cacheinfo we need to wait for
it to be ready. The cacheinfo information is populated in device_initcall()
so we wait for that.
Tested-by: Gavin Shan <[email protected]>
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Peter Newman <[email protected]>
Tested-by: Zeng Heng <[email protected]>
Tested-by: Punit Agrawal <[email protected]>
Tested-by: Jesse Chick <[email protected]>
Reviewed-by: Zeng Heng <[email protected]>
Reviewed-by: Shaopeng Tan <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Signed-off-by: James Morse <[email protected]>
When CDP is not enabled, the 'rmid_entry's in the limbo list,
rmid_busy_llc, map directly to a (PARTID,PMG) pair and when CDP is enabled
the mapping is to two different pairs. As the limbo list is reused between
mounts and CDP disabled on unmount this can lead to stale mapping and the
limbo handler will then make monitor reads with potentially out of range
PARTID. This may then cause an MPAM error interrupt and the driver will
disable MPAM.
No problems are expected if you just mount the resctrl file system
once with CDP enabled and never unmount it. Hide CDP emulation behind
CONFIG_EXPERT to protect the unwary.
Signed-off-by: Ben Horgan <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Reviewed-by: Zeng Heng <[email protected]>
Reviewed-by: James Morse <[email protected]>
Tested-by: Gavin Shan <[email protected]>
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Jesse Chick <[email protected]>
Signed-off-by: James Morse <[email protected]>
Intel RDT's CDP feature allows the cache to use a different control value
depending on whether the accesses was for instruction fetch or a data
access. MPAM's equivalent feature is the other way up: the CPU assigns a
different partid label to traffic depending on whether it was instruction
fetch or a data access, which causes the cache to use a different control
value based solely on the partid.
MPAM can emulate CDP, with the side effect that the alternative partid is
seen by all MSC, it can't be enabled per-MSC.
Add the resctrl hooks to turn this on or off. Add the helpers that match a
closid against a task, which need to be aware that the value written to
hardware is not the same as the one resctrl is using.
Update the 'arm64_mpam_global_default' variable the arch code uses during
context switch to know when the per-cpu value should be used instead. Also,
update these per-cpu values and sync the resulting mpam partid/pmg
configuration to hardware.
resctrl can enable CDP for L2 caches, L3 caches or both. When it is enabled
by one and not the other MPAM globally enabled CDP but hides the effect
on the other cache resource. This hiding is possible as CPOR is the only
supported cache control and that uses a resource bitmap; two partids with
the same bitmap act as one.
Awkwardly, the MB controls don't implement CDP and CDP can't be hidden as
the memory bandwidth control is a maximum per partid which can't be
modelled with more partids. If the total maximum is used for both the data
and instruction partids then then the maximum may be exceeded and if it is
split in two then the one using more bandwidth will hit a lower
limit. Hence, hide the MB controls completely if CDP is enabled for any
resource.
Tested-by: Gavin Shan <[email protected]>
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Peter Newman <[email protected]>
Tested-by: Zeng Heng <[email protected]>
Tested-by: Punit Agrawal <[email protected]>
Tested-by: Jesse Chick <[email protected]>
Cc: Dave Martin <[email protected]>
Cc: Amit Singh Tomar <[email protected]>
Reviewed-by: Zeng Heng <[email protected]>
Reviewed-by: Shaopeng Tan <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Co-developed-by: Ben Horgan <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Signed-off-by: James Morse <[email protected]>
resctrl has two helpers for updating the configuration.
resctrl_arch_update_one() updates a single value, and is used by the
software-controller to apply feedback to the bandwidth controls, it has to
be called on one of the CPUs in the resctrl:domain.
resctrl_arch_update_domains() copies multiple staged configurations, it can
be called from anywhere.
Both helpers should update any changes to the underlying hardware.
Implement resctrl_arch_update_domains() to use
resctrl_arch_update_one(). Neither need to be called on a specific CPU as
the mpam driver will send IPIs as needed.
Tested-by: Gavin Shan <[email protected]>
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Peter Newman <[email protected]>
Tested-by: Zeng Heng <[email protected]>
Tested-by: Punit Agrawal <[email protected]>
Tested-by: Jesse Chick <[email protected]>
Reviewed-by: Zeng Heng <[email protected]>
Reviewed-by: Shaopeng Tan <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Co-developed-by: Ben Horgan <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Signed-off-by: James Morse <[email protected]>
Systems with MPAM support may have a variety of control types at any point
of their system layout. We can only expose certain types of control, and
only if they exist at particular locations.
Start with the well-known caches. These have to be depth 2 or 3 and support
MPAM's cache portion bitmap controls, with a number of portions fewer than
resctrl's limit.
Tested-by: Gavin Shan <[email protected]>
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Peter Newman <[email protected]>
Tested-by: Zeng Heng <[email protected]>
Tested-by: Punit Agrawal <[email protected]>
Tested-by: Jesse Chick <[email protected]>
Reviewed-by: Zeng Heng <[email protected]>
Reviewed-by: Shaopeng Tan <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Co-developed-by: Ben Horgan <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Signed-off-by: James Morse <[email protected]>
resctrl has its own data structures to describe its resources. We can't use
these directly as we play tricks with the 'MBA' resource, picking the MPAM
controls or monitors that best apply. We may export the same component as
both L3 and MBA.
Add mpam_resctrl_res[] as the array of class->resctrl mappings we are
exporting, and add the cpuhp hooks that allocated and free the resctrl
domain structures. Only the mpam control feature are considered here and
monitor support will be added later.
While we're here, plumb in a few other obvious things.
CONFIG_ARM_CPU_RESCTRL is used to allow this code to be built even though
it can't yet be linked against resctrl.
Tested-by: Gavin Shan <[email protected]>
Tested-by: Shaopeng Tan <[email protected]>
Tested-by: Peter Newman <[email protected]>
Tested-by: Zeng Heng <[email protected]>
Tested-by: Punit Agrawal <[email protected]>
Tested-by: Jesse Chick <[email protected]>
Reviewed-by: Zeng Heng <[email protected]>
Reviewed-by: Shaopeng Tan <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Co-developed-by: Ben Horgan <[email protected]>
Signed-off-by: Ben Horgan <[email protected]>
Signed-off-by: James Morse <[email protected]>
James Morse