Linux 7.2
There was a lot of conflicts this round between fixes and next,
and I'd like to get the merge resolutions that we have in drm-tip.
Signed-off-by: Dave Airlie <[email protected]>
The q->replay_state is blindly overwritten, which can potentially leak
memory that was previously allocated by vmemdup_user().
Return an error if q->replay_state is not empty.
Discovered using AI-assisted static analysis confirmed by Intel Product
Security.
Reported-by: Martin Hodo <[email protected]>
Fixes: 1026c1a73a ("drm/xe: Implement DRM_XE_EXEC_QUEUE_SET_HANG_REPLAY_STATE")
Reviewed-by: Matthew Brost <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Michał Winiarski <[email protected]>
(cherry picked from commit f6b6cc1118bdbc4265fa8b3bdf8565b26f13e56e)
Signed-off-by: Rodrigo Vivi <[email protected]>
HWDRM is currently the only supported PXP type, and it is display
related, so it cannot be combined with multi-queue exec queue groups.
Reject exec queue creation that requests both multi-queue and PXP,
returning -EINVAL.
The secondary queue path already rejects any PXP property, so this
adds the missing check for the multi-queue primary,
which would otherwise allow the combination.
Validated with igt@xe_exec_multi_queue@sanity, which exercises both
the PXP-unsupported (-ENODEV) and PXP-supported (-EINVAL) paths.
v3:
- Change commit title prefix to drm/xe/multi_queue:.
- Add Niranjana's Reviewed-by.
v2:
- Move the multi-queue + PXP check to exec_queue_user_ext_check() to
bail out early, keyed off the properties bitmask (Niranjana).
Signed-off-by: Jagmeet Randhawa <[email protected]>
Reviewed-by: Niranjana Vishwanathapura <[email protected]>
Signed-off-by: Niranjana Vishwanathapura <[email protected]>
Link: https://patch.msgid.link/4d369249d52384bc93663055a3757a50614ebbfd.1784238312.git.jagmeet.randhawa@intel.com
In a multi-queue group only the group's primary queue interfaces with
GuC for scheduling; suspend/resume of secondary queues is handled
internally and is not forwarded to GuC. As a result, suspending a
secondary queue alone (e.g. on its preempt fence signalling) does not
disable the primary's GuC context, so in-flight GPU work of the group
is not actually preempted.
Make a secondary queue suspend/resume like any other queue, driven by
its own xe_guc_exec_queue.suspend_count, and additionally forward the
suspend/resume to the primary so the GPU is actually preempted. The
forward is gated on the secondary's own 0->1 / 1->0 suspend_count
transition, so each group member contributes exactly one suspend
reference to the primary: the primary keeps its GuC context disabled
until every member that suspended it has resumed, including across the
resume-all-queues-each-rebind-cycle behavior. group->suspend_lock makes
the secondary transition and the primary forward atomic, and a member
leaving while still suspended (queue teardown) drops its reference on
the primary.
v2: Add comment about suspend_wait() in drop_suspend()
v3: Do not suspend a secondary if primary is killed,
wait for primay suspend to complete before drop_suspend()
Assisted-by: Github-Copilot:Claude-opus-4.8
Signed-off-by: Niranjana Vishwanathapura <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Link: https://patch.msgid.link/[email protected]
On newer platforms, the paging configuration is now configured by the PF
via the ADS object, where VF side should ensure that everything
configured as GUC_PAGING_CLASS is correctly mirrored on VF side. For
example PF could in theory reserve two BCS instances, and we expect VF
to mirror that.
With that move towards having a logical mask of all the paging engines,
and also generalise selecting those engines, based on the number of
paging engines. Also cache the first designated paging engine, which
will makes things a little cleaner here, and in later patches.
No functional changes for existing platforms.
v2 (Sashiko):
- Rework the loop slightly so that we don't needlessly check for the
paging engine, before we have correctly set the logical instance.
- Add a proper error return, if we encounter a bogus paging config.
Thinking ahead to VF where the config is defined by the PF, we
should just gracefully exit the probe sequence.
v3:
- Move paging_engines > copy_engines engines check to VF patch.
Signed-off-by: Matthew Auld <[email protected]>
Cc: Daniele Ceraolo Spurio <[email protected]>
Cc: Thomas Hellström <[email protected]>
Cc: Matthew Brost <[email protected]>
Reviewed-by: Francois Dugast <[email protected]>
Link: https://patch.msgid.link/[email protected]
For an LRC belonging to the secondary queue, in order to check if its
context group is active, we need to check the LRC of the primary queue.
In addition to that we want to compare the secondary queue position to
CSMQDEBUG register to check if the queue itself is active.
To do so, store primary LRC and position information in the LRC.
A note on references involved:
- In general the Queue takes a ref on its LRC.
- In addition, for multi-queue,
a. Primary Queue takes a ref for each Secondary LRC.
b. Each Secondary Queue takes a ref to the Primary Queue
In the current patch, each LRC in the queue group is storing a pointer
to primary LRC. Both primary and secondary LRCs are freed only when
primary queue is destroyed. At this time, all secondary queues are
already destroyed, so there is no one using secondary LRCs. We should be
good without taking any additional references.
Signed-off-by: Umesh Nerlige Ramappa <[email protected]>
Reviewed-by: Niranjana Vishwanathapura <[email protected]>
Link: https://patch.msgid.link/[email protected]
xe exec queue code is using a check to see if a class of engines support
multi queue. This check is also needed by other code, so move it to
xe_gt and export it for others.
Signed-off-by: Umesh Nerlige Ramappa <[email protected]>
Reviewed-by: Niranjana Vishwanathapura <[email protected]>
Link: https://patch.msgid.link/[email protected]
Two error handling issues exist in xe_exec_queue_create_ioctl():
1. When xe_hw_engine_group_add_exec_queue() fails, the error path jumps
to put_exec_queue which skips xe_exec_queue_kill(). If the VM is in
preempt fence mode, xe_vm_add_compute_exec_queue() has already added
the queue to the VM's compute exec queue list. Skipping the kill
leaves the queue on that list, leading to a dangling pointer after
the queue is freed.
2. When xa_alloc() fails after xe_hw_engine_group_add_exec_queue() has
succeeded, the error path does not call
xe_hw_engine_group_del_exec_queue() to remove the queue from the hw
engine group list. The queue is then freed while still linked into
the hw engine group, causing a use-after-free.
Fix both by:
- Changing the xe_hw_engine_group_add_exec_queue() failure path to jump
to kill_exec_queue so that xe_exec_queue_kill() properly removes the
queue from the VM's compute list.
- Adding a del_hw_engine_group label before kill_exec_queue for the
xa_alloc() failure path, which removes the queue from the hw engine
group before proceeding with the rest of the cleanup.
Fixes: 7970cb3696 ("'drm/xe/hw_engine_group: Register hw engine group's exec queues")
Cc: Francois Dugast <[email protected]>
Cc: Matthew Brost <[email protected]>
Cc: Niranjana Vishwanathapura <[email protected]>
Assisted-by: Claude:claude-opus-4.6
Reviewed-by: Matthew Brost <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Shuicheng Lin <[email protected]>
xe_exec_queue_tlb_inval_last_fence_put_unlocked() uses q->vm->xe as the
first argument to xe_assert(). This function is called unconditionally
from xe_exec_queue_destroy() for all queues, including kernel queues
that have q->vm == NULL (e.g., queues created during GT init in
xe_gt_record_default_lrcs() with vm=NULL).
While current compilers optimize away the q->vm->xe dereference (even
in CONFIG_DRM_XE_DEBUG=y builds, the compiler pushes the dereference
into the WARN branch that is only taken when the assert condition is
false), the code is semantically incorrect and constitutes undefined
behavior in the C abstract machine for the NULL pointer case.
Use gt_to_xe(q->gt) instead, which is always valid for any exec queue.
This is consistent with how xe_exec_queue_destroy() itself obtains the
xe_device pointer in its own xe_assert at the top of the function.
Fixes: b2d7ec41f2 ("drm/xe: Attach last fence to TLB invalidation job queues")
Assisted-by: Claude:claude-opus-4.6
Reviewed-by: Matthew Brost <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Shuicheng Lin <[email protected]>
Similar to i915's commit cebc13de7e
("drm/i915: Whitelist COMMON_SLICE_CHICKEN3 for UMD access"), except
that instead of putting the register on the allowlist for UMD to
program, the KMD is doing the programming at context initialization
based on a queue creation flag.
This is a recommended tuning setting for both gen12 and Xe_HP
platforms.
If a render queue is created with
DRM_XE_EXEC_QUEUE_SET_STATE_CACHE_PERF_FIX, COMMON_SLICE_CHICKEN3 will
be programmed at initialization to enable the render color cache to
key with BTP+BTI (binding table pool + binding table entry) instead of
just BTI (binding table entry). This enables the UMD to avoid emitting
render-target-cache-flush + stall-at-pixel-scoreboard every time a
binding table entry pointing to a render target is changed.
v2: Use xe_lrc_write_ring()
v3: Update xe_query.c to report availability
v4: Rename defines to add DISABLE_
v5: update commit message
v6: rebase
Mesa MR: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39982
Bspec: 73993, 73994, 72161, 31870, 68331
Acked-by: Rodrigo Vivi <[email protected]>
Reviewed-by: José Roberto de Souza <[email protected]>
Signed-off-by: Lionel Landwerlin <[email protected]>
Signed-off-by: José Roberto de Souza <[email protected]>
Link: https://patch.msgid.link/[email protected]
If the xe module within a VM was creating a new LRC during save/
restore, this LRC will be invalid. The fixups procedure may not
be able to reach it, as there will be a race to add the new LRC
reference to an exec queue.
Even if the new LRC which was being created during VM migration is
added to EQ in time for fixups, said LRC may still remain damaged.
In a small percentage of specially crafted test cases, the resulting
LRC was still damaged and caused GPU hang.
Any LRC which could be created in such a situation, have to be
re-created.
Due to VM having arbitrarily set amount of CPU cores, it is possible
to limit the amount to 1. In such case, there is a possibility that
kernel will switch CPU contexts in a way which allows to miss
VF migration recovery running in parallel (by simply not switching
to the LRC creation thread during recovery). Therefore checking
if the migration is in progress just after LRC creation, is not
enough to ensure detection.
Free the incorrectly created LRC, and trigger a re-run of the
creation, but only after waiting for default LRC to get fixups.
Use additional atomic value increased after fixups, to ensure any VF
migration that avoided detection by just checking for recovery in
progress, will be caught.
v2: Merge marker and wait for default LRC, reducing amount of calls
within xe_init_eq(). Alter the LRC creation loop to remove a race
with post-migration fixups worker.
v3: Kerneldoc fixes. Rename fixups_complete_count.
Signed-off-by: Tomasz Lis <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Signed-off-by: Michal Wajdeczko <[email protected]>
Link: https://patch.msgid.link/[email protected]
There is a small but non-zero chance that VF post migration fixups
are running on an exec queue during teardown. The chances are
decreased by starting the teardown by releasing guc_id, but remain
non-zero. On the other hand the sync between fixups and EQ creation
(wait_valid_ggtt) drastically increases the chance for such parallel
teardown if queue creation error path is entered (err_lrc label).
The exec queue itself is not going to cause an issue, but LRCs have
a small chance of getting freed during the fixups.
Creating a setter and a getter makes it easier to protect the fixup
operations with a lock. For other driver activities, the original
access method (without any protection) can still be used.
v2: Separate lock, only for LRCs. Kerneldoc fixes. Subject tag fix.
Signed-off-by: Tomasz Lis <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Signed-off-by: Michal Wajdeczko <[email protected]>
Link: https://patch.msgid.link/[email protected]
Every call to queue init should have a corresponding fini call.
Skipping this would mean skipping removal of the queue from GuC list
(which is part of guc_id allocation). A damaged queue stored in
exec_queue_lookup list would lead to invalid memory reference,
sooner or later.
Call fini to free guc_id. This must be done before any internal
LRCs are freed.
Since the finalization with this extra call became very similar to
__xe_exec_queue_fini(), reuse that. To make this reuse possible,
alter xe_lrc_put() so it can survive NULL parameters, like other
similar functions.
v2: Reuse _xe_exec_queue_fini(). Make xe_lrc_put() aware of NULLs.
Fixes: 3c1fa4aa60 ("drm/xe: Move queue init before LRC creation")
Signed-off-by: Tomasz Lis <[email protected]>
Reviewed-by: Matthew Brost <[email protected]> (v1)
Signed-off-by: Michal Wajdeczko <[email protected]>
Link: https://patch.msgid.link/[email protected]
Tighten uapi validation to restrict multi-lrc support to VIDEO_DECODE and
VIDEO_ENHANCE engines only. This check should have been in place from the
start, as the driver typically avoids allowing uapi cases that we have
no userspace consumer for.
Additionally, the GuC firmware on ModSched platforms no longer supports
multi-lrc on non-media engines.
V4:
- use a unified mask for all platforms since engine instance count
is an independent runtime check (Matt Roper, Matthew Brost)
V3:
- store a multi-lrc enable class mask in xe->info and populate from
xe_device_desc in xe_pci.c (Matthew Brost)
V2:
- correct the typo (Shuicheng)
- move the check earlier to avoid VM lookup (Shuicheng, Matt Roper)
- remove the graphics version check (Matt Roper)
- input more details in the commit info (Matt Roper)
Cc: Shuicheng Lin <[email protected]>
Cc: Matt Roper <[email protected]>
Cc: Matthew Brost <[email protected]>
Signed-off-by: Xin Wang <[email protected]>
Reviewed-by: Shuicheng Lin <[email protected]>
Reviewed-by: Matt Roper <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Matt Roper <[email protected]>
This is the exact same thing as the 'alloc_obj()' version, only much
smaller because there are a lot fewer users of the *alloc_flex()
interface.
As with alloc_obj() version, this was done entirely with mindless brute
force, using the same script, except using 'flex' in the pattern rather
than 'objs*'.
Signed-off-by: Linus Torvalds <[email protected]>
This was done entirely with mindless brute force, using
git grep -l '\<k[vmz]*alloc_objs*(.*, GFP_KERNEL)' |
xargs sed -i 's/\(alloc_objs*(.*\), GFP_KERNEL)/\1)/'
to convert the new alloc_obj() users that had a simple GFP_KERNEL
argument to just drop that argument.
Note that due to the extreme simplicity of the scripting, any slightly
more complex cases spread over multiple lines would not be triggered:
they definitely exist, but this covers the vast bulk of the cases, and
the resulting diff is also then easier to check automatically.
For the same reason the 'flex' versions will be done as a separate
conversion.
Signed-off-by: Linus Torvalds <[email protected]>
This is the result of running the Coccinelle script from
scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to
avoid scalar types (which need careful case-by-case checking), and
instead replace kmalloc-family calls that allocate struct or union
object instances:
Single allocations: kmalloc(sizeof(TYPE), ...)
are replaced with: kmalloc_obj(TYPE, ...)
Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...)
are replaced with: kmalloc_objs(TYPE, COUNT, ...)
Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...)
are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...)
(where TYPE may also be *VAR)
The resulting allocations no longer return "void *", instead returning
"TYPE *".
Signed-off-by: Kees Cook <[email protected]>
Use a spinlock to protect multi-queue priority being concurrently
updated by multiple set_priority ioctls and to protect against
concurrent read and write to this field.
v2: Update documentation, remove WRITE/READ_LOCK() (Thomas)
Use scoped_guard, reduced lock scope (Matt Brost)
v3: Fix author (checkpatch)
Signed-off-by: Niranjana Vishwanathapura <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Reviewed-by: Thomas Hellström <[email protected]>
Link: https://patch.msgid.link/[email protected]
Currently this is very broken if someone attempts to create a bind
queue and share it across multiple VMs. For example currently we assume
it is safe to acquire the user VM lock to protect some of the bind queue
state, but if allow sharing the bind queue with multiple VMs then this
quickly breaks down.
To fix this reject using a bind queue with any VM that is not the same
VM that was originally passed when creating the bind queue. This a uAPI
change, however this was more of an oversight on kernel side that we
didn't reject this, and expectation is that userspace shouldn't be using
bind queues in this way, so in theory this change should go unnoticed.
Based on a patch from Matt Brost.
v2 (Matt B):
- Hold the vm lock over queue create, to ensure it can't be closed as
we attach the user_vm to the queue.
- Make sure we actually check for NULL user_vm in destruction path.
v3:
- Fix error path handling.
Fixes: dd08ebf6c3 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Reported-by: Thomas Hellström <[email protected]>
Signed-off-by: Matthew Auld <[email protected]>
Cc: José Roberto de Souza <[email protected]>
Cc: Matthew Brost <[email protected]>
Cc: Michal Mrozek <[email protected]>
Cc: Carl Zhang <[email protected]>
Cc: <[email protected]> # v6.8+
Acked-by: José Roberto de Souza <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Reviewed-by: Arvind Yadav <[email protected]>
Acked-by: Michal Mrozek <[email protected]>
Link: https://patch.msgid.link/[email protected]
(cherry picked from commit 9dd08fdecc)
Signed-off-by: Thomas Hellström <[email protected]>
Currently this is very broken if someone attempts to create a bind
queue and share it across multiple VMs. For example currently we assume
it is safe to acquire the user VM lock to protect some of the bind queue
state, but if allow sharing the bind queue with multiple VMs then this
quickly breaks down.
To fix this reject using a bind queue with any VM that is not the same
VM that was originally passed when creating the bind queue. This a uAPI
change, however this was more of an oversight on kernel side that we
didn't reject this, and expectation is that userspace shouldn't be using
bind queues in this way, so in theory this change should go unnoticed.
Based on a patch from Matt Brost.
v2 (Matt B):
- Hold the vm lock over queue create, to ensure it can't be closed as
we attach the user_vm to the queue.
- Make sure we actually check for NULL user_vm in destruction path.
v3:
- Fix error path handling.
Fixes: dd08ebf6c3 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Reported-by: Thomas Hellström <[email protected]>
Signed-off-by: Matthew Auld <[email protected]>
Cc: José Roberto de Souza <[email protected]>
Cc: Matthew Brost <[email protected]>
Cc: Michal Mrozek <[email protected]>
Cc: Carl Zhang <[email protected]>
Cc: <[email protected]> # v6.8+
Acked-by: José Roberto de Souza <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Reviewed-by: Arvind Yadav <[email protected]>
Acked-by: Michal Mrozek <[email protected]>
Link: https://patch.msgid.link/[email protected]
Maintain a list of exec queues per vm which will be used by TLB
invalidation code to do context-ID based tlb invalidations.
v4:
- More asserts (Stuart)
- Per GT list (CI)
- Skip adding / removal if context TLB invalidatiions not supported
(Stuart)
Signed-off-by: Nirmoy Das <[email protected]>
Signed-off-by: Matthew Brost <[email protected]>
Reviewed-by: Stuart Summers <[email protected]>
Tested-by: Stuart Summers <[email protected]>
Link: https://patch.msgid.link/[email protected]
clangd reports many "unused header" warnings throughout the Xe driver.
Start working to clean this up by removing unnecessary includes in our
.c files and/or replacing them with explicit includes of other headers
that were previously being included indirectly.
By far the most common offender here was unnecessary inclusion of
xe_gt.h. That likely originates from the early days of xe.ko when
xe_mmio did not exist and all register accesses, including those
unrelated to GTs, were done with GT functions.
There's still a lot of additional #include cleanup that can be done in
the headers themselves; that will come as a followup series.
v2:
- Squash the 79-patch series down to a single patch. (MattB)
Reviewed-by: Matthew Brost <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Matt Roper <[email protected]>
Since engines in the same class can be divided across multiple groups,
the GuC does not allow scheduler groups to be active if there are
multi-lrc contexts. This means that:
1) if a MLRC context is registered when we enable scheduler groups, the
GuC will silently ignore the configuration
2) if a MLRC context is registered after scheduler groups are enabled,
the GuC will disable the groups and generate an adverse event.
The expectation is that the admin will ensure that all apps that use
MLRC on PF have been terminated before scheduler groups are created. A
check is added anyway to make sure we don't still have contexts waiting
to be cleaned up laying around. A check is also added at queue creation
time to block MLRC queue creation if scheduler groups have been enabled.
Signed-off-by: Daniele Ceraolo Spurio <[email protected]>
Cc: Michal Wajdeczko <[email protected]>
Reviewed-by: Michal Wajdeczko <[email protected]>
Link: https://patch.msgid.link/[email protected]
Add support to keep the group active after the primary queue is
destroyed. Instead of killing the primary queue during exec_queue
destroy ioctl, kill it when all the secondary queues of the group
are killed.
Signed-off-by: Niranjana Vishwanathapura <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Link: https://patch.msgid.link/[email protected]
As all queues of a multi queue group use the primary queue of the group
to interface with GuC. Hence there is a dependency between the queues of
the group. So, when primary queue of a multi queue group is cleaned up,
also trigger a cleanup of the secondary queues also. During cleanup, stop
and re-start submission for all queues of a multi queue group to avoid
any submission happening in parallel when a queue is being cleaned up.
v2: Initialize group->list_lock, add fs_reclaim dependency, remove
unwanted secondary queues cleanup (Matt Brost)
v3: Properly handle cleanup of multi-queue group (Matt Brost)
v4: Fix IS_ENABLED(CONFIG_LOCKDEP) check (Matt Brost)
Revert stopping/restarting of submissions on queues of the
group in TDR as it is not needed.
Signed-off-by: Niranjana Vishwanathapura <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Link: https://patch.msgid.link/[email protected]
Support dynamic priority change for multi queue group queues via
exec queue set_property ioctl. Issue CGP_SYNC command to GuC through
the drm scheduler message interface for priority to take effect.
v2: Move is_multi_queue check to exec_queue layer and assert
is_multi_queue being set in guc submission layer (Matt Brost)
v3: Assert CGP_SYNC message length is valid (Matt Brost)
Signed-off-by: Pallavi Mishra <[email protected]>
Signed-off-by: Niranjana Vishwanathapura <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Link: https://patch.msgid.link/[email protected]
Only MULTI_QUEUE_PRIORITY property is valid for secondary queues of a
multi queue group. MULTI_QUEUE_PRIORITY only applies to multi queue group
queues. Detect invalid user queue property setting and return error.
Signed-off-by: Niranjana Vishwanathapura <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Link: https://patch.msgid.link/[email protected]
Add support for queues of a multi queue group to set
their priority within the queue group by adding property
DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_QUEUE_PRIORITY.
This is the only other property supported by secondary
queues of a multi queue group, other than
DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_QUEUE.
v2: Add kernel doc for enum xe_multi_queue_priority,
Add assert for priority values, fix includes and
declarations (Matt Brost)
v3: update uapi kernel-doc (Matt Brost)
v4: uapi change due to rebase
Signed-off-by: Niranjana Vishwanathapura <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Link: https://patch.msgid.link/[email protected]
Multi Queue is a new mode of execution supported by the compute and
blitter copy command streamers (CCS and BCS, respectively). It is an
enhancement of the existing hardware architecture and leverages the
same submission model. It enables support for efficient, parallel
execution of multiple queues within a single context. All the queues
of a group must use the same address space (VM).
The new DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_QUEUE execution queue
property supports creating a multi queue group and adding queues to
a queue group. All queues of a multi queue group share the same
context.
A exec queue create ioctl call with above property specified with value
DRM_XE_SUPER_GROUP_CREATE will create a new multi queue group with the
queue being created as the primary queue (aka q0) of the group. To add
secondary queues to the group, they need to be created with the above
property with id of the primary queue as the value. The properties of
the primary queue (like priority, timeslice) applies to the whole group.
So, these properties can't be set for secondary queues of a group.
Once destroyed, the secondary queues of a multi queue group can't be
replaced. However, they can be dynamically added to the group up to a
total of 64 queues per group. Once the primary queue is destroyed,
secondary queues can't be added to the queue group.
v2: Remove group->lock, fix xe_exec_queue_group_add()/delete()
function semantics, add additional comments, remove unused
group->list_lock, add XE_BO_FLAG_GGTT_INVALIDATE for cgp bo,
Assert LRC is valid, update uapi kernel doc.
(Matt Brost)
v3: Use XE_BO_FLAG_PINNED_LATE_RESTORE/USER_VRAM/GGTT_INVALIDATE
flags for cgp bo (Matt)
v4: Ensure queue is not a vm_bind queue
uapi change due to rebase
Signed-off-by: Stuart Summers <[email protected]>
Signed-off-by: Niranjana Vishwanathapura <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Link: https://patch.msgid.link/[email protected]
Backmerging to bring in a needed dependency for the Xe VFIO
driver variant. This should ideally have been done before we
commited that, so we now have a small window in drm-xe-next
where that driver doesn't compile.
Reported-by: kernel test robot <[email protected]>
Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
Signed-off-by: Thomas Hellström <[email protected]>
Implement DRM_XE_EXEC_QUEUE_SET_HANG_REPLAY_STATE which sets the exec
queue default state to user data passed in. The intent is for a Mesa
tool to use this replay GPU hangs.
v2:
- Enable the flag DRM_XE_EXEC_QUEUE_SET_HANG_REPLAY_STATE
- Fix the page size math calculation to avoid a crash
v4:
- Use vmemdup_user (Maarten)
- Copy default state first into LRC, then replay state (Testing, Carlos)
Cc: José Roberto de Souza <[email protected]>
Signed-off-by: Matthew Brost <[email protected]>
Reviewed-by: Maarten Lankhorst <[email protected]>
Reviewed-by: Jonathan Cavitt <[email protected]>
Link: https://patch.msgid.link/[email protected]
Prevent application hangs caused by out-of-order fence signaling when
user fences are attached. Use drm_syncobj (via dma-fence-chain) to
guarantee that each user fence signals in order, regardless of the
signaling order of the attached fences. Ensure user fence writebacks to
user space occur in the correct sequence.
v7:
- Skip drm_syncbj create of error (CI)
Fixes: dd08ebf6c3 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Signed-off-by: Matthew Brost <[email protected]>
Reviewed-by: Thomas Hellström <[email protected]>
Link: https://patch.msgid.link/[email protected]
(cherry picked from commit adda4e855a)
Signed-off-by: Lucas De Marchi <[email protected]>
Eliminate redundant last fence dependency checks in exec and bind jobs,
as they are now equivalent to xe_exec_queue_is_idle. Simplify the code
by removing this dead logic.
Signed-off-by: Matthew Brost <[email protected]>
Reviewed-by: Thomas Hellström <[email protected]>
Link: https://patch.msgid.link/[email protected]
Add support for attaching the last fence to TLB invalidation job queues
to address serialization issues during bursts of unbind jobs. Ensure
that user fence signaling for a bind job reflects both the bind job
itself and the last fences of all related TLB invalidations. Maintain
submission order based solely on the state of the bind and TLB
invalidation queues.
Introduce support functions for last fence attachment to TLB
invalidation queues.
v3:
- Fix assert in xe_exec_queue_tlb_inval_last_fence_set (CI)
- Ensure migrate lock held for migrate queues (Testing)
v5:
- Style nits (Thomas)
- Rewrite commit message (Thomas)
Signed-off-by: Matthew Brost <[email protected]>
Reviewed-by: Thomas Hellström <[email protected]>
Link: https://patch.msgid.link/[email protected]
Prevent application hangs caused by out-of-order fence signaling when
user fences are attached. Use drm_syncobj (via dma-fence-chain) to
guarantee that each user fence signals in order, regardless of the
signaling order of the attached fences. Ensure user fence writebacks to
user space occur in the correct sequence.
v7:
- Skip drm_syncbj create of error (CI)
Fixes: dd08ebf6c3 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Signed-off-by: Matthew Brost <[email protected]>
Reviewed-by: Thomas Hellström <[email protected]>
Link: https://patch.msgid.link/[email protected]
Add a limit to the number of jobs that can be queued in a single
exec queue to avoid potential resource exhaustion.
A new field `job_cnt` is introduced in `struct xe_exec_queue` to
track the number of active DRM jobs, along with a maximum limit
`XE_MAX_JOB_COUNT_PER_EXEC_QUEUE` set to 1000.
If the job count exceeds this threshold, `xe_exec_ioctl()` now
returns `-EAGAIN` to signal that the caller should retry later.
A trace event is added to track when the limit is reached:
"xe_exec_queue_reach_max_job_count: dev=0000:03:00.0, job count
exceeded the maximum limit (1000) per exec queue. engine_class=0x3,
logical_mask=0x1, guc_id=2"
v3: add assert in xe_exec_queue_destroy that q->job_cnt is zero. (Matt)
v2 (Matt):
- add log to trace the limit is hit.
- Change max count from 0x1000 to 1000.
- Use atomic_t for job_cnt.
Suggested-by: Matthew Brost <[email protected]>
Signed-off-by: Shuicheng Lin <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Signed-off-by: Matthew Brost <[email protected]>
Link: https://patch.msgid.link/[email protected]
A queue must be in the submission backend's tracking state before the
LRC is created to avoid a race condition where the LRC's GGTT addresses
are not properly fixed up during VF post-migration recovery.
Move the queue initialization—which adds the queue to the submission
backend's tracking state—before LRC creation.
Also wait on pending GGTT fixups before allocating LRCs to avoid racing
with fixups.
v2:
- Wait on VF GGTT fixes before creating LRC (testing)
v5:
- Adjust comment in code (Tomasz)
- Reduce race window
v7:
- Only wakeup waiters in recovery path (CI)
- Wakeup waiters on abort
- Use GT warn on (Michal)
- Fix kernel doc for LRC ring size function (Tomasz)
v8:
- Guard against migration not supported or no memirq (CI)
Signed-off-by: Matthew Brost <[email protected]>
Reviewed-by: Tomasz Lis <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Dave Airlie