sched/fair: Avoid overflow in place_entity()

place_entity() inflates an entity's virtual lag with

    lag * (load + weight) / load

Commit 4823725d9d ("sched/fair: Increase weight bits for avg_vruntime")
replaced the scaled-down weights used in this calculation with
avg_vruntime_weight(), removing the arithmetic headroom provided by
scale_load_down().

The multiplication can overflow s64 before the division even when the
final quotient is representable. A task's hierarchical effective weight
can be as small as 2 after avg_vruntime_weight() scaling under group
scheduling, while entity_lag() bounds |vlag| by

    calc_delta_fair(cfs_rq_max_slice(cfs_rq) + TICK_NSEC, se)

For example, with HZ=1000, NICE_0_LOAD=1048576, and the minimum scaled
hierarchical weight scale_load(2)=2048, this bound is 891289600. With 114
maximum-weight entities, the runqueue load is 10361604096. The
intermediate product is:

    891289600 * (10361604096 + 2)
        = 9235189971864780800
        > S64_MAX

The signed overflow corrupts the entity's placement.

Rewrite

    lag * (load + weight) / load

as

    lag + lag * weight / load

The two expressions are equivalent, but the latter avoids multiplying the
virtual lag by the total runqueue weight. entity_lag() bounds vlag against
the entity's effective weight, so the remaining lag * weight product stays
bounded. rescale_entity() preserves this scale across h_load changes and
already relies on the same product when rescaling vlag.

Keep the original zero-load warning semantics by multiplying lag by weight
after replacing the divisor with one.

Fixes: 4823725d9d ("sched/fair: Increase weight bits for avg_vruntime")
Signed-off-by: Hui Su <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Link: https://patch.msgid.link/[email protected]
This commit is contained in:
Hui Su authored and Peter Zijlstra committed 2026-10-16 09:49:00 +02:00
1 parent 118a2cf4ce
commit 2f93bcb3f9
1 file changed
+12 -4
+12 -4
View File
@@ -6463,12 +6463,20 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
load = cfs_rq->sum_weight;
if (curr && curr->on_rq)
load += avg_vruntime_weight(cfs_rq, curr->h_load.weight);
weight = avg_vruntime_weight(cfs_rq, se->h_load.weight);
lag *= load + weight;
if (WARN_ON_ONCE(!load))
load = 1;
lag = div64_long(lag, load);
weight = avg_vruntime_weight(cfs_rq, se->h_load.weight);
/*
* Rewrite, because the product (W + w_i)*vl'_i might
* overflow s64:
*
* vl_i = (W + w_i)*vl'_i / W
* = (W*vl'_i + w_i*vl'_i) / W
* = vl'_i + w_i*vl'_i/W
*/
lag += div64_long(lag * weight, load);
/*
* A heavy entity (relative to the tree) will pull the