L2ARC: bound the rebuild by the write hand on a first sweep

l2arc_log_blkptr_valid() ends with (!evicted || dev->l2ad_first), which
disables the eviction-overlap test entirely on a first sweep. That test
is meaningless in that state, since l2arc_evict() returns immediately
and l2ad_evict never advances off l2ad_start, but dropping it leaves the
log block bounded only by device geometry. On a first sweep only the
region below the write hand has been written, so a block at or beyond
l2ad_hand describes data this incarnation never wrote.
l2arc_hdr_restore() performs no per-entry validation, so every such
entry inflates arcstat_l2_psize and, via vdev_space_update(), the cache
vdev's vs_alloc. Nothing reconciles it because l2arc_evict() never runs
on a first sweep, and once vs_alloc exceeds vs_space the unclamped
subtraction in zpool(8) wraps and the device reports 16.0E free. Stale
entries also let the L2ARC read offsets that were never written, which
then fail checksum verification.

Removing and re-adding a cache vdev is enough to set this up:
l2arc_add_vdev() resets l2ad_hand to l2ad_start while the previous
incarnation's log blocks remain higher up the device, and the backward
walk then crosses l2ad_start and wraps into them.

Bound the first-sweep case by the write hand instead of disabling the
check, and state the geometry conditions once rather than duplicating
them across both branches.

The 16.0E symptom has been reported since 2015. #10224 carries the only
prior analysis, which suspected a leak in l2arc_evict() and was closed
incidentally by #9789 rather than by a fix. It is also open on FreeBSD
as PR 250323.

External-issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=250323
Reviewed-by: Ameer Hamza <[email protected]>
Reviewed-by: Alexander Motin <[email protected]>
Signed-off-by: Nick Price <[email protected]>
Closes #3114
Closes #3400
Closes #5583
Closes #10224
Closes #12779
Closes #18827

module/zfs/arc.c

@@ -11580,9 +11580,15 @@ l2arc_log_blkptr_valid(l2arc_dev_t *dev, const l2arc_log_blkptr_t *lbp)
 	    l2arc_range_check_overlap(dev->l2ad_hand, dev->l2ad_evict, start) ||
 	    l2arc_range_check_overlap(dev->l2ad_hand, dev->l2ad_evict, end);
 
-	return (start >= dev->l2ad_start && end <= dev->l2ad_end &&
-	    asize > 0 && asize <= sizeof (l2arc_log_blk_phys_t) &&
-	    (!evicted || dev->l2ad_first));
+	if (asize == 0 || asize > sizeof (l2arc_log_blk_phys_t) ||
+	    start < dev->l2ad_start || end > dev->l2ad_end)
+		return (B_FALSE);
+
+	/* On a first sweep only the region below the write hand was written. */
+	if (dev->l2ad_first)
+		return (end < dev->l2ad_hand);
+
+	return (!evicted);
 }
 
 /*