Fix self-deadlock when cloning a range within the same file

zfs_clone_range() takes an RL_READER range lock on the source range
and an RL_WRITER range lock on the destination range.  When the
source and destination are the same file (inzp == outzp) both locks
are on the same znode's rangelock.  If the destination write has to
grow the file's block size, zfs_rangelock_enter() grows the writer
lock to cover the whole file (see zfs_rlock.c), which then conflicts
with the source reader held by the same thread.  The result is an
unrecoverable self-deadlock (the thread waits forever in
zfs_rangelock_enter_impl(), D state, freed only by a reboot).

Reproduced by fstests generic/564, whose copy_range probe does a
sub-recordsize copy_file_range() within one file that extends it (so
the destination block size grows).

For a same-file clone, take a single writer lock spanning both ranges
instead of a separate reader and writer.  The source block pointers
are read before zfs_clone_range_locked() reduces the lock, so the
source stays covered, and a single lock cannot conflict with itself.
Clones between different files are unchanged.

zfs_dedupe_range() takes two locks on one znode the same way but
cannot hit this, because both ranges are checked to lie within the
file before the locks are taken, so neither lock grows to cover the
whole file.  A later change that lets dedupe extend the file would
need the same fix.

Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Patrick Domack <[email protected]>
Closes #19072

module/zfs/zfs_vnops.c

@@ -2083,7 +2083,23 @@ zfs_clone_range(znode_t *inzp, uint64_t *inoffp, znode_t *outzp,
 	/*
 	 * Maintain predictable lock order.
 	 */
-	if (inzp < outzp || (inzp == outzp && inoff < outoff)) {
+	if (inzp == outzp) {
+		/*
+		 * Within one file, one writer lock spans both
+		 * ranges.  Two locks on the same znode can deadlock
+		 * this thread against itself: a write that grows the
+		 * file's block size grows its lock to cover the
+		 * whole file (see zfs_rlock.c), which then conflicts
+		 * with the other.  The source block pointers are
+		 * read before the lock is reduced, so the source
+		 * stays covered.
+		 */
+		uint64_t lo = MIN(inoff, outoff);
+		uint64_t hi = MAX(inoff + len, outoff + len);
+		outlr = zfs_rangelock_enter(&outzp->z_rangelock, lo,
+		    hi - lo, RL_WRITER);
+		inlr = NULL;
+	} else if (inzp < outzp) {
 		inlr = zfs_rangelock_enter(&inzp->z_rangelock, inoff, len,
 		    RL_READER);
 		outlr = zfs_rangelock_enter(&outzp->z_rangelock, outoff, len,
@@ -2099,7 +2115,8 @@ zfs_clone_range(znode_t *inzp, uint64_t *inoffp, znode_t *outzp,
 	    outlr, B_FALSE, &done);
 
 	zfs_rangelock_exit(outlr);
-	zfs_rangelock_exit(inlr);
+	if (inlr != NULL)
+		zfs_rangelock_exit(inlr);
 
 	if (done > 0) {
 		/*