zstream: refactor common functions

### Motivation

In the current version of `zstream`, each subcommand is independent and
is responsible for implementing its own stream-processing pipeline. It
started as a stream dumper, but as additional subcommands were added,
contributors typically copied an existing subcommand's pipeline and
adapted it for different purposes.

This pattern has led to quite a bit of duplicated code and has also led
to some functional nonuniformities. For example, some subcommands
support opposite-endian streams and others don't.

### Overview

This PR segregates functions that most subcommands need into
free-standing modules and reimplements the existing subcommands in
terms of those modules. The current modules are:

* I/O
* Checksum validation and generation
* Byte-order detection and byteswapping
* Data compression
* Data decompression
* General validity checking

This PR also adds a generic pipeline mechanism that subcommands can use
to declare the processing they want. For example, the pipeline for
`zstream recompress` is:

```
	zstream_chain_t recompress_chain = {
		serial_read_stream(infile),
		serial_validate_fletcher4(),
		serial_byteswap(BS_INCOMING),
		serial_validate_records(),
		serial_decompress_writes(&spec),
		serial_compress_writes(&spec),
		serial_byteswap(BS_OUTGOING),
		serial_add_fletcher4(),
		serial_write_stream(outfile),
		chain_terminator()
	};
```

Or more succinctly:

```
	zstream_chain_t recompress_chain = {
		STANDARD_INPUT_STACK(infile),
		serial_decompress_writes(&spec),
		serial_compress_writes(&spec),
		STANDARD_OUTPUT_STACK(outfile)
	};
```

To execute the pipeline:

```
	chain_attrs_t attrs = { .ca_command_opts = CA_FORBID_DEDUP };
	zstream_chain_exec(recompress_chain, &attrs);
```

### Explanation of `serial_` prefixes

The `serial_` prefixes above indicate that the steps run sequentially
and that each module sees records in their original order. A future PR
will also allow multithreaded execution for individual steps and will
handle the marshaling involved in integrating serial and parallel
operations.

### What this is not

This PR is not a general cleanup. Original code that is not subsumed by
one of the standardized modules is left largely unchanged, although
in some cases a few unavoidable modifications have been made to adapt
the prior code to the pipeline context.

This PR does not change command syntax or behavior in any way except
to the extent that standard modules may fix bugs or allow additional
cases that formerly failed.

### Comprehensive test suite

This PR defines a `zstream` test category and adds tests to exercise all
features of the `zstream` command.

My original intent was to submit the PR for this test suite first and
independently of the modular refactoring, with an eye toward verifying
that the later restructuring did not change existing behavior. However,
I did find occasional issues in the existing code as I worked on the
refactoring and added tests for these issues to the test suite.
Ultimately, the tests ended up flagging enough issues in the existing
code that they are no longer very useful as an "everything behaves the
same" check, so I'm just including them here. They are still a
comprehensive workout for all of `zstream`'s functions, however.

### Effect on checksum generation

This PR incorporates changes equivalent to those in draft PR #18293,
which I submitted but later changed to draft status because I knew
they'd have to be fixed in a different way for this PR.

Draft PR #18293 includes a detailed explanation, but the TL;DR is:
`zstream` and `zfs send` formerly diverged somewhat in their checksum
generation patterns. `zstream` added checksums in some cases where `zfs
send` did not. It didn't matter functionally because `zfs receive` does
not attempt to validate those checksums anyway. However, the result was
that even null operations (such as `zstream redup`'ing a stream that
was not deduplicated) changed the contents of the stream.

The Fletcher4 component of this PR emulates the behavior of `zfs send`
more closely so that any no-op invocation of `zstream` leaves the
stream bit-for-bit identical to the input.

Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: Tony Hutter <[email protected]>
Signed-off-by: Garth Snyder <[email protected]>
Closes #18509

cmd/zstream/Makefile.am

@@ -7,14 +7,26 @@ CPPCHECKTARGETS += zstream
 zstream_SOURCES = \
 	%D%/zstream.c \
 	%D%/zstream.h \
+	%D%/zstream_byteswap.c \
+	%D%/zstream_byteswap.h \
+	%D%/zstream_chain.c \
+	%D%/zstream_chain.h \
 	%D%/zstream_decompress.c \
 	%D%/zstream_drop_record.c \
 	%D%/zstream_dump.c \
+	%D%/zstream_fletcher4.c \
+	%D%/zstream_fletcher4.h \
+	%D%/zstream_io.c \
+	%D%/zstream_io.h \
+	%D%/zstream_modules.h \
 	%D%/zstream_recompress.c \
+	%D%/zstream_recompress.h \
 	%D%/zstream_redup.c \
 	%D%/zstream_token.c \
 	%D%/zstream_util.c \
-	%D%/zstream_util.h
+	%D%/zstream_util.h \
+	%D%/zstream_validate.c \
+	%D%/zstream_validate.h
 
 zstream_LDADD = \
 	libzfs.la \

cmd/zstream/zstream.c

@@ -18,19 +18,11 @@
  * Copyright (c) 2020 by Delphix. All rights reserved.
  * Copyright (c) 2020 by Datto Inc. All rights reserved.
  */
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <ctype.h>
+
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
-#include <unistd.h>
-#include <libintl.h>
-#include <stddef.h>
-#include <libzfs.h>
-#include <signal.h>
-#include <sys/backtrace.h>
+
 #include "zstream.h"
 
 void
@@ -55,43 +47,9 @@ zstream_usage(void)
 	exit(1);
 }
 
-static void sig_handler(int signo)
-{
-	struct sigaction action;
-	libspl_backtrace(STDERR_FILENO);
-
-	/*
-	 * Restore default action and re-raise signal so SIGSEGV and
-	 * SIGABRT can trigger a core dump.
-	 */
-	action.sa_handler = SIG_DFL;
-	sigemptyset(&action.sa_mask);
-	action.sa_flags = 0;
-	(void) sigaction(signo, &action, NULL);
-	raise(signo);
-}
-
-
 int
 main(int argc, char *argv[])
 {
-	/*
-	 * Set up signal handlers, so if we crash due to bad data in the stream
-	 * we can get more info. Unlike ztest, we don't bail out if we can't
-	 * set up signal handlers, because zstream is very useful without them.
-	 */
-	struct sigaction action = { .sa_handler = sig_handler };
-	sigemptyset(&action.sa_mask);
-	action.sa_flags = 0;
-	if (sigaction(SIGSEGV, &action, NULL) < 0) {
-		(void) fprintf(stderr, "zstream: cannot catch SIGSEGV: %s\n",
-		    strerror(errno));
-	}
-	if (sigaction(SIGABRT, &action, NULL) < 0) {
-		(void) fprintf(stderr, "zstream: cannot catch SIGABRT: %s\n",
-		    strerror(errno));
-	}
-
 	char *basename = strrchr(argv[0], '/');
 	basename = basename ? (basename + 1) : argv[0];
 	if (argc >= 1 && strcmp(basename, "zstreamdump") == 0)

cmd/zstream/zstream_byteswap.c

@@ -0,0 +1,198 @@
+// SPDX-License-Identifier: CDDL-1.0
+/*
+ * CDDL HEADER START
+ *
+ * This file and its contents are supplied under the terms of the Common
+ * Development and Distribution License ("CDDL"), version 1.0. You may only use
+ * this file in accordance with the terms of version 1.0 of the CDDL.
+ *
+ * A full copy of the text of the CDDL should have accompanied this source. A
+ * copy of the CDDL is also available via the Internet at
+ * http://www.illumos.org/license/CDDL.
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Copyright (c) 2026 by Garth Snyder. All rights reserved.
+ */
+
+#include <err.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <sys/byteorder.h>
+#include <sys/spa_checksum.h>
+#include <sys/stdtypes.h>
+#include <sys/zfs_ioctl.h>
+
+#include "zstream_modules.h"
+
+/*
+ * Mostly from dmu_recv.c
+ */
+
+#define	DO64(X) (drr->drr_u.X = BSWAP_64(drr->drr_u.X))
+#define	DO32(X) (drr->drr_u.X = BSWAP_32(drr->drr_u.X))
+
+typedef byteswap_stage_t byteswap_context_t;
+
+static byteswap_context_t byteswap_contexts[MAX_BYTESWAP];
+static int next_context = 0;
+
+static disposition_t
+chain_byteswap(drr_packet_t *item, byteswap_context_t *context)
+{
+	if (item == NULL) {
+		return (D_OK);
+	}
+
+	struct dmu_replay_record *drr = &item->dp_drr;
+	boolean_t input_swapped = *context == BS_INCOMING &&
+	    ATTR_IS_SET(CA_BYTESWAPPED);
+	boolean_t swap = input_swapped || (*context == BS_OUTGOING &&
+	    OPTION_ENABLED(CA_BYTESWAP_ON_OUTPUT));
+	uint32_t drr_type =
+	    input_swapped ? BSWAP_32(drr->drr_type) : drr->drr_type;
+
+	if (swap) {
+		byteswap_record(drr, drr_type);
+	}
+	return (D_OK);
+}
+
+/*
+ * Unconditionally byteswap a DMU replay record. drr_type is passed in
+ * separately because we don't know whether we're doing input or output
+ * swapping.
+ */
+void
+byteswap_record(dmu_replay_record_t *drr, uint32_t drr_type)
+{
+	drr->drr_type = BSWAP_32(drr->drr_type);
+	drr->drr_payloadlen = BSWAP_32(drr->drr_payloadlen);
+
+	switch (drr_type) {
+
+	case DRR_BEGIN:
+		DO64(drr_begin.drr_magic);
+		DO64(drr_begin.drr_versioninfo);
+		DO64(drr_begin.drr_creation_time);
+		DO32(drr_begin.drr_type);
+		DO32(drr_begin.drr_flags);
+		DO64(drr_begin.drr_toguid);
+		DO64(drr_begin.drr_fromguid);
+		break;
+
+	case DRR_END:
+		DO64(drr_end.drr_toguid);
+		ZIO_CHECKSUM_BSWAP(&drr->drr_u.drr_end.drr_checksum);
+		break;
+
+	case DRR_OBJECT:
+		DO64(drr_object.drr_object);
+		DO32(drr_object.drr_type);
+		DO32(drr_object.drr_bonustype);
+		DO32(drr_object.drr_blksz);
+		DO32(drr_object.drr_bonuslen);
+		DO32(drr_object.drr_raw_bonuslen);
+		DO64(drr_object.drr_toguid);
+		DO64(drr_object.drr_maxblkid);
+		break;
+
+	case DRR_FREEOBJECTS:
+		DO64(drr_freeobjects.drr_firstobj);
+		DO64(drr_freeobjects.drr_numobjs);
+		DO64(drr_freeobjects.drr_toguid);
+		break;
+
+	case DRR_WRITE:
+		DO64(drr_write.drr_object);
+		DO32(drr_write.drr_type);
+		DO64(drr_write.drr_offset);
+		DO64(drr_write.drr_logical_size);
+		DO64(drr_write.drr_toguid);
+		ZIO_CHECKSUM_BSWAP(&drr->drr_u.drr_write.drr_key.ddk_cksum);
+		DO64(drr_write.drr_key.ddk_prop);
+		DO64(drr_write.drr_compressed_size);
+		break;
+
+	case DRR_WRITE_BYREF:
+		DO64(drr_write_byref.drr_object);
+		DO64(drr_write_byref.drr_offset);
+		DO64(drr_write_byref.drr_length);
+		DO64(drr_write_byref.drr_toguid);
+		DO64(drr_write_byref.drr_refguid);
+		DO64(drr_write_byref.drr_refobject);
+		DO64(drr_write_byref.drr_refoffset);
+		ZIO_CHECKSUM_BSWAP(
+		    &drr->drr_u.drr_write_byref.drr_key.ddk_cksum);
+		DO64(drr_write_byref.drr_key.ddk_prop);
+		break;
+
+	case DRR_FREE:
+		DO64(drr_free.drr_object);
+		DO64(drr_free.drr_offset);
+		DO64(drr_free.drr_length);
+		/* Note: toguid not byte-swapped in original zstream_dump.c */
+		DO64(drr_free.drr_toguid);
+		break;
+
+	case DRR_SPILL:
+		DO64(drr_spill.drr_object);
+		DO64(drr_spill.drr_length);
+		/* Note: toguid not byte-swapped in original zstream_dump.c */
+		DO64(drr_spill.drr_toguid);
+		DO64(drr_spill.drr_compressed_size);
+		DO32(drr_spill.drr_type);
+		break;
+
+	case DRR_WRITE_EMBEDDED:
+		DO64(drr_write_embedded.drr_object);
+		DO64(drr_write_embedded.drr_offset);
+		DO64(drr_write_embedded.drr_length);
+		DO64(drr_write_embedded.drr_toguid);
+		DO32(drr_write_embedded.drr_lsize);
+		DO32(drr_write_embedded.drr_psize);
+		break;
+
+	case DRR_OBJECT_RANGE:
+		DO64(drr_object_range.drr_firstobj);
+		DO64(drr_object_range.drr_numslots);
+		DO64(drr_object_range.drr_toguid);
+		break;
+
+	case DRR_REDACT:
+		DO64(drr_redact.drr_object);
+		DO64(drr_redact.drr_offset);
+		DO64(drr_redact.drr_length);
+		DO64(drr_redact.drr_toguid);
+		break;
+
+	default:
+		errx(1, "unknown record type %llu, aborting...",
+		    (u_longlong_t)drr_type);
+	}
+
+	if (drr_type != DRR_BEGIN) {
+		ZIO_CHECKSUM_BSWAP(&drr->drr_u.drr_checksum.drr_checksum);
+	}
+}
+
+chain_step_t
+serial_byteswap(byteswap_stage_t stage)
+{
+	int context_ix = next_context++ % MAX_BYTESWAP;
+	byteswap_context_t *bsc = &byteswap_contexts[context_ix];
+
+	*bsc = stage;
+	chain_step_t step = {
+		.cs_type = CS_SERIAL,
+		.cs_in_size = sizeof (drr_packet_t),
+		.cs_out_size = sizeof (drr_packet_t),
+		.cs_context = bsc,
+		.cs_serial = {
+			.process = (zc_serial_process_f *)chain_byteswap,
+		}
+	};
+	return (step);
+}

cmd/zstream/zstream_byteswap.h

@@ -0,0 +1,55 @@
+// SPDX-License-Identifier: CDDL-1.0
+/*
+ * CDDL HEADER START
+ *
+ * This file and its contents are supplied under the terms of the Common
+ * Development and Distribution License ("CDDL"), version 1.0. You may only use
+ * this file in accordance with the terms of version 1.0 of the CDDL.
+ *
+ * A full copy of the text of the CDDL should have accompanied this source. A
+ * copy of the CDDL is also available via the Internet at
+ * http://www.illumos.org/license/CDDL.
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Copyright (c) 2026 by Garth Snyder. All rights reserved.
+ */
+
+#ifndef _ZSTREAM_BYTESWAP_H
+#define	_ZSTREAM_BYTESWAP_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "zstream_io.h"
+
+#define	MAX_BYTESWAP 4  /* Most swapping ops in a chain */
+
+/*
+ * Byteswapping is generally done both on input and on output. By default,
+ * the stream's endianness is preserved. That is, opposite-endian streams
+ * are byteswapped for processing by other modules, then ultimately
+ * de-byteswapped for output.
+ */
+typedef enum { BS_INCOMING, BS_OUTGOING } byteswap_stage_t;
+
+chain_step_t
+serial_byteswap(byteswap_stage_t stage);
+
+/*
+ * Unconditionally swap a record. drr_type is passed in separately because
+ * we don't know whether we're doing input or output swapping. We need
+ * that value in native byte order to know how to swap the rest of the
+ * record.
+ */
+extern void
+byteswap_record(dmu_replay_record_t *drr, uint32_t drr_type);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif  /* _ZSTREAM_BYTESWAP_H */

cmd/zstream/zstream_chain.c

@@ -0,0 +1,155 @@
+// SPDX-License-Identifier: CDDL-1.0
+/*
+ * CDDL HEADER START
+ *
+ * This file and its contents are supplied under the terms of the Common
+ * Development and Distribution License ("CDDL"), version 1.0. You may only use
+ * this file in accordance with the terms of version 1.0 of the CDDL.
+ *
+ * A full copy of the text of the CDDL should have accompanied this source. A
+ * copy of the CDDL is also available via the Internet at
+ * http://www.illumos.org/license/CDDL.
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Copyright (c) 2026 by Garth Snyder. All rights reserved.
+ */
+
+#include <assert.h>
+#include <err.h>
+#include <libspl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/abd.h>
+#include <sys/param.h>
+#include <sys/stdtypes.h>
+#include <sys/zio.h>
+#include <sys/zstd/zstd.h>
+#include <zfs_fletcher.h>
+
+#include "zstream_chain.h"
+
+#define	MAX_CHAIN_LENGTH 32
+
+chain_attrs_t *chain_attrs;
+
+static disposition_t
+chain_null_step(void *item, void *context)
+{
+	(void) item;
+	(void) context;
+	return (D_OK);
+}
+
+chain_step_t
+serial_null_step(void)
+{
+	chain_step_t step = {
+		.cs_type = CS_SERIAL,
+		.cs_serial = {
+		    .process = (zc_serial_process_f *)chain_null_step
+		}
+	};
+	return (step);
+}
+
+chain_step_t
+chain_terminator(void)
+{
+	chain_step_t step = { .cs_type = CS_TERMINATE };
+	return (step);
+}
+
+static void
+libraries_init(void)
+{
+	zfs_refcount_init();
+	abd_init();
+	zio_init();
+	zstd_init();
+	libspl_init();
+	fletcher_4_init();
+}
+
+static void
+libraries_fini(void)
+{
+	fletcher_4_fini();
+	libspl_fini();
+	zio_fini();
+	zstd_fini();
+	abd_fini();
+	zfs_refcount_fini();
+}
+
+/*
+ * Execute a chain of serial processing steps.
+ *
+ * For simplicity, we normalize the chain item size to that of the largest
+ * output of any step. Packets with data beyond the base drr_record_t should
+ * add their additional data to the end of the packet, and this area may be
+ * reused for different purposes as items travel down the chain.
+ *
+ * Each item traverses the entire chain before the next item is read.
+ */
+void
+zstream_chain_exec(zstream_chain_t chain, chain_attrs_t *attrs)
+{
+	int num_steps = 0;
+	size_t packet_size = 0;
+	chain_attrs_t backup_attrs = {0};
+
+	chain_attrs = attrs ? attrs : &backup_attrs;
+
+	while (chain[num_steps].cs_type != CS_TERMINATE) {
+		packet_size = MAX(packet_size, chain[num_steps].cs_out_size);
+		num_steps++;
+		if (num_steps >= MAX_CHAIN_LENGTH) {
+			errx(1, "unterminated zstream_chain");
+		}
+	}
+	VERIFY3U(num_steps, >, 0);
+
+	/*
+	 * Check for consistency of input and output packet sizes in
+	 * adjacent steps. A declared packet size of zero waives this check.
+	 */
+	for (int i = 0; i < num_steps; i++) {
+		boolean_t mismatch = i > 0 &&
+		    chain[i].cs_in_size != 0 &&
+		    chain[i-1].cs_out_size != 0 &&
+		    chain[i].cs_in_size != chain[i-1].cs_out_size;
+		if (mismatch) {
+			warnx("note - chain steps %d and %d have "
+			    "mismatched packet sizes", i - 1, i);
+		}
+	}
+
+	libraries_init();
+
+	uint8_t buffer[packet_size];
+	boolean_t done = B_FALSE;
+
+	while (!done) {
+		for (int i = 0; i < num_steps; i++) {
+			if (done) {
+				(void) chain[i].cs_serial.process(NULL,
+				    chain[i].cs_context);
+			} else {
+				disposition_t dispo =
+				    chain[i].cs_serial.process(buffer,
+				    chain[i].cs_context);
+				if (dispo == D_EOF) {
+					VERIFY0(i);
+					done = B_TRUE;
+				} else if (dispo == D_DROP) {
+					break;
+				}
+			}
+		}
+	}
+
+	libraries_fini();
+}

cmd/zstream/zstream_chain.h

@@ -0,0 +1,181 @@
+// SPDX-License-Identifier: CDDL-1.0
+/*
+ * CDDL HEADER START
+ *
+ * This file and its contents are supplied under the terms of the Common
+ * Development and Distribution License ("CDDL"), version 1.0. You may only use
+ * this file in accordance with the terms of version 1.0 of the CDDL.
+ *
+ * A full copy of the text of the CDDL should have accompanied this source. A
+ * copy of the CDDL is also available via the Internet at
+ * http://www.illumos.org/license/CDDL.
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Copyright (c) 2026 by Garth Snyder. All rights reserved.
+ */
+
+#ifndef _ZSTREAM_CHAIN_H
+#define	_ZSTREAM_CHAIN_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stddef.h>
+#include <stdint.h>
+#include <sys/zfs_ioctl.h>
+
+/*
+ * A chain is a linear series of steps that process packets of data. It's
+ * designed to modularize common functionality, reduce code duplication, and
+ * separate processing structure from implementation.
+ *
+ * Some terms:
+ *
+ * **STEP** - A chain_step_t struct that represents a packet-processing
+ * module and any arguments or context that it needs. Chain modules
+ * generally define a function named serial_* that produces a chain_step_t
+ * that can be incorporated directly into a chain.
+ *
+ * **CHAIN** - An array of chain_step_t's. It's just data, so you can create
+ * the array however you like. But normally you'd just declare the whole
+ * thing:
+ *
+ *	zstream_chain_t dump_chain = {
+ *		serial_read_stream(infile),
+ *		serial_validate_fletcher4(),
+ *		serial_byteswap(BS_INCOMING),
+ *		serial_validate_records(),
+ *		serial_dump_records(&dump_args),
+ *		serial_null_output(),
+ *		chain_terminator()
+ *	}
+ *
+ * Or more succinctly:
+ *
+ *	zstream_chain_t dump_chain = {
+ *		STANDARD_INPUT_STACK(infile),
+ *		serial_dump_records(&dump_args),
+ *		NULL_OUTPUT_STACK()
+ *	};
+ *
+ * Chains must be terminated by a step of type CS_TERMINATE.
+ *
+ * **ITEMS** - The data packets that flow through a chain. Each step accepts
+ * items of one size and emits items of another size, which may be smaller,
+ * larger, or the same size. Items will generally be structs that start with
+ * a drr_packet_t (defined in zstream_io.h) and may include additional
+ * module-specific fields.
+ *
+ * **PROCESSING FUNCTION** - Each step names a processing function that does
+ * the actual work of transforming an input buffer into an output buffer.
+ * The transformation happens in place, in a single buffer provided by the
+ * chain.
+ *
+ * The processing function should return a disposition_t, normally D_OK. A
+ * function can return D_DROP to remove an item from the stream entirely. It
+ * can also return D_EOF to indicate that no more data will be forthcoming.
+ * However, only the first step in the chain should ever return D_EOF.
+ *
+ * Functions are called with a NULL packet pointer when the end of the
+ * stream passes by them.
+ *
+ * **CONTEXT** - An arbitrary void * that the chain passes along to the
+ * processing function as an argument.
+ *
+ * **CHAIN ATTRIBUTES** - A global set of flags available to all steps.
+ */
+
+#define	CA_BYTESWAPPED			(1ULL << 0)	/* ca_attrs */
+#define	CA_BIG_ENDIAN_INPUT		(1ULL << 1)
+#define	CA_LITTLE_ENDIAN_INPUT		(1ULL << 2)
+
+#define	CA_VERBOSE			(1ULL << 0)	/* ca_command_opts */
+#define	CA_VERY_VERBOSE			(1ULL << 1)
+#define	CA_DUMP_DATA			(1ULL << 2)
+#define	CA_IGNORE_CKSUMS		(1ULL << 3)
+#define	CA_DO_NOT_VALIDATE		(1ULL << 4)
+#define	CA_FORBID_DEDUP			(1ULL << 5)
+#define	CA_REQUIRE_DEDUP		(1ULL << 6)
+#define	CA_REQUIRE_NATIVE_ENDIAN	(1ULL << 7)
+#define	CA_BYTESWAP_ON_OUTPUT		(1ULL << 8)
+#define	CA_BIG_ENDIAN_OUT		(1ULL << 9)
+#define	CA_LITTLE_ENDIAN_OUT		(1ULL << 10)
+#define	CA_OPPOSITE_ENDIAN_OUT		(1ULL << 11)
+
+#define	OPTION_ENABLED(option) (!!(chain_attrs->ca_command_opts & (option)))
+#define	STREAM_HAS_FEATURE(feat) (!!(chain_attrs->ca_feature_flags & (feat)))
+#define	ATTR_IS_SET(attr) (!!(chain_attrs->ca_attrs & (attr)))
+
+#define	ENABLE_OPTION(attrs, opt) ((attrs)->ca_command_opts |= (opt))
+#define	SET_ATTR(attr) (chain_attrs->ca_attrs |= (attr))
+
+typedef struct {
+	uint64_t	rs_num_records;
+	uint64_t	rs_total_header_bytes;
+	uint64_t	rs_total_payload_bytes;
+} record_stats_t;
+
+/*
+ * Chain attribute flags that describe the stream. Statistics are maintained
+ * by the zstream_io modules.
+ */
+typedef struct {
+	uint64_t	ca_feature_flags;	/* From drr_versioninfo */
+	uint64_t	ca_attrs;		/* Discovered attributes */
+	uint64_t	ca_command_opts;	/* Command line options */
+	record_stats_t	ca_totals_in;
+	record_stats_t	ca_totals_out;
+	record_stats_t	ca_stats_in[DRR_NUMTYPES];
+	record_stats_t	ca_stats_out[DRR_NUMTYPES];
+} chain_attrs_t;
+
+typedef enum { CS_SERIAL, CS_TERMINATE } step_type_t;
+typedef enum { D_OK, D_EOF, D_DROP } disposition_t;
+
+typedef disposition_t
+zc_serial_process_f(void *item, void *context);
+
+typedef struct chain_step
+{
+	step_type_t	cs_type;
+	size_t		cs_in_size;
+	size_t		cs_out_size;
+	void		*cs_context;
+	struct {
+		zc_serial_process_f	*process;
+	} cs_serial;
+} chain_step_t;
+
+typedef chain_step_t zstream_chain_t[];
+
+/*
+ * Chain attributes accessible to any step on the chain. In theory this
+ * could cause a race condition between reading and setting, but all
+ * attributes are typically set by the time the first record has been read.
+ * Ergo, nobody else will be executing while that first chain_read() runs.
+ */
+extern chain_attrs_t *chain_attrs;
+
+/*
+ * Execute a chain. Returns once execution is complete. You can pass NULL
+ * for the attrs if you're not interested in preserving them after the chain
+ * has run.
+ */
+void
+zstream_chain_exec(zstream_chain_t chain, chain_attrs_t *attrs);
+
+chain_step_t
+serial_null_step(void);
+
+chain_step_t
+chain_terminator(void);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif	/* _ZSTREAM_CHAIN_H */

cmd/zstream/zstream_decompress.c

@@ -25,40 +25,135 @@
  * Use is subject to license terms.
  *
  * Copyright (c) 2024, Klara, Inc.
+ * Copyright (c) 2026 by Garth Snyder
  */
 
 #include <err.h>
+#include <errno.h>
 #include <search.h>
+#include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
-#include <unistd.h>
+#include <string.h>
+#include <sys/stdtypes.h>
 #include <sys/zfs_ioctl.h>
-#include <sys/zio_checksum.h>
-#include <sys/zstd/zstd.h>
-#include "zfs_fletcher.h"
+#include <sys/zio_compress.h>
+#include <unistd.h>
+
 #include "zstream.h"
+#include "zstream_modules.h"
 #include "zstream_util.h"
 
+#define	KEYSIZE 64
+
+static disposition_t
+chain_decompress_named_writes(drr_packet_t *item, void *context)
+{
+	(void) context;
+	if (item == NULL) {
+		return (D_OK);
+	}
+
+	dmu_replay_record_t *drr = &item->dp_drr;
+	struct drr_write *drrw = &drr->drr_u.drr_write;
+	char key[KEYSIZE];
+	uint8_t *dcbuff;
+
+	if (drr->drr_type != DRR_WRITE) {
+		return (D_OK);
+	}
+
+	snprintf(key, KEYSIZE, "%llu,%llu",
+	    (u_longlong_t)drrw->drr_object, (u_longlong_t)drrw->drr_offset);
+	ENTRY e = { .key = key };
+	ENTRY *p = hsearch(e, FIND);
+	if (p == NULL) {
+		return (D_OK);
+	}
+
+	enum zio_compress ctype = (enum zio_compress)(intptr_t)p->data;
+	if (ctype == ZIO_COMPRESS_INHERIT) {
+		/* Unspecified */
+		ctype = drrw->drr_compressiontype;
+	}
+	if (ctype_is_uncompressed(ctype)) {
+		drrw->drr_compressiontype = 0;
+		drrw->drr_logical_size = drrw->drr_compressed_size;
+		drrw->drr_compressed_size = 0;
+		if (OPTION_ENABLED(CA_VERBOSE)) {
+			fprintf(stderr,
+			    "Resetting compression type to "
+			    "off for ino %llu offset %llu\n",
+			    (u_longlong_t)drrw->drr_object,
+			    (u_longlong_t)drrw->drr_offset);
+		}
+		return (D_OK);
+	}
+
+	if (write_is_encrypted(drrw)) {
+		warnx("the write for ino %llu offset %llu is marked "
+		    "as encrypted. Attempting decompression anyway...",
+		    (u_longlong_t)drrw->drr_object,
+		    (u_longlong_t)drrw->drr_offset);
+	}
+
+	dcbuff = decompress_buffer(item->dp_payload, item->dp_payload_size,
+	    drrw->drr_logical_size, ctype);
+
+	if (dcbuff == NULL) {
+		/*
+		 * The block must not be compressed, at least not with this
+		 * compression type, possibly because it gets written
+		 * multiple times in this stream.
+		 */
+		warnx("decompression failed for ino %llu offset %llu",
+		    (u_longlong_t)drrw->drr_object,
+		    (u_longlong_t)drrw->drr_offset);
+	} else {
+		free(item->dp_payload);
+		item->dp_payload = dcbuff;
+		item->dp_payload_size = drrw->drr_logical_size;
+		drrw->drr_compressiontype = 0;
+		drrw->drr_compressed_size = 0;
+		if (OPTION_ENABLED(CA_VERBOSE)) {
+			fprintf(stderr,
+			    "Successfully decompressed ino %llu offset %llu\n",
+			    (u_longlong_t)drrw->drr_object,
+			    (u_longlong_t)drrw->drr_offset);
+		}
+	}
+	return (D_OK);
+}
+
+static chain_step_t
+serial_decompress_named_writes(void)
+{
+	chain_step_t step = {
+		.cs_type = CS_SERIAL,
+		.cs_in_size = sizeof (drr_packet_t),
+		.cs_out_size = sizeof (drr_packet_t),
+		.cs_context = NULL,
+		.cs_serial = {
+		    .process =
+			(zc_serial_process_f *)chain_decompress_named_writes
+		}
+	};
+	return (step);
+}
+
 int
 zstream_do_decompress(int argc, char *argv[])
 {
-	const int KEYSIZE = 64;
-	int bufsz = SPA_MAXBLOCKSIZE;
-	char *buf = safe_malloc(bufsz);
-	dmu_replay_record_t thedrr;
-	dmu_replay_record_t *drr = &thedrr;
-	zio_cksum_t stream_cksum;
+	chain_attrs_t attrs = {0};
 	int c;
-	boolean_t verbose = B_FALSE;
 
 	while ((c = getopt(argc, argv, "v")) != -1) {
 		switch (c) {
 		case 'v':
-			verbose = B_TRUE;
+			ENABLE_OPTION(&attrs, CA_VERBOSE);
 			break;
 		case '?':
-			(void) fprintf(stderr, "invalid option '%c'\n",
-			    optopt);
+			fprintf(stderr, "invalid option '%c'\n", optopt);
 			zstream_usage();
 			break;
 		}
@@ -69,16 +164,16 @@ zstream_do_decompress(int argc, char *argv[])
 
 	if (argc < 0)
 		zstream_usage();
-
 	if (hcreate(argc) == 0)
-		errx(1, "hcreate");
+		errx(1, "hcreate failed");
+
 	for (int i = 0; i < argc; i++) {
 		uint64_t object, offset;
 		char *obj_str;
 		char *offset_str;
 		char *key;
 		char *end;
-		enum zio_compress type = ZIO_COMPRESS_LZ4;
+		enum zio_compress type = ZIO_COMPRESS_INHERIT;
 
 		obj_str = strsep(&argv[i], ",");
 		if (argv[i] == NULL) {
@@ -107,266 +202,33 @@ zstream_do_decompress(int argc, char *argv[])
 			else if (0 == strcmp("zstd", argv[i]))
 				type = ZIO_COMPRESS_ZSTD;
 			else {
-				fprintf(stderr, "Invalid compression type %s.\n"
+				errx(2, "invalid compression type %s. "
 				    "Supported types are off, lz4, lzjb, gzip, "
-				    "zle, and zstd\n",
-				    argv[i]);
-				exit(2);
+				    "zle, and zstd", argv[i]);
 			}
 		}
 
-		if (asprintf(&key, "%llu,%llu", (u_longlong_t)object,
-		    (u_longlong_t)offset) < 0) {
+		int n_chars = asprintf(&key, "%llu,%llu", (u_longlong_t)object,
+		    (u_longlong_t)offset);
+		if (n_chars < 0)
 			err(1, "asprintf");
-		}
-		ENTRY e = {.key = key};
+		ENTRY e = { .key = key };
 		ENTRY *p;
-
 		p = hsearch(e, ENTER);
 		if (p == NULL)
-			errx(1, "hsearch");
-		p->data = (void*)(intptr_t)type;
+			errx(1, "hsearch failed");
+		p->data = (void *)(intptr_t)type;
 	}
 
-	if (isatty(STDIN_FILENO)) {
-		(void) fprintf(stderr,
-		    "Error: The send stream is a binary format "
-		    "and can not be read from a\n"
-		    "terminal.  Standard input must be redirected.\n");
-		exit(1);
-	}
+	ENABLE_OPTION(&attrs, CA_FORBID_DEDUP);
 
-	fletcher_4_init();
-	int begin = 0;
-	boolean_t seen = B_FALSE;
-	while (sfread(drr, sizeof (*drr), stdin) != 0) {
-		struct drr_write *drrw;
-		uint64_t payload_size = 0;
+	zstream_chain_t decompress_chain = {
+		STANDARD_INPUT_STACK(NULL),
+		serial_decompress_named_writes(),
+		STANDARD_OUTPUT_STACK(NULL)
+	};
+	zstream_chain_exec(decompress_chain, &attrs);
 
-		/*
-		 * We need to regenerate the checksum.
-		 */
-		if (drr->drr_type != DRR_BEGIN) {
-			memset(&drr->drr_u.drr_checksum.drr_checksum, 0,
-			    sizeof (drr->drr_u.drr_checksum.drr_checksum));
-		}
-
-		switch (drr->drr_type) {
-		case DRR_BEGIN:
-		{
-			ZIO_SET_CHECKSUM(&stream_cksum, 0, 0, 0, 0);
-			VERIFY0(begin++);
-			seen = B_TRUE;
-
-			uint32_t sz = drr->drr_payloadlen;
-
-			VERIFY3U(sz, <=, 1U << 28);
-
-			if (sz != 0) {
-				if (sz > bufsz) {
-					buf = realloc(buf, sz);
-					if (buf == NULL)
-						err(1, "realloc");
-					bufsz = sz;
-				}
-				(void) sfread(buf, sz, stdin);
-			}
-			payload_size = sz;
-			break;
-		}
-		case DRR_END:
-		{
-			struct drr_end *drre = &drr->drr_u.drr_end;
-			/*
-			 * We would prefer to just check --begin == 0, but
-			 * replication streams have an end of stream END
-			 * record, so we must avoid tripping it.
-			 */
-			VERIFY3B(seen, ==, B_TRUE);
-			begin--;
-			/*
-			 * Use the recalculated checksum, unless this is
-			 * the END record of a stream package, which has
-			 * no checksum.
-			 */
-			if (!ZIO_CHECKSUM_IS_ZERO(&drre->drr_checksum))
-				drre->drr_checksum = stream_cksum;
-			break;
-		}
-
-		case DRR_OBJECT:
-		{
-			struct drr_object *drro = &drr->drr_u.drr_object;
-			VERIFY3S(begin, ==, 1);
-
-			if (drro->drr_bonuslen > 0) {
-				payload_size = DRR_OBJECT_PAYLOAD_SIZE(drro);
-				(void) sfread(buf, payload_size, stdin);
-			}
-			break;
-		}
-
-		case DRR_SPILL:
-		{
-			struct drr_spill *drrs = &drr->drr_u.drr_spill;
-			VERIFY3S(begin, ==, 1);
-			payload_size = DRR_SPILL_PAYLOAD_SIZE(drrs);
-			(void) sfread(buf, payload_size, stdin);
-			break;
-		}
-
-		case DRR_WRITE_BYREF:
-			VERIFY3S(begin, ==, 1);
-			fprintf(stderr,
-			    "Deduplicated streams are not supported\n");
-			exit(1);
-			break;
-
-		case DRR_WRITE:
-		{
-			VERIFY3S(begin, ==, 1);
-			drrw = &thedrr.drr_u.drr_write;
-			payload_size = DRR_WRITE_PAYLOAD_SIZE(drrw);
-			ENTRY *p;
-			char key[KEYSIZE];
-
-			snprintf(key, KEYSIZE, "%llu,%llu",
-			    (u_longlong_t)drrw->drr_object,
-			    (u_longlong_t)drrw->drr_offset);
-			ENTRY e = {.key = key};
-
-			p = hsearch(e, FIND);
-			if (p == NULL) {
-				/*
-				 * Read the contents of the block unaltered
-				 */
-				(void) sfread(buf, payload_size, stdin);
-				break;
-			}
-
-			/*
-			 * Read and decompress the block
-			 */
-			enum zio_compress c =
-			    (enum zio_compress)(intptr_t)p->data;
-
-			if (c == ZIO_COMPRESS_OFF) {
-				(void) sfread(buf, payload_size, stdin);
-				drrw->drr_compressiontype = 0;
-				drrw->drr_compressed_size = 0;
-				if (verbose)
-					fprintf(stderr,
-					    "Resetting compression type to "
-					    "off for ino %llu offset %llu\n",
-					    (u_longlong_t)drrw->drr_object,
-					    (u_longlong_t)drrw->drr_offset);
-				break;
-			}
-
-			uint64_t lsize = drrw->drr_logical_size;
-			ASSERT3U(payload_size, <=, lsize);
-
-			char *lzbuf = safe_calloc(payload_size);
-			(void) sfread(lzbuf, payload_size, stdin);
-
-			abd_t sabd, dabd;
-			abd_get_from_buf_struct(&sabd, lzbuf, payload_size);
-			abd_get_from_buf_struct(&dabd, buf, lsize);
-			int err = zio_decompress_data(c, &sabd, &dabd,
-			    payload_size, lsize, NULL);
-			abd_free(&dabd);
-			abd_free(&sabd);
-
-			if (err == 0) {
-				drrw->drr_compressiontype = 0;
-				drrw->drr_compressed_size = 0;
-				payload_size = lsize;
-				if (verbose) {
-					fprintf(stderr,
-					    "successfully decompressed "
-					    "ino %llu offset %llu\n",
-					    (u_longlong_t)drrw->drr_object,
-					    (u_longlong_t)drrw->drr_offset);
-				}
-			} else {
-				/*
-				 * The block must not be compressed, at least
-				 * not with this compression type, possibly
-				 * because it gets written multiple times in
-				 * this stream.
-				 */
-				warnx("decompression failed for "
-				    "ino %llu offset %llu",
-				    (u_longlong_t)drrw->drr_object,
-				    (u_longlong_t)drrw->drr_offset);
-				memcpy(buf, lzbuf, payload_size);
-			}
-
-			free(lzbuf);
-			break;
-		}
-
-		case DRR_WRITE_EMBEDDED:
-		{
-			VERIFY3S(begin, ==, 1);
-			struct drr_write_embedded *drrwe =
-			    &drr->drr_u.drr_write_embedded;
-			payload_size =
-			    P2ROUNDUP((uint64_t)drrwe->drr_psize, 8);
-			(void) sfread(buf, payload_size, stdin);
-			break;
-		}
-
-		case DRR_FREEOBJECTS:
-		case DRR_FREE:
-		case DRR_OBJECT_RANGE:
-			VERIFY3S(begin, ==, 1);
-			break;
-
-		default:
-			(void) fprintf(stderr, "INVALID record type 0x%x\n",
-			    drr->drr_type);
-			/* should never happen, so assert */
-			assert(B_FALSE);
-		}
-
-		if (feof(stdout)) {
-			fprintf(stderr, "Error: unexpected end-of-file\n");
-			exit(1);
-		}
-		if (ferror(stdout)) {
-			fprintf(stderr, "Error while reading file: %s\n",
-			    strerror(errno));
-			exit(1);
-		}
-
-		/*
-		 * We need to recalculate the checksum, and it needs to be
-		 * initially zero to do that.  BEGIN records don't have
-		 * a checksum.
-		 */
-		if (drr->drr_type != DRR_BEGIN) {
-			memset(&drr->drr_u.drr_checksum.drr_checksum, 0,
-			    sizeof (drr->drr_u.drr_checksum.drr_checksum));
-		}
-		if (dump_record(drr, buf, payload_size,
-		    &stream_cksum, STDOUT_FILENO) != 0)
-			break;
-		if (drr->drr_type == DRR_END) {
-			/*
-			 * Typically the END record is either the last
-			 * thing in the stream, or it is followed
-			 * by a BEGIN record (which also zeros the checksum).
-			 * However, a stream package ends with two END
-			 * records.  The last END record's checksum starts
-			 * from zero.
-			 */
-			ZIO_SET_CHECKSUM(&stream_cksum, 0, 0, 0, 0);
-		}
-	}
-	free(buf);
-	fletcher_4_fini();
 	hdestroy();
-
 	return (0);
 }

cmd/zstream/zstream_drop_record.c

@@ -26,37 +26,104 @@
  */
 
 #include <err.h>
+#include <errno.h>
 #include <search.h>
+#include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
-#include <unistd.h>
+#include <string.h>
+#include <sys/stdtypes.h>
 #include <sys/zfs_ioctl.h>
-#include <sys/zio_checksum.h>
-#include <sys/zstd/zstd.h>
-#include "zfs_fletcher.h"
+#include <unistd.h>
+
 #include "zstream.h"
-#include "zstream_util.h"
+#include "zstream_modules.h"
+
+#define	KEYSIZE 64
+
+static disposition_t
+chain_drop_records(drr_packet_t *item, void *context)
+{
+	(void) context;
+	if (item == NULL)
+		return (D_OK);
+
+	dmu_replay_record_t *drr = &item->dp_drr;
+	struct drr_write *drrw = &drr->drr_u.drr_write;
+	struct drr_write_embedded *drrwe = &drr->drr_u.drr_write_embedded;
+	char key[KEYSIZE];
+	u_longlong_t object, offset;
+	const char *record_type;
+	ENTRY e = {.key = key};
+
+	if (drr->drr_type == DRR_WRITE) {
+		object = drrw->drr_object;
+		offset = drrw->drr_offset;
+		record_type = "WRITE";
+	} else if (drr->drr_type == DRR_WRITE_EMBEDDED) {
+		object = drrwe->drr_object;
+		offset = drrwe->drr_offset;
+		record_type = "WRITE_EMBEDDED";
+	} else {
+		return (D_OK);
+	}
+
+	snprintf(key, KEYSIZE, "%llu,%llu", object, offset);
+	if (hsearch(e, FIND) != NULL) {
+		if (OPTION_ENABLED(CA_VERBOSE)) {
+			warnx("dropping %s record for object %llu "
+			    "offset %llu", record_type, object, offset);
+		}
+		/*
+		 * It really feels like the chain executor ought to be
+		 * responsible for freeing this payload. However, it
+		 * operates at a more abstract level and knows nothing about
+		 * DMU records and their payloads, so this'll have to be
+		 * done here when the drop decision is made.
+		 *
+		 * Fine for now, but if another case like this comes up in
+		 * the future, the issue probably needs to be handled
+		 * through a more clearly defined path.
+		 */
+		if (item->dp_payload_size && item->dp_payload != NULL) {
+			free(item->dp_payload);
+			item->dp_payload = NULL;
+			item->dp_payload_size = 0;
+		}
+		return (D_DROP);
+	}
+
+	return (D_OK);
+}
+
+static chain_step_t
+serial_drop_records(void)
+{
+	chain_step_t step = {
+		.cs_type = CS_SERIAL,
+		.cs_in_size = sizeof (drr_packet_t),
+		.cs_out_size = sizeof (drr_packet_t),
+		.cs_context = NULL,
+		.cs_serial = {
+			.process = (zc_serial_process_f *)chain_drop_records
+		}
+	};
+	return (step);
+}
 
 int
 zstream_do_drop_record(int argc, char *argv[])
 {
-	const int KEYSIZE = 64;
-	int bufsz = SPA_MAXBLOCKSIZE;
-	char *buf = safe_malloc(bufsz);
-	dmu_replay_record_t thedrr;
-	dmu_replay_record_t *drr = &thedrr;
-	zio_cksum_t stream_cksum;
 	int c;
-	boolean_t verbose = B_FALSE;
+	chain_attrs_t attrs = {0};
 
 	while ((c = getopt(argc, argv, "v")) != -1) {
 		switch (c) {
 		case 'v':
-			verbose = B_TRUE;
+			ENABLE_OPTION(&attrs, CA_VERBOSE);
 			break;
 		case '?':
-			(void) fprintf(stderr, "invalid option '%c'\n",
-			    optopt);
+			warnx("invalid option '%c'\n", optopt);
 			zstream_usage();
 			break;
 		}
@@ -67,10 +134,11 @@ zstream_do_drop_record(int argc, char *argv[])
 
 	if (argc < 0)
 		zstream_usage();
-
 	if (hcreate(argc) == 0)
-		errx(1, "hcreate");
+		errx(1, "hcreate failed");
+
 	for (int i = 0; i < argc; i++) {
+
 		uint64_t object, offset;
 		char *obj_str;
 		char *offset_str;
@@ -97,228 +165,21 @@ zstream_do_drop_record(int argc, char *argv[])
 		}
 		ENTRY e = {.key = key};
 		ENTRY *p;
-
 		p = hsearch(e, ENTER);
 		if (p == NULL)
 			errx(1, "hsearch");
-		p->data = (void*)(intptr_t)B_TRUE;
+		p->data = (void *)(intptr_t)B_TRUE;
 	}
 
-	if (isatty(STDIN_FILENO)) {
-		(void) fprintf(stderr,
-		    "Error: The send stream is a binary format "
-		    "and can not be read from a\n"
-		    "terminal.  Standard input must be redirected.\n");
-		exit(1);
-	}
+	ENABLE_OPTION(&attrs, CA_FORBID_DEDUP);
 
-	fletcher_4_init();
-	int begin = 0;
-	boolean_t seen = B_FALSE;
-	while (sfread(drr, sizeof (*drr), stdin) != 0) {
-		struct drr_write *drrw;
-		uint64_t payload_size = 0;
+	zstream_chain_t drop_chain = {
+		STANDARD_INPUT_STACK(NULL),
+		serial_drop_records(),
+		STANDARD_OUTPUT_STACK(NULL)
+	};
+	zstream_chain_exec(drop_chain, &attrs);
 
-		/*
-		 * We need to regenerate the checksum.
-		 */
-		if (drr->drr_type != DRR_BEGIN) {
-			memset(&drr->drr_u.drr_checksum.drr_checksum, 0,
-			    sizeof (drr->drr_u.drr_checksum.drr_checksum));
-		}
-
-		switch (drr->drr_type) {
-		case DRR_BEGIN:
-		{
-			ZIO_SET_CHECKSUM(&stream_cksum, 0, 0, 0, 0);
-			VERIFY0(begin++);
-			seen = B_TRUE;
-
-			uint32_t sz = drr->drr_payloadlen;
-
-			VERIFY3U(sz, <=, 1U << 28);
-
-			if (sz != 0) {
-				if (sz > bufsz) {
-					buf = realloc(buf, sz);
-					if (buf == NULL)
-						err(1, "realloc");
-					bufsz = sz;
-				}
-				(void) sfread(buf, sz, stdin);
-			}
-			payload_size = sz;
-			break;
-		}
-		case DRR_END:
-		{
-			struct drr_end *drre = &drr->drr_u.drr_end;
-			/*
-			 * We would prefer to just check --begin == 0, but
-			 * replication streams have an end of stream END
-			 * record, so we must avoid tripping it.
-			 */
-			VERIFY3B(seen, ==, B_TRUE);
-			begin--;
-			/*
-			 * Use the recalculated checksum, unless this is
-			 * the END record of a stream package, which has
-			 * no checksum.
-			 */
-			if (!ZIO_CHECKSUM_IS_ZERO(&drre->drr_checksum))
-				drre->drr_checksum = stream_cksum;
-			break;
-		}
-
-		case DRR_OBJECT:
-		{
-			struct drr_object *drro = &drr->drr_u.drr_object;
-			VERIFY3S(begin, ==, 1);
-
-			if (drro->drr_bonuslen > 0) {
-				payload_size = DRR_OBJECT_PAYLOAD_SIZE(drro);
-				(void) sfread(buf, payload_size, stdin);
-			}
-			break;
-		}
-
-		case DRR_SPILL:
-		{
-			struct drr_spill *drrs = &drr->drr_u.drr_spill;
-			VERIFY3S(begin, ==, 1);
-			payload_size = DRR_SPILL_PAYLOAD_SIZE(drrs);
-			(void) sfread(buf, payload_size, stdin);
-			break;
-		}
-
-		case DRR_WRITE_BYREF:
-			VERIFY3S(begin, ==, 1);
-			fprintf(stderr,
-			    "Deduplicated streams are not supported\n");
-			exit(1);
-			break;
-
-		case DRR_WRITE:
-		{
-			VERIFY3S(begin, ==, 1);
-			drrw = &thedrr.drr_u.drr_write;
-			payload_size = DRR_WRITE_PAYLOAD_SIZE(drrw);
-			ENTRY *p;
-			char key[KEYSIZE];
-
-			snprintf(key, KEYSIZE, "%llu,%llu",
-			    (u_longlong_t)drrw->drr_object,
-			    (u_longlong_t)drrw->drr_offset);
-			ENTRY e = {.key = key};
-
-			(void) sfread(buf, payload_size, stdin);
-			p = hsearch(e, FIND);
-			if (p == NULL) {
-				/*
-				 * Dump the contents of the block unaltered
-				 */
-			} else {
-				/*
-				 * Read and discard the block
-				 */
-				if (verbose)
-					fprintf(stderr,
-					    "Dropping WRITE record for object "
-					    "%llu offset %llu\n",
-					    (u_longlong_t)drrw->drr_object,
-					    (u_longlong_t)drrw->drr_offset);
-				continue;
-			}
-			break;
-		}
-
-		case DRR_WRITE_EMBEDDED:
-		{
-			ENTRY *p;
-			char key[KEYSIZE];
-
-			VERIFY3S(begin, ==, 1);
-			struct drr_write_embedded *drrwe =
-			    &drr->drr_u.drr_write_embedded;
-			payload_size =
-			    P2ROUNDUP((uint64_t)drrwe->drr_psize, 8);
-
-			snprintf(key, KEYSIZE, "%llu,%llu",
-			    (u_longlong_t)drrwe->drr_object,
-			    (u_longlong_t)drrwe->drr_offset);
-			ENTRY e = {.key = key};
-
-			(void) sfread(buf, payload_size, stdin);
-			p = hsearch(e, FIND);
-			if (p == NULL) {
-				/*
-				 * Dump the contents of the block unaltered
-				 */
-			} else {
-				/*
-				 * Read and discard the block
-				 */
-				if (verbose)
-					fprintf(stderr,
-					    "Dropping WRITE_EMBEDDED record for"
-					    " object %llu offset %llu\n",
-					    (u_longlong_t)drrwe->drr_object,
-					    (u_longlong_t)drrwe->drr_offset);
-				continue;
-			}
-			break;
-		}
-
-		case DRR_FREEOBJECTS:
-		case DRR_FREE:
-		case DRR_OBJECT_RANGE:
-			VERIFY3S(begin, ==, 1);
-			break;
-
-		default:
-			(void) fprintf(stderr, "INVALID record type 0x%x\n",
-			    drr->drr_type);
-			/* should never happen, so assert */
-			assert(B_FALSE);
-		}
-
-		if (feof(stdout)) {
-			fprintf(stderr, "Error: unexpected end-of-file\n");
-			exit(1);
-		}
-		if (ferror(stdout)) {
-			fprintf(stderr, "Error while reading file: %s\n",
-			    strerror(errno));
-			exit(1);
-		}
-
-		/*
-		 * We need to recalculate the checksum, and it needs to be
-		 * initially zero to do that.  BEGIN records don't have
-		 * a checksum.
-		 */
-		if (drr->drr_type != DRR_BEGIN) {
-			memset(&drr->drr_u.drr_checksum.drr_checksum, 0,
-			    sizeof (drr->drr_u.drr_checksum.drr_checksum));
-		}
-		if (dump_record(drr, buf, payload_size,
-		    &stream_cksum, STDOUT_FILENO) != 0)
-			break;
-		if (drr->drr_type == DRR_END) {
-			/*
-			 * Typically the END record is either the last
-			 * thing in the stream, or it is followed
-			 * by a BEGIN record (which also zeros the checksum).
-			 * However, a stream package ends with two END
-			 * records.  The last END record's checksum starts
-			 * from zero.
-			 */
-			ZIO_SET_CHECKSUM(&stream_cksum, 0, 0, 0, 0);
-		}
-	}
-	free(buf);
-	fletcher_4_fini();
 	hdestroy();
-
 	return (0);
 }

cmd/zstream/zstream_fletcher4.c

@@ -0,0 +1,180 @@
+// SPDX-License-Identifier: CDDL-1.0
+/*
+ * CDDL HEADER START
+ *
+ * This file and its contents are supplied under the terms of the Common
+ * Development and Distribution License ("CDDL"), version 1.0. You may only use
+ * this file in accordance with the terms of version 1.0 of the CDDL.
+ *
+ * A full copy of the text of the CDDL should have accompanied this source. A
+ * copy of the CDDL is also available via the Internet at
+ * http://www.illumos.org/license/CDDL.
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Copyright (c) 2026 by Garth Snyder. All rights reserved.
+ */
+
+#include <assert.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <sys/byteorder.h>
+#include <sys/spa_checksum.h>
+#include <sys/stdtypes.h>
+#include <sys/types.h>
+#include <sys/zfs_ioctl.h>
+#include <zfs_fletcher.h>
+
+#include "zstream_modules.h"
+#include "zstream_util.h"
+
+#define	CK_OFFSET offsetof(dmu_replay_record_t, drr_u.drr_checksum.drr_checksum)
+#define	END_CK_OFFSET offsetof(dmu_replay_record_t, drr_u.drr_end.drr_checksum)
+
+typedef enum { F4_SET, F4_VALIDATE } fletcher4_op_t;
+
+typedef zio_cksum_t fletcher4_context_t;
+
+static fletcher4_context_t	fletcher4_contexts[MAX_FLETCHER_4];
+static int			next_context = 0;
+
+static inline int
+fletcher_4_incremental(boolean_t swap, void *buff, size_t size, void *cksum)
+{
+	if (swap) {
+		return (fletcher_4_incremental_byteswap(buff, size, cksum));
+	} else {
+		return (fletcher_4_incremental_native(buff, size, cksum));
+	}
+}
+
+/*
+ * Emit or validate (below) a replay record with proper checksums and with
+ * proper maintenance of the stream checksum. That is:
+ *
+ *   1) Update stream checksum with the record header up to drr_checksum.
+ *   2) Update drr_checksum field in the record header from stream checksum.
+ *   3) Update stream checksum with the checksum field in the record header.
+ *   4) Update stream checksum with the contents of the payload.
+ *
+ * DRR_BEGIN records do not have record checksums. They can't, because the
+ * drr_begin struct overlaps with space that would otherwise be used for the
+ * end-record checksum.
+ */
+static disposition_t
+chain_add_fletcher4(drr_packet_t *item, zio_cksum_t *stream_cksum)
+{
+	if (item == NULL)
+		return (D_OK);
+
+	dmu_replay_record_t *drr   = &item->dp_drr;
+	struct drr_end *drre	   = &item->dp_drr.drr_u.drr_end;
+	zio_cksum_t *record_cksum  = &drr->drr_u.drr_checksum.drr_checksum;
+	zio_cksum_t *end_cksum	   = &drre->drr_checksum;
+
+	boolean_t swap = OPTION_ENABLED(CA_BYTESWAP_ON_OUTPUT);
+	uint32_t drr_type = swap ? BSWAP_32(drr->drr_type) : drr->drr_type;
+
+	if (drr_type == DRR_BEGIN) {
+		ZIO_SET_CHECKSUM(stream_cksum, 0, 0, 0, 0);
+	} else if (drr_type == DRR_END) {
+		*end_cksum = *stream_cksum;
+		if (swap)
+			ZIO_CHECKSUM_BSWAP(end_cksum);
+	}
+	fletcher_4_incremental(swap, drr, CK_OFFSET, stream_cksum);
+	if (drr_type != DRR_BEGIN && !IS_CONCLUSION(drr, drr_type)) {
+		*record_cksum = *stream_cksum;
+		if (swap)
+			ZIO_CHECKSUM_BSWAP(record_cksum);
+	}
+	if (drr_type == DRR_END) {
+		ZIO_SET_CHECKSUM(stream_cksum, 0, 0, 0, 0);
+	} else {
+		fletcher_4_incremental(swap, record_cksum,
+		    sizeof (drr->drr_u.drr_checksum.drr_checksum),
+		    stream_cksum);
+		if (item->dp_payload_size > 0) {
+			fletcher_4_incremental(swap, item->dp_payload,
+			    item->dp_payload_size, stream_cksum);
+		}
+	}
+	return (D_OK);
+}
+
+static disposition_t
+chain_validate_fletcher4(drr_packet_t *item, zio_cksum_t *stream_cksum)
+{
+	if (item == NULL || OPTION_ENABLED(CA_IGNORE_CKSUMS)) {
+		return (D_OK);
+	}
+
+	dmu_replay_record_t *drr   = &item->dp_drr;
+	struct drr_end *drre	   = &item->dp_drr.drr_u.drr_end;
+	zio_cksum_t *record_cksum  = &drr->drr_u.drr_checksum.drr_checksum;
+	zio_cksum_t *end_cksum	   = &drre->drr_checksum;
+
+	boolean_t swap = ATTR_IS_SET(CA_BYTESWAPPED);
+	uint32_t drr_type = swap ? BSWAP_32(drr->drr_type) : drr->drr_type;
+
+	if (drr_type == DRR_BEGIN) {
+		ZIO_SET_CHECKSUM(stream_cksum, 0, 0, 0, 0);
+	} else if (drr_type == DRR_END) {
+		off_t stream_offset = item->dp_stream_offset + END_CK_OFFSET;
+		validate_or_exit(stream_cksum, end_cksum, swap,
+		    "in DRR_END record", stream_offset);
+	}
+	fletcher_4_incremental(swap, drr, CK_OFFSET, stream_cksum);
+	if (drr_type != DRR_BEGIN && !IS_CONCLUSION(drr, drr_type)) {
+		off_t stream_offset = item->dp_stream_offset + CK_OFFSET;
+		validate_or_exit(stream_cksum, record_cksum,
+		    swap, "at DRR record end", stream_offset);
+	}
+	if (drr_type == DRR_END) {
+		ZIO_SET_CHECKSUM(stream_cksum, 0, 0, 0, 0);
+	} else {
+		fletcher_4_incremental(swap, record_cksum,
+		    sizeof (drr->drr_u.drr_checksum.drr_checksum),
+		    stream_cksum);
+		if (item->dp_payload_size > 0) {
+			fletcher_4_incremental(swap, item->dp_payload,
+			    item->dp_payload_size, stream_cksum);
+		}
+	}
+	return (D_OK);
+}
+
+static chain_step_t
+fletcher4_serial_step(fletcher4_op_t operation)
+{
+	int context_ix = next_context++ % MAX_FLETCHER_4;
+	fletcher4_context_t *context = &fletcher4_contexts[context_ix];
+
+	ZIO_SET_CHECKSUM(context, 0, 0, 0, 0);
+	chain_step_t step = {
+		.cs_type = CS_SERIAL,
+		.cs_in_size = sizeof (drr_packet_t),
+		.cs_out_size = sizeof (drr_packet_t),
+		.cs_context = context,
+		.cs_serial = {
+			.process = (zc_serial_process_f *)
+			    ((operation == F4_VALIDATE) ?
+			    chain_validate_fletcher4 : chain_add_fletcher4)
+		}
+	};
+	return (step);
+}
+
+chain_step_t
+serial_add_fletcher4(void)
+{
+	return (fletcher4_serial_step(F4_SET));
+}
+
+chain_step_t
+serial_validate_fletcher4(void)
+{
+	return (fletcher4_serial_step(F4_VALIDATE));
+}

cmd/zstream/zstream_fletcher4.h

@@ -0,0 +1,78 @@
+// SPDX-License-Identifier: CDDL-1.0
+/*
+ * CDDL HEADER START
+ *
+ * This file and its contents are supplied under the terms of the Common
+ * Development and Distribution License ("CDDL"), version 1.0. You may only use
+ * this file in accordance with the terms of version 1.0 of the CDDL.
+ *
+ * A full copy of the text of the CDDL should have accompanied this source. A
+ * copy of the CDDL is also available via the Internet at
+ * http://www.illumos.org/license/CDDL.
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Copyright (c) 2026 by Garth Snyder. All rights reserved.
+ */
+
+#ifndef _ZSTREAM_FLETCHER4_H
+#define	_ZSTREAM_FLETCHER4_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "zstream_io.h"
+
+/*
+ * zstream_chain module for calculating, validating, and inscribing
+ * Fletcher4 checksums.
+ *
+ * serial_validate_fletcher4() validates record checksums against the
+ * running stream checksum and fails loudly on any mismatch.
+ *
+ * serial_add_fletcher4() inscribes record checksums from the running
+ * stream checksum, in theory replacing whatever was there before. But
+ * see note in zstream_fletcher4.c regarding zero checksums generated
+ * by send_conclusion_record(), which are preserved.
+ */
+
+/*
+ * DRR_END records normally do have end-record checksums. However, records
+ * emitted by send_conclusion_record() in libzfs_sendrecv.c have the
+ * checksum set to zero. zfs receive ignores those checksums. DRR_END
+ * records also have an internal checksum that applies to the stream-to-date
+ * since the most recent DRR_BEGIN.
+ *
+ * Ideally, null zstream transformations should be idempotent. E.g., a
+ * zstream redup that does not redup anything should yield a stream that is
+ * identical to the original stream. So, it's helpful to emulate zfs send's
+ * checksumming practices just to minimize spurious differences between
+ * input and output streams.
+ *
+ * The IS_CONCLUSION macro recognizes the records generated by
+ * send_conclusion_record() so that they can be treated specially.
+ */
+#define	IS_CONCLUSION(drr, type)					       \
+	    ((type) == DRR_END &&					       \
+	    (drr)->drr_u.drr_end.drr_toguid == 0 &&			       \
+	    ZIO_CHECKSUM_IS_ZERO(&(drr)->drr_u.drr_checksum.drr_checksum))
+
+/*
+ * Maximum number of checksum operations in one chain
+ */
+#define	MAX_FLETCHER_4 8
+
+chain_step_t
+serial_validate_fletcher4(void);
+
+chain_step_t
+serial_add_fletcher4(void);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif  /* _ZSTREAM_FLETCHER4_H */

cmd/zstream/zstream_io.c

@@ -0,0 +1,462 @@
+// SPDX-License-Identifier: CDDL-1.0
+/*
+ * CDDL HEADER START
+ *
+ * This file and its contents are supplied under the terms of the Common
+ * Development and Distribution License ("CDDL"), version 1.0. You may only use
+ * this file in accordance with the terms of version 1.0 of the CDDL.
+ *
+ * A full copy of the text of the CDDL should have accompanied this source. A
+ * copy of the CDDL is also available via the Internet at
+ * http://www.illumos.org/license/CDDL.
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Copyright (c) 2026 by Garth Snyder. All rights reserved.
+ */
+
+#include <arpa/inet.h>
+#include <err.h>
+#include <errno.h>
+#include <libzutil.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/byteorder.h>
+#include <sys/stdtypes.h>
+#include <sys/sysmacros.h>
+#include <sys/types.h>
+#include <sys/zfs_ioctl.h>
+#include <time.h>
+#include <unistd.h>
+
+#include "zstream_chain.h"
+#include "zstream_modules.h"
+#include "zstream_util.h"
+
+/*
+ * Init only the filename; chain functions will prepare the FILE *
+ */
+typedef struct {
+	const char	*ic_filename;
+	FILE		*ic_fp;
+	boolean_t	ic_for_reading;
+	off_t		ic_offset;
+} io_context_t;
+
+typedef struct {
+	const char	*cc_name;
+	double		cc_last_sec;
+	double		cc_period_sec;
+	uint64_t	cc_last_bytes;
+} checkpoint_context_t;
+
+static io_context_t io_contexts[MAX_IO_STREAMS];
+static int next_io_context = 0;
+
+static checkpoint_context_t checkpoint_contexts[MAX_IO_STREAMS];
+static int next_checkpoint_context = 0;
+
+/*
+ * Run from within chain execution to initialize I/O. A NULL filename
+ * indicates stdin or stdout.
+ */
+static void
+open_file(io_context_t *context)
+{
+	if (context->ic_filename) {
+		context->ic_fp = fopen(context->ic_filename,
+		    context->ic_for_reading ? "rb" : "wb+");
+		if (!context->ic_fp) {
+			perror(context->ic_filename);
+			exit(1);
+		}
+	} else if (context->ic_for_reading && isatty(STDIN_FILENO)) {
+		errx(1, "stream cannot be read from a terminal. "
+		    "Name a file or take input from a pipe.");
+	} else if (context->ic_for_reading) {
+		context->ic_fp = stdin;
+	} else if (isatty(STDOUT_FILENO)) {
+		errx(1, "stream cannot be written to a terminal. "
+		    "Capture output to a file or pipe to another command.");
+	} else {
+		context->ic_fp = stdout;
+	}
+}
+
+/*
+ * Extract the payload size from a replay record that is potentially
+ * byteswapped. We want to leave the bulk of byteswapping to another module,
+ * so just take a quick, nondestructive peek.
+ *
+ * Record-specific macros such as DRR_WRITE_PAYLOAD_SIZE do not seem to be
+ * byteswap-aware. However, with the exception of DRR_OBJECT_PAYLOAD_SIZE,
+ * they happen to work with post-swapping since they are switching on either
+ * a uint8_t value or 0.
+ *
+ * DRR_WRITE and DRR_SPILL use 64-bit sizes. The other two record types have
+ * 32-bit sizes. The drr_payloadlen field shared by all record types (but
+ * used only by BEGIN records is also 32 bits.
+ */
+static size_t
+calc_payload_size(dmu_replay_record_t *drr)
+{
+	struct drr_object *drro		 = &drr->drr_u.drr_object;
+	struct drr_write *drrw		 = &drr->drr_u.drr_write;
+	struct drr_spill *drrs		 = &drr->drr_u.drr_spill;
+	struct drr_write_embedded *drrwe = &drr->drr_u.drr_write_embedded;
+
+	boolean_t swap = ATTR_IS_SET(CA_BYTESWAPPED);
+	uint32_t drr_type = swap ? BSWAP_32(drr->drr_type) : drr->drr_type;
+	uint64_t size, size64 = 0;
+	uint32_t size32 = 0;
+	boolean_t round = B_FALSE;
+
+	if (drr_type == DRR_OBJECT) {
+		round = drro->drr_raw_bonuslen == 0;
+		size32 = round ? drro->drr_bonuslen : drro->drr_raw_bonuslen;
+	} else if (drr_type == DRR_WRITE) {
+		size64 = DRR_WRITE_PAYLOAD_SIZE(drrw);
+	} else if (drr_type == DRR_SPILL) {
+		size64 = DRR_SPILL_PAYLOAD_SIZE(drrs);
+	} else if (drr_type == DRR_WRITE_EMBEDDED) {
+		size32 = drrwe->drr_psize;
+		round = B_TRUE;
+	} else if (drr_type == DRR_BEGIN) {
+		size32 = drr->drr_payloadlen;
+	} else {
+		return (0);
+	}
+	if (size32 != 0) {
+		size = swap ? BSWAP_32(size32) : size32;
+	} else {
+		size = swap ? BSWAP_64(size64) : size64;
+	}
+	return (round ? P2ROUNDUP(size, 8) : size);
+}
+
+/*
+ * Must be called only with the first record in a stream. Must be a
+ * DRR_BEGIN record or we'll terminate with "invalid stream".
+ */
+static void
+set_stream_attributes(drr_packet_t *item)
+{
+	dmu_replay_record_t *drr  = &item->dp_drr;
+	struct drr_begin *drrb    = &drr->drr_u.drr_begin;
+	uint64_t magic		  = drrb->drr_magic;
+	uint64_t versioninfo	  = drrb->drr_versioninfo;
+	boolean_t i_am_big_endian = htonl(0xFF00) == 0xFF00;
+
+	boolean_t swap_on_output, is_deduped;
+
+	if (magic == BSWAP_64(DMU_BACKUP_MAGIC)) {
+		SET_ATTR(CA_BYTESWAPPED);
+		versioninfo = BSWAP_64(versioninfo);
+	} else if (magic != DMU_BACKUP_MAGIC) {
+		errx(1, "invalid ZFS stream, bad magic number %llx",
+		    (u_longlong_t)magic);
+	}
+	if (i_am_big_endian == ATTR_IS_SET(CA_BYTESWAPPED)) {
+		SET_ATTR(CA_LITTLE_ENDIAN_INPUT);
+	} else {
+		SET_ATTR(CA_BIG_ENDIAN_INPUT);
+	}
+	chain_attrs->ca_feature_flags = DMU_GET_FEATUREFLAGS(versioninfo);
+
+	is_deduped =
+	    STREAM_HAS_FEATURE(DMU_BACKUP_FEATURE_DEDUP) ||
+	    STREAM_HAS_FEATURE(DMU_BACKUP_FEATURE_DEDUPPROPS);
+
+	if (OPTION_ENABLED(CA_FORBID_DEDUP) && is_deduped) {
+		errx(1, "input stream is deduplicated, but this subcommand "
+		    "does not support deduplicated streams. Use 'zstream "
+		    "redup' to reduplicate.");
+	}
+	boolean_t req_dedup = OPTION_ENABLED(CA_REQUIRE_DEDUP);
+	boolean_t is_dedup = STREAM_HAS_FEATURE(DMU_BACKUP_FEATURE_DEDUP);
+	if (req_dedup && !is_dedup) {
+		errx(1, "this subcommand requires a deduplicated input "
+		    "stream, but the stream is not deduplicated");
+	}
+	boolean_t req_native = OPTION_ENABLED(CA_REQUIRE_NATIVE_ENDIAN);
+	boolean_t is_byteswapped = ATTR_IS_SET(CA_BYTESWAPPED);
+	if (req_native && is_byteswapped) {
+		errx(1, "this subcommand requires a native-endian "
+		    "input stream");
+	}
+
+	/*
+	 * Figure out output endianness. In the absence of explicit byte
+	 * order instructions, we default to preserving the input byte
+	 * order. Record headers are always converted to native byte order
+	 * for processing, but they can be swapped back on output.
+	 *
+	 * zfs receive inspects the endianness of each DRR record
+	 * and assumes, at least in some cases, that payload data has the
+	 * same order as the DMU wrappers.
+	 */
+	if (OPTION_ENABLED(CA_BIG_ENDIAN_OUT))
+		swap_on_output = !i_am_big_endian;
+	else if (OPTION_ENABLED(CA_LITTLE_ENDIAN_OUT))
+		swap_on_output = i_am_big_endian;
+	else if (OPTION_ENABLED(CA_OPPOSITE_ENDIAN_OUT))
+		swap_on_output = !ATTR_IS_SET(CA_BYTESWAPPED);
+	else
+		swap_on_output = ATTR_IS_SET(CA_BYTESWAPPED);
+
+	if (swap_on_output) {
+		ENABLE_OPTION(chain_attrs, CA_BYTESWAP_ON_OUTPUT);
+	}
+}
+
+static disposition_t
+chain_read(drr_packet_t *item, io_context_t *context)
+{
+	if (item == NULL)
+		return (D_OK);
+
+	dmu_replay_record_t *drr = &item->dp_drr;
+
+	if (!context->ic_fp)
+		open_file(context);
+
+	if (fread(drr, sizeof (dmu_replay_record_t), 1, context->ic_fp) != 1) {
+		if (ferror(context->ic_fp)) {
+			err(1, "error reading record header at offset %llu",
+			    (u_longlong_t)context->ic_offset);
+		}
+		fclose(context->ic_fp);
+		return (D_EOF);
+	}
+
+	if (context->ic_offset == 0)
+		set_stream_attributes(item);
+
+	size_t payload_size = calc_payload_size(&item->dp_drr);
+	if (payload_size > UINT32_MAX) {
+		errx(1, "stated packet size is greater than uint32_t"
+		    "at offset %llu", (u_longlong_t)context->ic_offset);
+	}
+	item->dp_payload_size = payload_size;
+	if (item->dp_payload_size > 0) {
+		item->dp_payload = safe_malloc(item->dp_payload_size);
+		size_t n_read = fread(item->dp_payload, item->dp_payload_size,
+		    1, context->ic_fp);
+		if (n_read != 1) {
+			if (ferror(context->ic_fp)) {
+				err(1, "error reading record payload at "
+				    " offset %llu",
+				    (u_longlong_t)context->ic_offset);
+			} else {
+				/*
+				 * We can't exit here because the ZFS test
+				 * suite depends on being able to process
+				 * streams truncated at random places.
+				 */
+				warnx("input ends mid-record at offset %llu "
+				    "- stream is likely corrupt",
+				    (u_longlong_t)context->ic_offset);
+				fclose(context->ic_fp);
+				free(item->dp_payload);
+				return (D_EOF);
+			}
+		}
+	} else {
+		item->dp_payload = NULL;
+	}
+	item->dp_stream_offset = context->ic_offset;
+
+	uint32_t drr_type = ATTR_IS_SET(CA_BYTESWAPPED) ?
+	    BSWAP_32(drr->drr_type) : drr->drr_type;
+
+	if (drr_type >= DRR_NUMTYPES) {
+		err(1, "invalid record type %llu found at offset %llu",
+		    (u_longlong_t)drr_type, (u_longlong_t)context->ic_offset);
+	}
+
+	context->ic_offset += sizeof (*drr) + item->dp_payload_size;
+
+	record_stats_t *stats = &chain_attrs->ca_stats_in[drr_type];
+	stats->rs_num_records++;
+	stats->rs_total_header_bytes += sizeof (dmu_replay_record_t);
+	stats->rs_total_payload_bytes += item->dp_payload_size;
+
+	stats = &chain_attrs->ca_totals_in;
+	stats->rs_num_records++;
+	stats->rs_total_header_bytes += sizeof (dmu_replay_record_t);
+	stats->rs_total_payload_bytes += item->dp_payload_size;
+
+	return (D_OK);
+}
+
+static disposition_t
+chain_write(drr_packet_t *item, io_context_t *context)
+{
+	if (item == NULL) {
+		if (context->ic_fp) {
+			if (fclose(context->ic_fp) != 0)
+				err(1, "error closing output stream");
+			context->ic_fp = NULL;
+		}
+		return (D_OK);
+	}
+
+	if (!context->ic_fp) {
+		open_file(context);
+	}
+
+	dmu_replay_record_t *drr = &item->dp_drr;
+
+	if (fwrite(drr, sizeof (dmu_replay_record_t), 1, context->ic_fp) != 1) {
+		err(1, "error writing record header");
+	} else if (item->dp_payload_size > 0) {
+		size_t n_written = fwrite(item->dp_payload,
+		    item->dp_payload_size, 1, context->ic_fp);
+		if (n_written != 1) {
+			err(1, "error writing payload");
+		} else {
+			free(item->dp_payload);
+			item->dp_payload = NULL;
+		}
+	}
+
+	uint32_t drr_type = OPTION_ENABLED(CA_BYTESWAP_ON_OUTPUT) ?
+	    BSWAP_32(drr->drr_type) : drr->drr_type;
+
+	record_stats_t *stats = &chain_attrs->ca_stats_out[drr_type];
+	stats->rs_num_records++;
+	stats->rs_total_header_bytes += sizeof (dmu_replay_record_t);
+	stats->rs_total_payload_bytes += item->dp_payload_size;
+
+	stats = &chain_attrs->ca_totals_out;
+	stats->rs_num_records++;
+	stats->rs_total_header_bytes += sizeof (dmu_replay_record_t);
+	stats->rs_total_payload_bytes += item->dp_payload_size;
+
+	return (D_OK);
+}
+
+/*
+ * Even if the chain doesn't write out a stream, payloads still need freed.
+ */
+static disposition_t
+chain_null_output(drr_packet_t *item, void *context)
+{
+	(void) context;
+	if (item && item->dp_payload != NULL && item->dp_payload_size > 0) {
+		free(item->dp_payload);
+		item->dp_payload = NULL;
+		item->dp_payload_size = 0;
+	}
+	return (D_OK);
+}
+
+/*
+ * Storage for the filename must remain valid during chain execution
+ */
+static chain_step_t
+setup_io(const char *filename, boolean_t for_reading)
+{
+	int context_num = next_io_context++ % MAX_IO_STREAMS;
+
+	io_context_t context = {
+		.ic_filename = filename,
+		.ic_for_reading = for_reading
+	};
+	io_contexts[context_num] = context;
+
+	chain_step_t step = {
+		.cs_type = CS_SERIAL,
+		.cs_in_size = 0,
+		.cs_out_size = sizeof (drr_packet_t),
+		.cs_context = &io_contexts[context_num],
+		.cs_serial = {
+			.process = (zc_serial_process_f *)
+			    (for_reading ? chain_read : chain_write),
+		}
+	};
+	return (step);
+}
+
+chain_step_t
+serial_read_stream(const char *filename)
+{
+	return (setup_io(filename, B_TRUE));
+}
+
+chain_step_t
+serial_write_stream(const char *filename)
+{
+	return (setup_io(filename, B_FALSE));
+}
+
+chain_step_t
+serial_null_output(void)
+{
+	chain_step_t step = {
+		.cs_type = CS_SERIAL,
+		.cs_in_size = sizeof (drr_packet_t),
+		.cs_out_size = 0,
+		.cs_context = NULL,
+		.cs_serial = {
+			.process = (zc_serial_process_f *)chain_null_output
+		}
+	};
+	return (step);
+}
+
+static disposition_t
+chain_checkpoint(drr_packet_t *item, checkpoint_context_t *ctxt)
+{
+	struct timespec now;
+	char buff[32];
+	uint64_t delta_b, dbdt;
+	double now_sec, delta_t;
+
+	if (item == NULL)
+		return (D_OK);
+
+	clock_gettime(CLOCK_MONOTONIC, &now);
+	now_sec = now.tv_sec + (double)now.tv_nsec / 1E9;
+	if (ctxt->cc_last_sec > 1E-9) {
+		delta_t = now_sec - ctxt->cc_last_sec;
+		if (delta_t < ctxt->cc_period_sec)
+			return (D_OK);
+		delta_b = item->dp_stream_offset - ctxt->cc_last_bytes;
+		dbdt = delta_b / delta_t;
+		zfs_nicenum(dbdt, buff, sizeof (buff));
+		fprintf(stderr, "Checkpoint %s: %s/s\n", ctxt->cc_name, buff);
+	}
+	ctxt->cc_last_sec = now_sec;
+	ctxt->cc_last_bytes = item->dp_stream_offset;
+	return (D_OK);
+}
+
+/*
+ * Storage for name must remain valid throughout chain execution
+ */
+chain_step_t
+serial_checkpoint(const char *name)
+{
+	int context_no = next_checkpoint_context++ % MAX_IO_STREAMS;
+
+	checkpoint_context_t context = {
+		.cc_name = name,
+		.cc_period_sec = 1.0
+	};
+	checkpoint_contexts[context_no] = context;
+
+	chain_step_t step = {
+		.cs_type = CS_SERIAL,
+		.cs_in_size = sizeof (drr_packet_t),
+		.cs_out_size = sizeof (drr_packet_t),
+		.cs_context = &checkpoint_contexts[context_no],
+		.cs_serial = {
+			.process = (zc_serial_process_f *)chain_checkpoint
+		},
+	};
+	return (step);
+}

cmd/zstream/zstream_io.h

@@ -0,0 +1,74 @@
+// SPDX-License-Identifier: CDDL-1.0
+/*
+ * CDDL HEADER START
+ *
+ * This file and its contents are supplied under the terms of the Common
+ * Development and Distribution License ("CDDL"), version 1.0. You may only use
+ * this file in accordance with the terms of version 1.0 of the CDDL.
+ *
+ * A full copy of the text of the CDDL should have accompanied this source. A
+ * copy of the CDDL is also available via the Internet at
+ * http://www.illumos.org/license/CDDL.
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Copyright (c) 2026 by Garth Snyder. All rights reserved.
+ */
+
+#ifndef _ZSTREAM_IO_H
+#define	_ZSTREAM_IO_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <sys/types.h>
+#include <sys/zfs_ioctl.h>
+
+#include "zstream_chain.h"
+
+#define	MAX_IO_STREAMS 4
+
+/*
+ * The stream offset is the offset within the original source stream.
+ * Changes to the stream (e.g., recompression) will necessarily change
+ * offsets within the final stream. The original stream offset is raw data;
+ * it should never be updated.
+ */
+typedef struct {
+	dmu_replay_record_t	dp_drr;
+	uint8_t			*dp_payload;
+	uint32_t		dp_payload_size;
+	off_t			dp_stream_offset;
+} drr_packet_t;
+
+/*
+ * In the following, the filename or checkpoint names must remain valid
+ * as long as the chain is executing.
+ */
+
+chain_step_t
+serial_read_stream(const char *filename);
+
+chain_step_t
+serial_write_stream(const char *filename);
+
+/* Report throughput periodically */
+chain_step_t
+serial_checkpoint(const char *name);
+
+/*
+ * Usually the output step is responsible for freeing payloads. Subcommands
+ * that don't have stream outputs still need to free this memory. A
+ * serial_null_output step does this and nothing more.
+ */
+chain_step_t
+serial_null_output(void);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif  /* _ZSTREAM_IO_H */

cmd/zstream/zstream_modules.h

@@ -0,0 +1,62 @@
+// SPDX-License-Identifier: CDDL-1.0
+/*
+ * CDDL HEADER START
+ *
+ * This file and its contents are supplied under the terms of the Common
+ * Development and Distribution License ("CDDL"), version 1.0. You may only use
+ * this file in accordance with the terms of version 1.0 of the CDDL.
+ *
+ * A full copy of the text of the CDDL should have accompanied this source. A
+ * copy of the CDDL is also available via the Internet at
+ * http://www.illumos.org/license/CDDL.
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Copyright (c) 2026 by Garth Snyder. All rights reserved.
+ */
+
+#ifndef _ZSTREAM_MODULES_H
+#define	_ZSTREAM_MODULES_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/*
+ * This file aggregates all zstream_chain utility modules into a single
+ * header and defines macros for standard input and output operations.
+ */
+
+#include "zstream_byteswap.h"
+#include "zstream_chain.h"
+#include "zstream_fletcher4.h"
+#include "zstream_io.h"
+#include "zstream_recompress.h"
+#include "zstream_util.h"
+#include "zstream_validate.h"
+
+#define	READ_STEP 0
+
+#define	STANDARD_INPUT_STACK(infile)					\
+	serial_read_stream(infile),					\
+	serial_validate_fletcher4(),					\
+	serial_byteswap(BS_INCOMING),					\
+	serial_validate_records()
+
+#define	STANDARD_OUTPUT_STACK(outfile)					\
+	serial_byteswap(BS_OUTGOING),					\
+	serial_add_fletcher4(),						\
+	serial_write_stream(outfile),					\
+	chain_terminator()
+
+#define	NULL_OUTPUT_STACK()						\
+	serial_null_output(),						\
+	chain_terminator()
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif  /* _ZSTREAM_MODULES_H */

cmd/zstream/zstream_recompress.c

@@ -26,43 +26,235 @@
  *
  * Copyright (c) 2022 by Delphix. All rights reserved.
  * Copyright (c) 2024, Klara, Inc.
+ * Copyright (c) 2026 by Garth Snyder
  */
 
+#include <assert.h>
 #include <err.h>
+#include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
-#include <unistd.h>
+#include <string.h>
 #include <sys/zfs_ioctl.h>
-#include <sys/zio_checksum.h>
+#include <sys/zio_compress.h>
 #include <sys/zstd/zstd.h>
-#include "zfs_fletcher.h"
+#include <unistd.h>
+#include <sys/stdtypes.h>
+
 #include "zstream.h"
+#include "zstream_chain.h"
+#include "zstream_modules.h"
 #include "zstream_util.h"
 
+#define	MAX_COMPRESSION_STEPS 4
+
+static compression_spec_t	specs[MAX_COMPRESSION_STEPS];
+static int			next_spec = 0;
+
+/*
+ * Item is known to be a DRR_WRITE packet. Determine whether current
+ * compression is compatible with desired compression and whether the
+ * current record is modifiable at all.
+ */
+static boolean_t
+needs_modification(drr_packet_t *item, compression_spec_t *target)
+{
+	dmu_replay_record_t *drr = &item->dp_drr;
+	struct drr_write *drrw	 = &drr->drr_u.drr_write;
+	enum zio_compress ctype  = drrw->drr_compressiontype;
+	uint8_t cur_level;
+
+	/*
+	 * Do not modify metadata records. It's a general stream invariant
+	 * that metadata is never compressed. See comments at
+	 * dmu_receive.c:flush_write_batch_impl().
+	 */
+	if (DMU_OT_IS_METADATA(drrw->drr_type)) {
+		return (B_FALSE);
+	}
+	boolean_t ctype_uncompressed = ctype_is_uncompressed(ctype);
+	if (target == NULL) {
+		return (!ctype_uncompressed && !write_is_encrypted(drrw));
+	}
+	boolean_t target_uncompressed = ctype_is_uncompressed(target->cs_type);
+	if (target_uncompressed && ctype_uncompressed) {
+		return (B_FALSE);
+	}
+	/*
+	 * In order to recompress an encrypted block, you have to decrypt,
+	 * decompress, recompress, and re-encrypt. That can be a future
+	 * enhancement (along with decryption or re-encryption), but for now
+	 * we skip encrypted blocks.
+	 */
+	if (write_is_encrypted(drrw)) {
+		return (B_FALSE);
+	}
+	if (ctype != target->cs_type) {
+		return (B_TRUE);
+	}
+	if (target->cs_type == ZIO_COMPRESS_ZSTD) {
+		cur_level = zfs_get_hdrlevel((void *)item->dp_payload);
+		if (target->cs_level == ZIO_COMPLEVEL_DEFAULT) {
+			return (cur_level != ZIO_ZSTD_LEVEL_DEFAULT);
+		}
+		return (target->cs_level != cur_level);
+	}
+	return (B_FALSE);
+}
+
+static boolean_t
+needs_compression(drr_packet_t *item, compression_spec_t *context)
+{
+	return (needs_modification(item, context));
+}
+
+/*
+ * Don't decompress packets that aren't compressed. And don't decompress
+ * them if their ultimate fate is to be recompressed using the compression
+ * profile that's already in use.
+ */
+static boolean_t
+needs_decompression(drr_packet_t *item, compression_spec_t *context)
+{
+	dmu_replay_record_t *drr = &item->dp_drr;
+	struct drr_write *drrw	 = &drr->drr_u.drr_write;
+	enum zio_compress ctype	 = drrw->drr_compressiontype;
+
+	if (ctype_is_uncompressed(ctype))
+		return (B_FALSE);
+	return (needs_modification(item, context));
+}
+
+static disposition_t
+chain_decompress_writes(drr_packet_t *item, compression_spec_t *context)
+{
+	if (item == NULL)
+		return (D_OK);
+
+	dmu_replay_record_t *drr = &item->dp_drr;
+	struct drr_write *drrw	 = &drr->drr_u.drr_write;
+	uint8_t *debuff;
+
+	if (drr->drr_type != DRR_WRITE || !needs_decompression(item, context)) {
+		return (D_OK);
+	}
+
+	debuff = decompress_buffer(item->dp_payload, item->dp_payload_size,
+	    drrw->drr_logical_size, drrw->drr_compressiontype);
+	if (debuff == NULL) {
+		errx(4, "decompression type %d failed for ino %llu offset %llu",
+		    drrw->drr_compressiontype,
+		    (u_longlong_t)drrw->drr_object,
+		    (u_longlong_t)drrw->drr_offset);
+	}
+	free(item->dp_payload);
+	item->dp_payload = debuff;
+	item->dp_payload_size = drrw->drr_logical_size;
+	drrw->drr_compressed_size = 0;
+	drrw->drr_compressiontype = 0;
+	return (D_OK);
+}
+
+static disposition_t
+chain_compress_writes(drr_packet_t *item, compression_spec_t *context)
+{
+	if (item == NULL)
+		return (D_OK);
+
+	dmu_replay_record_t *drr = &item->dp_drr;
+
+	if (drr->drr_type != DRR_WRITE || !needs_compression(item, context)) {
+		return (D_OK);
+	}
+
+	struct drr_write *drrw  = &drr->drr_u.drr_write;
+	enum zio_compress ctype = drrw->drr_compressiontype;
+	uint8_t *cbuff;
+	size_t	csize;
+
+	VERIFY3B(ctype_is_uncompressed(ctype), ==, B_TRUE);
+	cbuff = compress_buffer(item->dp_payload, item->dp_payload_size,
+	    *context, &csize);
+	if (cbuff == NULL) {
+		drrw->drr_compressiontype = 0;
+		drrw->drr_compressed_size = 0;
+	} else {
+		free(item->dp_payload);
+		item->dp_payload = cbuff;
+		item->dp_payload_size = csize;
+		drrw->drr_compressed_size = csize;
+		drrw->drr_compressiontype = context->cs_type;
+	}
+	return (D_OK);
+}
+
+/*
+ * Decompress writes, but only if they don't match a target compression
+ * type. Pass NULL to uncompress unconditionally (if not already
+ * uncompressed).
+ */
+chain_step_t
+serial_decompress_writes(compression_spec_t *target)
+{
+	int this_spec = next_spec++ % MAX_COMPRESSION_STEPS;
+	compression_spec_t *context = &specs[this_spec];
+
+	if (target == NULL) {
+		context = NULL;
+	} else {
+		*context = *target;
+	}
+	chain_step_t step = {
+		.cs_type = CS_SERIAL,
+		.cs_in_size = sizeof (drr_packet_t),
+		.cs_out_size = sizeof (drr_packet_t),
+		.cs_context = context,
+		.cs_serial = {
+		    .process = (zc_serial_process_f *)chain_decompress_writes
+		}
+	};
+	return (step);
+}
+
+chain_step_t
+serial_compress_writes(compression_spec_t *target)
+{
+	int this_spec = next_spec++ % MAX_COMPRESSION_STEPS;
+	compression_spec_t *context = &specs[this_spec];
+
+	VERIFY3P(target, !=, NULL);
+	*context = *target;
+	chain_step_t step = {
+		.cs_type = CS_SERIAL,
+		.cs_in_size = sizeof (drr_packet_t),
+		.cs_out_size = sizeof (drr_packet_t),
+		.cs_context = context,
+		.cs_serial = {
+			.process =
+			    (zc_serial_process_f *)chain_compress_writes
+		}
+	};
+	return (step);
+}
+
 int
 zstream_do_recompress(int argc, char *argv[])
 {
-	int bufsz = SPA_MAXBLOCKSIZE;
-	char *buf = safe_malloc(bufsz);
-	dmu_replay_record_t thedrr;
-	dmu_replay_record_t *drr = &thedrr;
-	zio_cksum_t stream_cksum;
 	int c;
-	int level = 0;
+	int level = ZIO_COMPLEVEL_DEFAULT;
+
+	chain_attrs_t attrs = { .ca_command_opts = CA_FORBID_DEDUP };
 
 	while ((c = getopt(argc, argv, "l:")) != -1) {
 		switch (c) {
 		case 'l':
 			if (sscanf(optarg, "%d", &level) != 1) {
-				fprintf(stderr,
-				    "failed to parse level '%s'\n",
-				    optarg);
+				warnx("failed to parse level '%s'", optarg);
 				zstream_usage();
 			}
 			break;
 		case '?':
-			(void) fprintf(stderr, "invalid option '%c'\n",
-			    optopt);
+			warnx("invalid option '%c'", optopt);
 			zstream_usage();
 			break;
 		}
@@ -74,287 +266,29 @@ zstream_do_recompress(int argc, char *argv[])
 	if (argc != 1)
 		zstream_usage();
 
-	enum zio_compress ctype;
+	compression_spec_t spec = { .cs_level = level };
 	if (strcmp(argv[0], "off") == 0) {
-		ctype = ZIO_COMPRESS_OFF;
+		spec.cs_type = ZIO_COMPRESS_OFF;
 	} else {
-		for (ctype = 0; ctype < ZIO_COMPRESS_FUNCTIONS; ctype++) {
-			if (strcmp(argv[0],
-			    zio_compress_table[ctype].ci_name) == 0)
+		enum zio_compress ct;
+		for (ct = 0; ct < ZIO_COMPRESS_FUNCTIONS; ct++) {
+			const char *ci_name = zio_compress_table[ct].ci_name;
+			if (strcmp(argv[0], ci_name) == 0)
 				break;
 		}
-		if (ctype == ZIO_COMPRESS_FUNCTIONS ||
-		    zio_compress_table[ctype].ci_compress == NULL) {
-			fprintf(stderr, "Invalid compression type %s.\n",
-			    argv[0]);
-			exit(2);
+		if (ct == ZIO_COMPRESS_FUNCTIONS || ctype_is_uncompressed(ct)) {
+			errx(2, "invalid compression type %s", argv[0]);
 		}
+		spec.cs_type = ct;
 	}
 
-	if (isatty(STDIN_FILENO)) {
-		(void) fprintf(stderr,
-		    "Error: The send stream is a binary format "
-		    "and can not be read from a\n"
-		    "terminal.  Standard input must be redirected.\n");
-		exit(1);
-	}
-
-	zfs_refcount_init();
-	abd_init();
-	fletcher_4_init();
-	zio_init();
-	zstd_init();
-	int begin = 0;
-	boolean_t seen = B_FALSE;
-	while (sfread(drr, sizeof (*drr), stdin) != 0) {
-		struct drr_write *drrw;
-		uint64_t payload_size = 0;
-
-		/*
-		 * We need to regenerate the checksum.
-		 */
-		if (drr->drr_type != DRR_BEGIN) {
-			memset(&drr->drr_u.drr_checksum.drr_checksum, 0,
-			    sizeof (drr->drr_u.drr_checksum.drr_checksum));
-		}
-
-
-		switch (drr->drr_type) {
-		case DRR_BEGIN:
-		{
-			ZIO_SET_CHECKSUM(&stream_cksum, 0, 0, 0, 0);
-			VERIFY0(begin++);
-			seen = B_TRUE;
-
-			uint32_t sz = drr->drr_payloadlen;
-
-			VERIFY3U(sz, <=, 1U << 28);
-
-			if (sz != 0) {
-				if (sz > bufsz) {
-					buf = realloc(buf, sz);
-					if (buf == NULL)
-						err(1, "realloc");
-					bufsz = sz;
-				}
-				(void) sfread(buf, sz, stdin);
-			}
-			payload_size = sz;
-			break;
-		}
-		case DRR_END:
-		{
-			struct drr_end *drre = &drr->drr_u.drr_end;
-			/*
-			 * We would prefer to just check --begin == 0, but
-			 * replication streams have an end of stream END
-			 * record, so we must avoid tripping it.
-			 */
-			VERIFY3B(seen, ==, B_TRUE);
-			begin--;
-			/*
-			 * Use the recalculated checksum, unless this is
-			 * the END record of a stream package, which has
-			 * no checksum.
-			 */
-			if (!ZIO_CHECKSUM_IS_ZERO(&drre->drr_checksum))
-				drre->drr_checksum = stream_cksum;
-			break;
-		}
-
-		case DRR_OBJECT:
-		{
-			struct drr_object *drro = &drr->drr_u.drr_object;
-			VERIFY3S(begin, ==, 1);
-
-			if (drro->drr_bonuslen > 0) {
-				payload_size = DRR_OBJECT_PAYLOAD_SIZE(drro);
-				(void) sfread(buf, payload_size, stdin);
-			}
-			break;
-		}
-
-		case DRR_SPILL:
-		{
-			struct drr_spill *drrs = &drr->drr_u.drr_spill;
-			VERIFY3S(begin, ==, 1);
-			payload_size = DRR_SPILL_PAYLOAD_SIZE(drrs);
-			(void) sfread(buf, payload_size, stdin);
-			break;
-		}
-
-		case DRR_WRITE_BYREF:
-			VERIFY3S(begin, ==, 1);
-			fprintf(stderr,
-			    "Deduplicated streams are not supported\n");
-			exit(1);
-			break;
-
-		case DRR_WRITE:
-		{
-			VERIFY3S(begin, ==, 1);
-			drrw = &thedrr.drr_u.drr_write;
-			payload_size = DRR_WRITE_PAYLOAD_SIZE(drrw);
-			/*
-			 * In order to recompress an encrypted block, you have
-			 * to decrypt, decompress, recompress, and
-			 * re-encrypt. That can be a future enhancement (along
-			 * with decryption or re-encryption), but for now we
-			 * skip encrypted blocks.
-			 */
-			boolean_t encrypted = B_FALSE;
-			for (int i = 0; i < ZIO_DATA_SALT_LEN; i++) {
-				if (drrw->drr_salt[i] != 0) {
-					encrypted = B_TRUE;
-					break;
-				}
-			}
-			if (encrypted) {
-				(void) sfread(buf, payload_size, stdin);
-				break;
-			}
-			enum zio_compress dtype = drrw->drr_compressiontype;
-			if (dtype >= ZIO_COMPRESS_FUNCTIONS) {
-				fprintf(stderr, "Invalid compression type in "
-				    "stream: %d\n", dtype);
-				exit(3);
-			}
-			if (zio_compress_table[dtype].ci_decompress == NULL)
-				dtype = ZIO_COMPRESS_OFF;
-
-			/* Set up buffers to minimize memcpys */
-			char *cbuf, *dbuf;
-			if (ctype == ZIO_COMPRESS_OFF)
-				dbuf = buf;
-			else
-				dbuf = safe_calloc(bufsz);
-
-			if (dtype == ZIO_COMPRESS_OFF)
-				cbuf = dbuf;
-			else
-				cbuf = safe_calloc(payload_size);
-
-			/* Read and decompress the payload */
-			(void) sfread(cbuf, payload_size, stdin);
-			if (dtype != ZIO_COMPRESS_OFF) {
-				abd_t cabd, dabd;
-				abd_get_from_buf_struct(&cabd,
-				    cbuf, payload_size);
-				abd_get_from_buf_struct(&dabd, dbuf,
-				    MIN(bufsz, drrw->drr_logical_size));
-				if (zio_decompress_data(dtype, &cabd, &dabd,
-				    payload_size, abd_get_size(&dabd),
-				    NULL) != 0) {
-					warnx("decompression type %d failed "
-					    "for ino %llu offset %llu",
-					    dtype,
-					    (u_longlong_t)drrw->drr_object,
-					    (u_longlong_t)drrw->drr_offset);
-					exit(4);
-				}
-				payload_size = drrw->drr_logical_size;
-				abd_free(&dabd);
-				abd_free(&cabd);
-				free(cbuf);
-			}
-
-			/* Recompress the payload */
-			if (ctype != ZIO_COMPRESS_OFF) {
-				abd_t dabd, abd;
-				abd_get_from_buf_struct(&dabd,
-				    dbuf, drrw->drr_logical_size);
-				abd_t *pabd =
-				    abd_get_from_buf_struct(&abd, buf, bufsz);
-				size_t csize = zio_compress_data(ctype, &dabd,
-				    &pabd, drrw->drr_logical_size,
-				    drrw->drr_logical_size, level);
-				size_t rounded =
-				    P2ROUNDUP(csize, SPA_MINBLOCKSIZE);
-				if (rounded >= drrw->drr_logical_size) {
-					memcpy(buf, dbuf, payload_size);
-					drrw->drr_compressiontype = 0;
-					drrw->drr_compressed_size = 0;
-				} else {
-					abd_zero_off(pabd, csize,
-					    rounded - csize);
-					drrw->drr_compressiontype = ctype;
-					drrw->drr_compressed_size =
-					    payload_size = rounded;
-				}
-				abd_free(&abd);
-				abd_free(&dabd);
-				free(dbuf);
-			} else {
-				drrw->drr_compressiontype = 0;
-				drrw->drr_compressed_size = 0;
-			}
-			break;
-		}
-
-		case DRR_WRITE_EMBEDDED:
-		{
-			struct drr_write_embedded *drrwe =
-			    &drr->drr_u.drr_write_embedded;
-			VERIFY3S(begin, ==, 1);
-			payload_size =
-			    P2ROUNDUP((uint64_t)drrwe->drr_psize, 8);
-			(void) sfread(buf, payload_size, stdin);
-			break;
-		}
-
-		case DRR_FREEOBJECTS:
-		case DRR_FREE:
-		case DRR_OBJECT_RANGE:
-			VERIFY3S(begin, ==, 1);
-			break;
-
-		default:
-			(void) fprintf(stderr, "INVALID record type 0x%x\n",
-			    drr->drr_type);
-			/* should never happen, so assert */
-			assert(B_FALSE);
-		}
-
-		if (feof(stdout)) {
-			fprintf(stderr, "Error: unexpected end-of-file\n");
-			exit(1);
-		}
-		if (ferror(stdout)) {
-			fprintf(stderr, "Error while reading file: %s\n",
-			    strerror(errno));
-			exit(1);
-		}
-
-		/*
-		 * We need to recalculate the checksum, and it needs to be
-		 * initially zero to do that.  BEGIN records don't have
-		 * a checksum.
-		 */
-		if (drr->drr_type != DRR_BEGIN) {
-			memset(&drr->drr_u.drr_checksum.drr_checksum, 0,
-			    sizeof (drr->drr_u.drr_checksum.drr_checksum));
-		}
-		if (dump_record(drr, buf, payload_size,
-		    &stream_cksum, STDOUT_FILENO) != 0)
-			break;
-		if (drr->drr_type == DRR_END) {
-			/*
-			 * Typically the END record is either the last
-			 * thing in the stream, or it is followed
-			 * by a BEGIN record (which also zeros the checksum).
-			 * However, a stream package ends with two END
-			 * records.  The last END record's checksum starts
-			 * from zero.
-			 */
-			ZIO_SET_CHECKSUM(&stream_cksum, 0, 0, 0, 0);
-		}
-	}
-	free(buf);
-	fletcher_4_fini();
-	zio_fini();
-	zstd_fini();
-	abd_fini();
-	zfs_refcount_fini();
+	zstream_chain_t recompress_chain = {
+		STANDARD_INPUT_STACK(NULL),
+		serial_decompress_writes(&spec),
+		serial_compress_writes(&spec),
+		STANDARD_OUTPUT_STACK(NULL)
+	};
 
+	zstream_chain_exec(recompress_chain, &attrs);
 	return (0);
 }

cmd/zstream/zstream_recompress.h

@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: CDDL-1.0
+/*
+ * CDDL HEADER START
+ *
+ * This file and its contents are supplied under the terms of the Common
+ * Development and Distribution License ("CDDL"), version 1.0. You may only use
+ * this file in accordance with the terms of version 1.0 of the CDDL.
+ *
+ * A full copy of the text of the CDDL should have accompanied this source. A
+ * copy of the CDDL is also available via the Internet at
+ * http://www.illumos.org/license/CDDL.
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Copyright (c) 2026 by Garth Snyder. All rights reserved.
+ */
+
+#ifndef _ZSTREAM_RECOMPRESS_H
+#define	_ZSTREAM_RECOMPRESS_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "zstream_util.h"
+#include "zstream_io.h"
+
+chain_step_t
+serial_decompress_writes(compression_spec_t *target);
+
+chain_step_t
+serial_compress_writes(compression_spec_t *target);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif  /* _ZSTREAM_RECOMPRESS_H */

cmd/zstream/zstream_redup.c

@@ -20,35 +20,34 @@
 
 #include <assert.h>
 #include <cityhash.h>
-#include <ctype.h>
+#include <err.h>
 #include <errno.h>
-#include <fcntl.h>
-#include <libzfs.h>
 #include <libzutil.h>
-#include <stddef.h>
+#include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+#include <sys/bitops.h>
+#include <sys/param.h>
+#include <sys/stdtypes.h>
+#include <sys/sysmacros.h>
+#include <sys/zfs_ioctl.h>
 #include <umem.h>
 #include <unistd.h>
-#include <sys/debug.h>
-#include <sys/stat.h>
-#include <sys/zfs_ioctl.h>
-#include <sys/zio_checksum.h>
-#include "zfs_fletcher.h"
+
 #include "zstream.h"
+#include "zstream_modules.h"
 #include "zstream_util.h"
 
-
 #define	MAX_RDT_PHYSMEM_PERCENT		20
-#define	SMALLEST_POSSIBLE_MAX_RDT_MB		128
+#define	SMALLEST_POSSIBLE_MAX_RDT_MB	128
 
 typedef struct redup_entry {
 	struct redup_entry	*rde_next;
-	uint64_t rde_guid;
-	uint64_t rde_object;
-	uint64_t rde_offset;
-	uint64_t rde_stream_offset;
+	uint64_t		rde_guid;
+	uint64_t		rde_object;
+	uint64_t		rde_offset;
+	uint64_t		rde_stream_offset;
 } redup_entry_t;
 
 typedef struct redup_table {
@@ -58,24 +57,10 @@ typedef struct redup_table {
 	int		numhashbits;
 } redup_table_t;
 
-/*
- * Safe version of pread(), exits on error.
- */
-static void
-spread(int fd, void *buf, size_t count, off_t offset)
-{
-	ssize_t err = pread(fd, buf, count, offset);
-	if (err == -1) {
-		(void) fprintf(stderr,
-		    "Error while reading file: %s\n",
-		    strerror(errno));
-		exit(1);
-	} else if (err != count) {
-		(void) fprintf(stderr,
-		    "Error while reading file: short read\n");
-		exit(1);
-	}
-}
+typedef struct {
+	redup_table_t	rc_rdt;
+	FILE		*rc_fp;
+} redup_context_t;
 
 static void
 rdt_insert(redup_table_t *rdt,
@@ -116,283 +101,114 @@ rdt_lookup(redup_table_t *rdt,
 	assert(!"could not find expected redup table entry");
 }
 
-/*
- * Convert a dedup stream (generated by "zfs send -D") to a
- * non-deduplicated stream.  The entire infd will be converted, including
- * any substreams in a stream package (generated by "zfs send -RD"). The
- * infd must be seekable.
- */
-static void
-zfs_redup_stream(int infd, int outfd, boolean_t verbose)
+static disposition_t
+chain_redup_writes(drr_packet_t *item, redup_context_t *context)
 {
-	int bufsz = SPA_MAXBLOCKSIZE;
-	dmu_replay_record_t thedrr;
-	dmu_replay_record_t *drr = &thedrr;
-	redup_table_t rdt;
-	zio_cksum_t stream_cksum;
-	uint64_t numbuckets;
-	uint64_t num_records = 0;
-	uint64_t num_write_byref_records = 0;
-
-	memset(&thedrr, 0, sizeof (dmu_replay_record_t));
-
-#ifdef _ILP32
-	uint64_t max_rde_size = SMALLEST_POSSIBLE_MAX_RDT_MB << 20;
-#else
-	uint64_t physbytes = sysconf(_SC_PHYS_PAGES) * sysconf(_SC_PAGESIZE);
-	uint64_t max_rde_size =
-	    MAX((physbytes * MAX_RDT_PHYSMEM_PERCENT) / 100,
-	    SMALLEST_POSSIBLE_MAX_RDT_MB << 20);
-#endif
-
-	numbuckets = max_rde_size / (sizeof (redup_entry_t));
-
-	/*
-	 * numbuckets must be a power of 2.  Increase number to
-	 * a power of 2 if necessary.
-	 */
-	if (!ISP2(numbuckets))
-		numbuckets = 1ULL << highbit64(numbuckets);
-
-	rdt.redup_hash_array =
-	    safe_calloc(numbuckets * sizeof (redup_entry_t *));
-	rdt.ddecache = umem_cache_create("rde", sizeof (redup_entry_t), 0,
-	    NULL, NULL, NULL, NULL, NULL, 0);
-	rdt.numhashbits = highbit64(numbuckets) - 1;
-	rdt.ddt_count = 0;
-
-	char *buf = safe_calloc(bufsz);
-	FILE *ofp = fdopen(infd, "r");
-	long offset = ftell(ofp);
-	int begin = 0;
-	boolean_t seen = B_FALSE;
-	while (sfread(drr, sizeof (*drr), ofp) != 0) {
-		num_records++;
-
-		/*
-		 * We need to regenerate the checksum.
-		 */
-		if (drr->drr_type != DRR_BEGIN) {
-			memset(&drr->drr_u.drr_checksum.drr_checksum, 0,
-			    sizeof (drr->drr_u.drr_checksum.drr_checksum));
-		}
-
-		uint64_t payload_size = 0;
-		switch (drr->drr_type) {
-		case DRR_BEGIN:
-		{
-			struct drr_begin *drrb = &drr->drr_u.drr_begin;
-			int fflags;
-			ZIO_SET_CHECKSUM(&stream_cksum, 0, 0, 0, 0);
-			VERIFY0(begin++);
-			seen = B_TRUE;
-
-			assert(drrb->drr_magic == DMU_BACKUP_MAGIC);
-
-			/* clear the DEDUP feature flag for this stream */
-			fflags = DMU_GET_FEATUREFLAGS(drrb->drr_versioninfo);
-			fflags &= ~(DMU_BACKUP_FEATURE_DEDUP |
-			    DMU_BACKUP_FEATURE_DEDUPPROPS);
-			/* cppcheck-suppress syntaxError */
-			DMU_SET_FEATUREFLAGS(drrb->drr_versioninfo, fflags);
-
-			uint32_t sz = drr->drr_payloadlen;
-
-			VERIFY3U(sz, <=, 1U << 28);
-
-			if (sz != 0) {
-				if (sz > bufsz) {
-					free(buf);
-					buf = safe_calloc(sz);
-					bufsz = sz;
-				}
-				(void) sfread(buf, sz, ofp);
-			}
-			payload_size = sz;
-			break;
-		}
-
-		case DRR_END:
-		{
-			struct drr_end *drre = &drr->drr_u.drr_end;
-			/*
-			 * We would prefer to just check --begin == 0, but
-			 * replication streams have an end of stream END
-			 * record, so we must avoid tripping it.
-			 */
-			VERIFY3B(seen, ==, B_TRUE);
-			begin--;
-			/*
-			 * Use the recalculated checksum, unless this is
-			 * the END record of a stream package, which has
-			 * no checksum.
-			 */
-			if (!ZIO_CHECKSUM_IS_ZERO(&drre->drr_checksum))
-				drre->drr_checksum = stream_cksum;
-			break;
-		}
-
-		case DRR_OBJECT:
-		{
-			struct drr_object *drro = &drr->drr_u.drr_object;
-			VERIFY3S(begin, ==, 1);
-
-			if (drro->drr_bonuslen > 0) {
-				payload_size = DRR_OBJECT_PAYLOAD_SIZE(drro);
-				(void) sfread(buf, payload_size, ofp);
-			}
-			break;
-		}
-
-		case DRR_SPILL:
-		{
-			struct drr_spill *drrs = &drr->drr_u.drr_spill;
-			VERIFY3S(begin, ==, 1);
-			payload_size = DRR_SPILL_PAYLOAD_SIZE(drrs);
-			(void) sfread(buf, payload_size, ofp);
-			break;
-		}
-
-		case DRR_WRITE_BYREF:
-		{
-			struct drr_write_byref drrwb =
-			    drr->drr_u.drr_write_byref;
-			VERIFY3S(begin, ==, 1);
-
-			num_write_byref_records++;
-
-			/*
-			 * Look up in hash table by drrwb->drr_refguid,
-			 * drr_refobject, drr_refoffset.  Replace this
-			 * record with the found WRITE record, but with
-			 * drr_object,drr_offset,drr_toguid replaced with ours.
-			 */
-			uint64_t stream_offset = 0;
-			rdt_lookup(&rdt, drrwb.drr_refguid,
-			    drrwb.drr_refobject, drrwb.drr_refoffset,
-			    &stream_offset);
-
-			spread(infd, drr, sizeof (*drr), stream_offset);
-
-			assert(drr->drr_type == DRR_WRITE);
-			struct drr_write *drrw = &drr->drr_u.drr_write;
-			assert(drrw->drr_toguid == drrwb.drr_refguid);
-			assert(drrw->drr_object == drrwb.drr_refobject);
-			assert(drrw->drr_offset == drrwb.drr_refoffset);
-
-			payload_size = DRR_WRITE_PAYLOAD_SIZE(drrw);
-			spread(infd, buf, payload_size,
-			    stream_offset + sizeof (*drr));
-
-			drrw->drr_toguid = drrwb.drr_toguid;
-			drrw->drr_object = drrwb.drr_object;
-			drrw->drr_offset = drrwb.drr_offset;
-			break;
-		}
-
-		case DRR_WRITE:
-		{
-			struct drr_write *drrw = &drr->drr_u.drr_write;
-			VERIFY3S(begin, ==, 1);
-			payload_size = DRR_WRITE_PAYLOAD_SIZE(drrw);
-			(void) sfread(buf, payload_size, ofp);
-
-			rdt_insert(&rdt, drrw->drr_toguid,
-			    drrw->drr_object, drrw->drr_offset, offset);
-			break;
-		}
-
-		case DRR_WRITE_EMBEDDED:
-		{
-			struct drr_write_embedded *drrwe =
-			    &drr->drr_u.drr_write_embedded;
-			VERIFY3S(begin, ==, 1);
-			payload_size =
-			    P2ROUNDUP((uint64_t)drrwe->drr_psize, 8);
-			(void) sfread(buf, payload_size, ofp);
-			break;
-		}
-
-		case DRR_FREEOBJECTS:
-		case DRR_FREE:
-		case DRR_OBJECT_RANGE:
-			VERIFY3S(begin, ==, 1);
-			break;
-
-		default:
-			(void) fprintf(stderr, "INVALID record type 0x%x\n",
-			    drr->drr_type);
-			/* should never happen, so assert */
-			assert(B_FALSE);
-		}
-
-		if (feof(ofp)) {
-			fprintf(stderr, "Error: unexpected end-of-file\n");
-			exit(1);
-		}
-		if (ferror(ofp)) {
-			fprintf(stderr, "Error while reading file: %s\n",
-			    strerror(errno));
-			exit(1);
-		}
-
-		/*
-		 * We need to recalculate the checksum, and it needs to be
-		 * initially zero to do that.  BEGIN records don't have
-		 * a checksum.
-		 */
-		if (drr->drr_type != DRR_BEGIN) {
-			memset(&drr->drr_u.drr_checksum.drr_checksum, 0,
-			    sizeof (drr->drr_u.drr_checksum.drr_checksum));
-		}
-		if (dump_record(drr, buf, payload_size,
-		    &stream_cksum, outfd) != 0)
-			break;
-		if (drr->drr_type == DRR_END) {
-			/*
-			 * Typically the END record is either the last
-			 * thing in the stream, or it is followed
-			 * by a BEGIN record (which also zeros the checksum).
-			 * However, a stream package ends with two END
-			 * records.  The last END record's checksum starts
-			 * from zero.
-			 */
-			ZIO_SET_CHECKSUM(&stream_cksum, 0, 0, 0, 0);
-		}
-		offset = ftell(ofp);
+	if (item == NULL) {
+		return (D_OK);
 	}
 
-	if (verbose) {
-		char mem_str[16];
-		zfs_nicenum(rdt.ddt_count * sizeof (redup_entry_t),
-		    mem_str, sizeof (mem_str));
-		fprintf(stderr, "converted stream with %llu total records, "
-		    "including %llu dedup records, using %sB memory.\n",
-		    (long long)num_records,
-		    (long long)num_write_byref_records,
-		    mem_str);
+	dmu_replay_record_t *drr = &item->dp_drr;
+	struct drr_write *drrw	 = &drr->drr_u.drr_write;
+	struct drr_begin *drrb	 = &drr->drr_u.drr_begin;
+
+	switch (drr->drr_type) {
+
+	case DRR_BEGIN:
+	{
+		uint64_t flags = DMU_GET_FEATUREFLAGS(drrb->drr_versioninfo);
+		flags &= ~(DMU_BACKUP_FEATURE_DEDUP |
+		    DMU_BACKUP_FEATURE_DEDUPPROPS);
+		DMU_SET_FEATUREFLAGS(drrb->drr_versioninfo, flags);
+		break;
 	}
 
-	umem_cache_destroy(rdt.ddecache);
-	free(rdt.redup_hash_array);
-	free(buf);
-	(void) fclose(ofp);
+	case DRR_WRITE_BYREF:
+	{
+		struct drr_write_byref drrwb = drr->drr_u.drr_write_byref;
+
+		/*
+		 * Look up in hash table by drrwb->drr_refguid,
+		 * drr_refobject, drr_refoffset.  Replace this
+		 * record with the found WRITE record, but with
+		 * drr_object,drr_offset,drr_toguid replaced with ours.
+		 */
+		uint64_t stream_offset = 0;
+		rdt_lookup(&context->rc_rdt, drrwb.drr_refguid,
+		    drrwb.drr_refobject, drrwb.drr_refoffset,
+		    &stream_offset);
+
+		if (fseeko(context->rc_fp, stream_offset, SEEK_SET) != 0) {
+			err(1, "seek into source file failed, offset %llu",
+			    (u_longlong_t)stream_offset);
+		}
+		if (fread(drr, sizeof (*drr), 1, context->rc_fp) != 1) {
+			err(1, "read of prior write failed");
+		}
+		if (ATTR_IS_SET(CA_BYTESWAPPED)) {
+			byteswap_record(drr, BSWAP_32(drr->drr_type));
+		}
+
+		VERIFY3U(drr->drr_type,    ==, DRR_WRITE);
+		VERIFY3U(drrw->drr_toguid, ==, drrwb.drr_refguid);
+		VERIFY3U(drrw->drr_object, ==, drrwb.drr_refobject);
+		VERIFY3U(drrw->drr_offset, ==, drrwb.drr_refoffset);
+
+		item->dp_payload_size = DRR_WRITE_PAYLOAD_SIZE(drrw);
+		item->dp_payload = safe_malloc(item->dp_payload_size);
+
+		size_t n_read = fread(item->dp_payload, item->dp_payload_size,
+		    1, context->rc_fp);
+		if (n_read != 1)
+			err(1, "read of prior payload failed");
+
+		drrw->drr_toguid = drrwb.drr_toguid;
+		drrw->drr_object = drrwb.drr_object;
+		drrw->drr_offset = drrwb.drr_offset;
+		break;
+	}
+
+	case DRR_WRITE:
+		rdt_insert(&context->rc_rdt, drrw->drr_toguid, drrw->drr_object,
+		    drrw->drr_offset, item->dp_stream_offset);
+		break;
+
+	default:
+		break;
+	}
+	return (D_OK);
+}
+
+static chain_step_t
+serial_redup_writes(redup_context_t *context)
+{
+	chain_step_t step = {
+		.cs_type = CS_SERIAL,
+		.cs_in_size = sizeof (drr_packet_t),
+		.cs_out_size = sizeof (drr_packet_t),
+		.cs_context = context,
+		.cs_serial = {
+			.process = (zc_serial_process_f *)chain_redup_writes
+		}
+	};
+	return (step);
 }
 
 int
 zstream_do_redup(int argc, char *argv[])
 {
-	boolean_t verbose = B_FALSE;
 	int c;
+	chain_attrs_t attrs = {0};
+	redup_context_t context = {0};
+	uint64_t numbuckets;
 
 	while ((c = getopt(argc, argv, "v")) != -1) {
 		switch (c) {
 		case 'v':
-			verbose = B_TRUE;
+			ENABLE_OPTION(&attrs, CA_VERBOSE);
 			break;
 		case '?':
-			(void) fprintf(stderr, "invalid option '%c'\n",
-			    optopt);
+			warnx("invalid option '%c'", optopt);
 			zstream_usage();
 			break;
 		}
@@ -404,28 +220,51 @@ zstream_do_redup(int argc, char *argv[])
 	if (argc != 1)
 		zstream_usage();
 
-	const char *filename = argv[0];
-
-	if (isatty(STDOUT_FILENO)) {
-		(void) fprintf(stderr,
-		    "Error: Stream can not be written to a terminal.\n"
-		    "You must redirect standard output.\n");
-		return (1);
+	context.rc_fp = fopen(argv[0], "rb");
+	if (context.rc_fp == NULL) {
+		err(1, "unable to open %s", argv[0]);
 	}
 
-	int fd = open(filename, O_RDONLY);
-	if (fd == -1) {
-		(void) fprintf(stderr,
-		    "Error while opening file '%s': %s\n",
-		    filename, strerror(errno));
-		exit(1);
+#ifdef _ILP32
+	uint64_t max_rde_size = SMALLEST_POSSIBLE_MAX_RDT_MB << 20;
+#else
+	uint64_t physbytes = sysconf(_SC_PHYS_PAGES) * sysconf(_SC_PAGESIZE);
+	uint64_t max_rde_size = MAX((physbytes * MAX_RDT_PHYSMEM_PERCENT) / 100,
+	    SMALLEST_POSSIBLE_MAX_RDT_MB << 20);
+#endif
+
+	numbuckets = max_rde_size / (sizeof (redup_entry_t));
+	if (!ISP2(numbuckets))
+		numbuckets = 1ULL << highbit64(numbuckets);
+
+	context.rc_rdt.redup_hash_array =
+	    safe_calloc(numbuckets * sizeof (redup_entry_t *));
+	context.rc_rdt.ddecache = umem_cache_create("rde",
+	    sizeof (redup_entry_t), 0, NULL, NULL, NULL, NULL, NULL, 0);
+	context.rc_rdt.numhashbits = highbit64(numbuckets) - 1;
+	context.rc_rdt.ddt_count = 0;
+
+	zstream_chain_t redup_chain = {
+		STANDARD_INPUT_STACK(argv[0]),
+		serial_redup_writes(&context),
+		STANDARD_OUTPUT_STACK(NULL)
+	};
+	zstream_chain_exec(redup_chain, &attrs);
+
+	if (attrs.ca_command_opts & CA_VERBOSE) {
+		char mem_str[16];
+		record_stats_t *acsi = attrs.ca_stats_in;
+		zfs_nicenum(context.rc_rdt.ddt_count * sizeof (redup_entry_t),
+		    mem_str, sizeof (mem_str));
+		fprintf(stderr, "Converted stream with %llu total records, "
+		    "including %llu dedup records, using %sB memory.\n",
+		    (u_longlong_t)attrs.ca_totals_in.rs_num_records,
+		    (u_longlong_t)acsi[DRR_WRITE_BYREF].rs_num_records,
+		    mem_str);
 	}
 
-	fletcher_4_init();
-	zfs_redup_stream(fd, STDOUT_FILENO, verbose);
-	fletcher_4_fini();
-
-	close(fd);
-
+	fclose(context.rc_fp);
+	umem_cache_destroy(context.rc_rdt.ddecache);
+	free(context.rc_rdt.redup_hash_array);
 	return (0);
 }

cmd/zstream/zstream_token.c

@@ -31,19 +31,12 @@
  * Copyright (c) 2020 by Datto Inc. All rights reserved.
  */
 
-#include <ctype.h>
+#include <errno.h>
 #include <libnvpair.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <unistd.h>
-#include <stddef.h>
-
 #include <libzfs.h>
-#include <libzfs_core.h>
+#include <stdio.h>
+#include <sys/nvpair.h>
 
-#include <sys/dmu.h>
-#include <sys/zfs_ioctl.h>
 #include "zstream.h"
 
 int

cmd/zstream/zstream_util.c

@@ -28,52 +28,31 @@
  * Copyright (c) 2024, Klara, Inc.
  */
 
-#include <sys/debug.h>
-#include <stddef.h>
+#include <assert.h>
+#include <err.h>
 #include <errno.h>
-#include <unistd.h>
-#include <stdlib.h>
+#include <stddef.h>
 #include <stdio.h>
+#include <stdlib.h>
 #include <string.h>
+#include <sys/abd.h>
+#include <sys/fs/zfs.h>
+#include <sys/stdtypes.h>
+#include <sys/sysmacros.h>
+#include <sys/zfs_ioctl.h>
+#include <sys/zio.h>
+#include <sys/zio_compress.h>
+#include <unistd.h>
 #include <zfs_fletcher.h>
-#include "zstream_util.h"
 
-/*
- * From libzfs_sendrecv.c
- */
-int
-dump_record(dmu_replay_record_t *drr, void *payload, size_t payload_len,
-    zio_cksum_t *zc, int outfd)
-{
-	ASSERT3U(offsetof(dmu_replay_record_t, drr_u.drr_checksum.drr_checksum),
-	    ==, sizeof (dmu_replay_record_t) - sizeof (zio_cksum_t));
-	fletcher_4_incremental_native(drr,
-	    offsetof(dmu_replay_record_t, drr_u.drr_checksum.drr_checksum), zc);
-	if (drr->drr_type != DRR_BEGIN) {
-		ASSERT(ZIO_CHECKSUM_IS_ZERO(&drr->drr_u.
-		    drr_checksum.drr_checksum));
-		drr->drr_u.drr_checksum.drr_checksum = *zc;
-	}
-	fletcher_4_incremental_native(&drr->drr_u.drr_checksum.drr_checksum,
-	    sizeof (zio_cksum_t), zc);
-	if (write(outfd, drr, sizeof (*drr)) == -1)
-		return (errno);
-	if (payload_len != 0) {
-		fletcher_4_incremental_native(payload, payload_len, zc);
-		if (write(outfd, payload, payload_len) == -1)
-			return (errno);
-	}
-	return (0);
-}
+#include "zstream_util.h"
 
 void *
 safe_malloc(size_t size)
 {
 	void *rv = malloc(size);
 	if (rv == NULL) {
-		(void) fprintf(stderr, "Error: failed to allocate %zu bytes\n",
-		    size);
-		exit(1);
+		errx(1, "failed to allocate %zu bytes, aborting...", size);
 	}
 	return (rv);
 }
@@ -83,24 +62,120 @@ safe_calloc(size_t size)
 {
 	void *rv = calloc(1, size);
 	if (rv == NULL) {
-		(void) fprintf(stderr,
-		    "Error: failed to allocate %zu bytes\n", size);
-		exit(1);
+		errx(1, "failed to allocate %zu bytes, aborting...", size);
 	}
 	return (rv);
 }
 
-/*
- * Safe version of fread(), exits on error.
- */
-int
-sfread(void *buf, size_t size, FILE *fp)
+char *
+checksum_str(zio_cksum_t *cksum, char *buff, size_t buff_size)
 {
-	int rv = fread(buf, size, 1, fp);
-	if (rv == 0 && ferror(fp)) {
-		(void) fprintf(stderr, "Error while reading file: %s\n",
-		    strerror(errno));
-		exit(1);
+	snprintf(buff, buff_size, "%.16llx / %.16llx / %.16llx / %.16llx",
+	    (long long unsigned int) cksum->zc_word[0],
+	    (long long unsigned int) cksum->zc_word[1],
+	    (long long unsigned int) cksum->zc_word[2],
+	    (long long unsigned int) cksum->zc_word[3]);
+	return (buff);
+}
+
+boolean_t
+validate_checksum(zio_cksum_t *expected, zio_cksum_t *actual,
+    boolean_t swap, const char *where, off_t stream_offset)
+{
+	static char buff[128];
+	zio_cksum_t swapped_actual;
+
+	if (swap) {
+		swapped_actual = *actual;
+		actual = &swapped_actual;
+		ZIO_CHECKSUM_BSWAP(actual);
 	}
-	return (rv);
+	/* cppcheck-suppress uninitvar */
+	if (ZIO_CHECKSUM_EQUAL(*expected, *actual)) {
+		return (B_TRUE);
+	}
+	fflush(stdout);
+	fprintf(stderr, "Incorrect checksum %s (stream offset %lld)\n", where,
+	    (longlong_t)stream_offset);
+	fprintf(stderr, "Expected = %s\n", checksum_str(expected, buff,
+	    sizeof (buff)));
+	fprintf(stderr, "  Actual = %s\n", checksum_str(actual, buff,
+	    sizeof (buff)));
+	return (B_FALSE);
+}
+
+boolean_t
+write_is_encrypted(struct drr_write *drrw)
+{
+	for (int i = 0; i < ZIO_DATA_SALT_LEN; i++) {
+		if (drrw->drr_salt[i] != 0) {
+			return (B_TRUE);
+		}
+	}
+	return (B_FALSE);
+}
+
+/*
+ * The specified compress_type must reflect the buffer's actual compression.
+ * Returns an allocated buffer if decompression was successful, NULL
+ * otherwise.
+ */
+uint8_t *
+decompress_buffer(uint8_t *inbuff, size_t inbuff_size, size_t logical_size,
+    enum zio_compress compress_type)
+{
+	uint8_t *outbuff = safe_malloc(logical_size);
+	abd_t sabd, dabd;
+	int ret;
+
+	VERIFY3B(ctype_is_uncompressed(compress_type), ==, B_FALSE);
+
+	abd_get_from_buf_struct(&sabd, inbuff, inbuff_size);
+	abd_get_from_buf_struct(&dabd, outbuff, logical_size);
+	ret = zio_decompress_data(compress_type, &sabd, &dabd,
+	    inbuff_size, abd_get_size(&dabd), NULL);
+
+	abd_free(&dabd);
+	abd_free(&sabd);
+
+	if (ret != 0) {
+		free(outbuff);
+		return (NULL);
+	}
+
+	return (outbuff);
+}
+
+/*
+ * Returns an allocated buffer if compression was successful, NULL
+ * otherwise.
+ */
+uint8_t *
+compress_buffer(uint8_t *inbuff, size_t inbuff_size,
+    compression_spec_t compress_type, size_t *compressed_size)
+{
+	uint8_t *outbuff = safe_malloc(inbuff_size);
+	abd_t	sabd, dabd;
+	size_t	csize, rounded;
+
+	VERIFY3B(ctype_is_uncompressed(compress_type.cs_type), ==, B_FALSE);
+
+	abd_t *pabd = abd_get_from_buf_struct(&dabd, outbuff, inbuff_size);
+	abd_get_from_buf_struct(&sabd, inbuff, inbuff_size);
+	csize = zio_compress_data(compress_type.cs_type, &sabd,
+	    &pabd, inbuff_size, inbuff_size, compress_type.cs_level);
+
+	rounded = P2ROUNDUP(csize, SPA_MINBLOCKSIZE);
+	if (rounded < inbuff_size) {
+		abd_zero_off(pabd, csize, rounded - csize);
+		*compressed_size = rounded;
+	} else {
+		free(outbuff);
+		outbuff = NULL;
+	}
+
+	abd_free(&sabd);
+	abd_free(&dabd);
+
+	return (outbuff);
 }

cmd/zstream/zstream_util.h

@@ -26,8 +26,15 @@ extern "C" {
 #endif
 
 #include <stddef.h>
+#include <stdlib.h>
 #include <sys/zfs_ioctl.h>
 #include <sys/zio_checksum.h>
+#include <sys/zio_compress.h>
+
+typedef struct {
+	enum zio_compress	cs_type;
+	int			cs_level;
+} compression_spec_t;
 
 /*
  * The safe_ versions of the functions below terminate the process if the
@@ -39,19 +46,46 @@ safe_malloc(size_t size);
 extern void *
 safe_calloc(size_t n);
 
-extern int
-sfread(void *buf, size_t size, FILE *fp);
+extern char *
+checksum_str(zio_cksum_t *cksum, char *buff, size_t buff_size);
 
 /*
- * 1) Update checksum with the record header up to drr_checksum.
- * 2) Update checksum field in the record header.
- * 3) Update checksum with the checksum field in the record header.
- * 4) Update checksum with the contents of the payload.
- * 5) Write header and payload to fd.
+ * Prints an error message if checksums don't match. Returns B_TRUE for
+ * a match, B_FALSE otherwise.
  */
-extern int
-dump_record(dmu_replay_record_t *drr, void *payload, size_t payload_len,
-	zio_cksum_t *zc, int outfd);
+boolean_t
+validate_checksum(zio_cksum_t *expect, zio_cksum_t *actual, boolean_t swap,
+    const char *where, off_t stream_offset);
+
+static inline void
+validate_or_exit(zio_cksum_t *expect, zio_cksum_t *actual, boolean_t swap,
+    const char *where, off_t stream_offset)
+{
+	if (!validate_checksum(expect, actual, swap, where, stream_offset)) {
+		exit(1);
+	}
+}
+
+/*
+ * Determine whether a compression type indicates no compression
+ */
+static inline boolean_t
+ctype_is_uncompressed(enum zio_compress ct)
+{
+	VERIFY3U((int)ct, <, (int)ZIO_COMPRESS_FUNCTIONS);
+	return (zio_compress_table[(int)(ct)].ci_compress == NULL);
+}
+
+boolean_t
+write_is_encrypted(struct drr_write *drrw);
+
+uint8_t *
+decompress_buffer(uint8_t *inbuff, size_t inbuff_size, size_t logical_size,
+	enum zio_compress compress_type);
+
+uint8_t *
+compress_buffer(uint8_t *inbuff, size_t inbuff_size,
+    compression_spec_t compress_type, size_t *compressed_size);
 
 #ifdef __cplusplus
 }

cmd/zstream/zstream_validate.c

@@ -0,0 +1,129 @@
+// SPDX-License-Identifier: CDDL-1.0
+/*
+ * CDDL HEADER START
+ *
+ * This file and its contents are supplied under the terms of the Common
+ * Development and Distribution License ("CDDL"), version 1.0. You may only use
+ * this file in accordance with the terms of version 1.0 of the CDDL.
+ *
+ * A full copy of the text of the CDDL should have accompanied this source. A
+ * copy of the CDDL is also available via the Internet at
+ * http://www.illumos.org/license/CDDL.
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Copyright (c) 2026 by Garth Snyder. All rights reserved.
+ */
+
+#include <assert.h>
+#include <err.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/stdtypes.h>
+#include <sys/zfs_ioctl.h>
+#include <sys/zio_compress.h>
+
+#include "zstream_modules.h"
+
+/*
+ * Validate consistency and well-formedness of the actual DRR records. I
+ * have swept all the existing validation code into this module, but it's
+ * still pretty sparse.
+ */
+
+#define	MAX_VALIDATIONS 4
+
+typedef struct {
+	int	nesting;
+} validate_context_t;
+
+static validate_context_t 	contexts[MAX_VALIDATIONS];
+static int			next_context = 0;
+
+static disposition_t
+chain_validate_records(drr_packet_t *item, validate_context_t *context)
+{
+	if (item == NULL)
+		return (D_OK);
+
+	struct dmu_replay_record *drr	= &item->dp_drr;
+	struct drr_write *drrw		= &drr->drr_u.drr_write;
+	struct drr_object *drro		= &drr->drr_u.drr_object;
+
+	if (OPTION_ENABLED(CA_DO_NOT_VALIDATE))
+		return (D_OK);
+
+	if (item->dp_stream_offset == 0 && drr->drr_type != DRR_BEGIN) {
+		warnx("warning - first record is not DRR_BEGIN");
+	}
+
+	if (drr->drr_type == DRR_BEGIN) {
+		VERIFY0(context->nesting);
+		context->nesting++;
+	} else if (drr->drr_type == DRR_END) {
+		VERIFY3S(context->nesting, >=, 0);
+		if (context->nesting > 0)
+			context->nesting--;
+	} else if (drr->drr_type >= DRR_NUMTYPES) {
+		errx(1, "unknown record type: %d", drr->drr_type);
+	} else {
+		VERIFY3S(context->nesting, ==, 1);
+	}
+
+	switch (drr->drr_type) {
+
+	case DRR_BEGIN:
+		VERIFY3U(item->dp_payload_size, <=, 1UL << 28);
+		break;
+
+	case DRR_OBJECT:
+		{
+			boolean_t is_raw = !!(chain_attrs->ca_feature_flags &
+			    DMU_BACKUP_FEATURE_RAW);
+			boolean_t bonus_gt_raw = drro->drr_bonuslen >
+			    drro->drr_raw_bonuslen;
+			if (is_raw && bonus_gt_raw) {
+				fprintf(stderr,
+				    "Warning: object %llu has bonuslen = "
+				    "%u > raw_bonuslen = %u\n\n",
+				    (u_longlong_t)drro->drr_object,
+				    drro->drr_bonuslen,
+				    drro->drr_raw_bonuslen);
+			}
+			break;
+		}
+
+	case DRR_WRITE:
+		if (drrw->drr_compressiontype >= ZIO_COMPRESS_FUNCTIONS) {
+			errx(1, "invalid compression type: %d",
+			    drrw->drr_compressiontype);
+		}
+		break;
+
+	default:
+		break;
+	}
+
+	return (D_OK);
+}
+
+chain_step_t
+serial_validate_records(void)
+{
+	int context_ix = next_context++ % MAX_VALIDATIONS;
+	validate_context_t *context = &contexts[context_ix];
+	context->nesting = 0;
+
+	chain_step_t step = {
+		.cs_type = CS_SERIAL,
+		.cs_in_size = sizeof (drr_packet_t),
+		.cs_out_size = sizeof (drr_packet_t),
+		.cs_context = context,
+		.cs_serial = {
+		    .process = (zc_serial_process_f *)chain_validate_records,
+		}
+	};
+	return (step);
+}

cmd/zstream/zstream_validate.h

@@ -0,0 +1,36 @@
+// SPDX-License-Identifier: CDDL-1.0
+/*
+ * CDDL HEADER START
+ *
+ * This file and its contents are supplied under the terms of the Common
+ * Development and Distribution License ("CDDL"), version 1.0. You may only use
+ * this file in accordance with the terms of version 1.0 of the CDDL.
+ *
+ * A full copy of the text of the CDDL should have accompanied this source. A
+ * copy of the CDDL is also available via the Internet at
+ * http://www.illumos.org/license/CDDL.
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Copyright (c) 2026 by Garth Snyder. All rights reserved.
+ */
+
+#ifndef _ZSTREAM_VALIDATE_H
+#define	_ZSTREAM_VALIDATE_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "zstream_io.h"
+
+chain_step_t
+serial_validate_records(void);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif  /* _ZSTREAM_VALIDATE_H */

tests/runfiles/common.run

@@ -997,7 +997,7 @@ tests = ['recv_dedup', 'recv_dedup_encrypted_zvol', 'rsend_001_pos',
     'rsend_030_pos', 'rsend_031_pos', 'rsend-exclude_001_pos',
     'rsend-exclude_002_pos', 'send-c_verify_ratio',
     'send-c_verify_contents', 'send-c_props', 'send-c_incremental',
-    'send-c_volume', 'send-c_zstream_recompress', 'send-c_zstreamdump',
+    'send-c_volume',
     'send-c_lz4_disabled', 'send-c_recv_lz4_disabled',
     'send-c_mixed_compression', 'send-c_stream_size_estimate',
     'send-c_embedded_blocks', 'send-c_resume', 'send-cpL_varied_recsize',
@@ -1006,7 +1006,7 @@ tests = ['recv_dedup', 'recv_dedup_encrypted_zvol', 'rsend_001_pos',
     'send_encrypted_props', 'send_encrypted_truncated_files',
     'send_freeobjects', 'send_realloc_files', 'send_realloc_encrypted_files',
     'send_spill_block', 'send_holds', 'send_hole_birth', 'send_mixed_raw',
-    'send-wR_encrypted_zvol', 'send-zstream_drop_record',
+    'send-wR_encrypted_zvol',
     'send_partial_dataset', 'send_invalid',
     'send_large_blocks_incremental', 'send_large_blocks_initial',
     'send_large_microzap_incremental', 'send_large_microzap_transitive',
@@ -1135,6 +1135,20 @@ tests = ['zoned_uid_001_pos', 'zoned_uid_002_pos', 'zoned_uid_003_pos',
     'zoned_uid_029_neg', 'zoned_uid_031_pos']
 tags = ['functional', 'zoned_uid']
 
+[tests/functional/zstream]
+tests = ['zstream_checksum_001_pos',
+    'zstream_decompress_001_pos', 'zstream_decompress_002_pos',
+    'zstream_decompress_003_neg', 'zstream_decompress_004_pos',
+    'zstream_decompress_005_pos', 'zstream_decompress_006_neg',
+    'zstream_drop_record_001_pos',
+    'zstream_dump_001_pos', 'zstream_dump_002_pos',
+    'zstream_dump_003_pos', 'zstream_dump_004_neg',
+    'zstream_recompress_001_pos', 'zstream_recompress_002_pos',
+    'zstream_recompress_003_pos', 'zstream_recompress_004_pos',
+    'zstream_recompress_005_pos',
+    'zstream_redup_001_pos']
+tags = ['functional', 'zstream']
+
 [tests/functional/zvol/zvol_ENOSPC]
 tests = ['zvol_ENOSPC_001_pos']
 tags = ['functional', 'zvol', 'zvol_ENOSPC']

tests/runfiles/sanity.run

@@ -546,7 +546,7 @@ tags = ['functional', 'reservation']
 tests = ['recv_dedup', 'recv_dedup_encrypted_zvol', 'rsend_001_pos',
     'rsend_002_pos', 'rsend_003_pos', 'rsend_009_pos', 'rsend_010_pos',
     'rsend_011_pos', 'rsend_016_neg', 'rsend-exclude_001_pos',
-    'rsend-exclude_002_pos', 'send-c_volume', 'send-c_zstreamdump',
+    'rsend-exclude_002_pos', 'send-c_volume',
     'send-c_recv_dedup', 'send-L_toggle', 'send_encrypted_hierarchy',
     'send_encrypted_props', 'send_encrypted_freeobjects',
     'send_encrypted_truncated_files', 'send_freeobjects', 'send_holds',
@@ -614,6 +614,11 @@ tests = ['xattr_001_pos', 'xattr_002_neg', 'xattr_003_neg', 'xattr_004_pos',
     'xattr_011_pos', 'xattr_013_pos', 'xattr_014_pos', 'xattr_compat']
 tags = ['functional', 'xattr']
 
+[tests/functional/zstream]
+tests = ['zstream_dump_001_pos', 'zstream_dump_002_pos',
+    'zstream_redup_001_pos']
+tags = ['functional', 'zstream']
+
 [tests/functional/zvol/zvol_ENOSPC]
 tests = ['zvol_ENOSPC_001_pos']
 tags = ['functional', 'zvol', 'zvol_ENOSPC']

tests/zfs-tests/tests/Makefile.am

@@ -403,6 +403,42 @@ nobase_dist_datadir_zfs_tests_tests_DATA += \
 	functional/zvol/zvol_ENOSPC/zvol_ENOSPC.cfg \
 	functional/zvol/zvol_misc/zvol_misc_common.kshlib \
 	functional/zvol/zvol_swap/zvol_swap.cfg \
+	functional/zstream/zstream.cfg \
+	functional/zstream/zstream.kshlib \
+	functional/zstream/big-endian-all-drr-types-base-NATIVE.zsend.bz2 \
+	functional/zstream/big-endian-all-drr-types-base-XDR.zsend.bz2 \
+	functional/zstream/big-endian-all-drr-types-incr-NATIVE.zsend.bz2 \
+	functional/zstream/big-endian-all-drr-types-incr-XDR.zsend.bz2 \
+	functional/zstream/little-endian-all-drr-types-base-NATIVE.zsend.bz2 \
+	functional/zstream/little-endian-all-drr-types-base-XDR.zsend.bz2 \
+	functional/zstream/little-endian-all-drr-types-incr-NATIVE.zsend.bz2 \
+	functional/zstream/little-endian-all-drr-types-incr-XDR.zsend.bz2 \
+	functional/zstream/decompress.zsend.bz2 \
+	functional/zstream/decompress-crypt.zsend.bz2 \
+	functional/zstream/big-endian-long-payloads.zsend.bz2 \
+	functional/zstream/little-endian-long-payloads.zsend.bz2 \
+	functional/zstream/beadtbn-new.dump.bz2 \
+	functional/zstream/beadtbn-old.dump.bz2 \
+	functional/zstream/beadtbx-new.dump.bz2 \
+	functional/zstream/beadtbx-old.dump.bz2 \
+	functional/zstream/beadtin-new.dump.bz2 \
+	functional/zstream/beadtin-old.dump.bz2 \
+	functional/zstream/beadtix-new.dump.bz2 \
+	functional/zstream/beadtix-old.dump.bz2 \
+	functional/zstream/belp-new.dump.bz2 \
+	functional/zstream/belp-old.dump.bz2 \
+	functional/zstream/d-new.dump.bz2 \
+	functional/zstream/dc-new.dump.bz2 \
+	functional/zstream/leadtbn-new.dump.bz2 \
+	functional/zstream/leadtbn-old.dump.bz2 \
+	functional/zstream/leadtbx-new.dump.bz2 \
+	functional/zstream/leadtbx-old.dump.bz2 \
+	functional/zstream/leadtin-new.dump.bz2 \
+	functional/zstream/leadtin-old.dump.bz2 \
+	functional/zstream/leadtix-new.dump.bz2 \
+	functional/zstream/leadtix-old.dump.bz2 \
+	functional/zstream/lelp-new.dump.bz2 \
+	functional/zstream/lelp-old.dump.bz2 \
 	functional/idmap_mount/idmap_mount.cfg \
 	functional/idmap_mount/idmap_mount_common.kshlib
 
@@ -2096,8 +2132,6 @@ nobase_dist_datadir_zfs_tests_tests_SCRIPTS += \
 	functional/rsend/send-c_verify_contents.ksh \
 	functional/rsend/send-c_verify_ratio.ksh \
 	functional/rsend/send-c_volume.ksh \
-	functional/rsend/send-c_zstream_recompress.ksh \
-	functional/rsend/send-c_zstreamdump.ksh \
 	functional/rsend/send-cpL_varied_recsize.ksh \
 	functional/rsend/send_doall.ksh \
 	functional/rsend/send_encrypted_incremental.ksh \
@@ -2126,7 +2160,6 @@ nobase_dist_datadir_zfs_tests_tests_SCRIPTS += \
 	functional/rsend/send_realloc_files.ksh \
 	functional/rsend/send_spill_block.ksh \
 	functional/rsend/send-wR_encrypted_zvol.ksh \
-	functional/rsend/send-zstream_drop_record.ksh \
 	functional/rsend/setup.ksh \
 	functional/scrub_mirror/cleanup.ksh \
 	functional/scrub_mirror/scrub_mirror_001_pos.ksh \
@@ -2354,6 +2387,26 @@ nobase_dist_datadir_zfs_tests_tests_SCRIPTS += \
 	functional/zpool_influxdb/cleanup.ksh \
 	functional/zpool_influxdb/setup.ksh \
 	functional/zpool_influxdb/zpool_influxdb.ksh \
+	functional/zstream/setup.ksh \
+	functional/zstream/cleanup.ksh \
+	functional/zstream/zstream_checksum_001_pos.ksh \
+	functional/zstream/zstream_decompress_001_pos.ksh \
+	functional/zstream/zstream_decompress_002_pos.ksh \
+	functional/zstream/zstream_decompress_003_neg.ksh \
+	functional/zstream/zstream_decompress_004_pos.ksh \
+	functional/zstream/zstream_decompress_005_pos.ksh \
+	functional/zstream/zstream_decompress_006_neg.ksh \
+	functional/zstream/zstream_drop_record_001_pos.ksh \
+	functional/zstream/zstream_dump_001_pos.ksh \
+	functional/zstream/zstream_dump_002_pos.ksh \
+	functional/zstream/zstream_dump_003_pos.ksh \
+	functional/zstream/zstream_dump_004_neg.ksh \
+	functional/zstream/zstream_recompress_001_pos.ksh \
+	functional/zstream/zstream_recompress_002_pos.ksh \
+	functional/zstream/zstream_recompress_003_pos.ksh \
+	functional/zstream/zstream_recompress_004_pos.ksh \
+	functional/zstream/zstream_recompress_005_pos.ksh \
+	functional/zstream/zstream_redup_001_pos.ksh \
 	functional/zvol/zvol_cli/cleanup.ksh \
 	functional/zvol/zvol_cli/setup.ksh \
 	functional/zvol/zvol_cli/zvol_cli_001_pos.ksh \

tests/zfs-tests/tests/functional/zstream/cleanup.ksh

@@ -0,0 +1,33 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+#
+# CDDL HEADER START
+#
+# The contents of this file are subject to the terms of the
+# Common Development and Distribution License (the "License").
+# You may not use this file except in compliance with the License.
+#
+# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
+# or https://opensource.org/licenses/CDDL-1.0.
+# See the License for the specific language governing permissions
+# and limitations under the License.
+#
+# When distributing Covered Code, include this CDDL HEADER in each
+# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
+# If applicable, add the following below this CDDL HEADER, with the
+# fields enclosed by brackets "[]" replaced with your own identifying
+# information: Portions Copyright [yyyy] [name of copyright owner]
+#
+# CDDL HEADER END
+#
+
+#
+# Copyright (c) 2026 by ConnectWise. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+verify_runnable "both"
+
+log_must rm -rf $BACKDIR
+default_cleanup $POOL

tests/zfs-tests/tests/functional/zstream/setup.ksh

@@ -0,0 +1,33 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+#
+# CDDL HEADER START
+#
+# The contents of this file are subject to the terms of the
+# Common Development and Distribution License (the "License").
+# You may not use this file except in compliance with the License.
+#
+# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
+# or https://opensource.org/licenses/CDDL-1.0.
+# See the License for the specific language governing permissions
+# and limitations under the License.
+#
+# When distributing Covered Code, include this CDDL HEADER in each
+# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
+# If applicable, add the following below this CDDL HEADER, with the
+# fields enclosed by brackets "[]" replaced with your own identifying
+# information: Portions Copyright [yyyy] [name of copyright owner]
+#
+# CDDL HEADER END
+#
+
+#
+# Copyright (c) 2026 by ConnectWise. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+verify_runnable "both"
+
+log_must mkdir -p $BACKDIR
+default_setup $DISK1

tests/zfs-tests/tests/functional/zstream/test-stream-creation-scripts/README

@@ -0,0 +1,19 @@
+The scripts in this directory were used to generate most of the static
+test streams used by the zstream test suite. They're not really
+intended to be run again and are here more as documentation than as
+code.
+
+Some of the python scripts use the 'lorem' pip module, so you need to
+run the make-venv.sh to set up before running them. The venv is created
+in /tmp and the scripts hard-code that path for python.
+
+The "all-drr-types" streams have to be generated in two pieces because
+of the way redaction works. A *base* stream and an *incr* stream
+collectively include every possible type of record, but neither
+substream is complete on its own.
+
+Many test streams are big-endian, so given hardware's convergence on
+little-endian order, you will probably need to create big-endian
+streams under emulation with, e.g., QEMU. As of mid-2026, the easiest
+target for big-endian emulation is an IBM s390x CPU, which is supported
+by both QEMU and Ubuntu.

tests/zfs-tests/tests/functional/zstream/test-stream-creation-scripts/add-xattrs.py

@@ -0,0 +1,112 @@
+#!/tmp/zstream-venv/bin/python3
+"""Add at least 1024 bytes of random extended attributes to files."""
+
+#
+# SPDX-License-Identifier: CDDL-1.0
+#
+# CDDL HEADER START
+#
+# This file and its contents are supplied under the terms of the Common
+# Development and Distribution License ("CDDL"), version 1.0. You may only use
+# this file in accordance with the terms of version 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this source. A
+# copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+# CDDL HEADER END
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+import argparse
+import os
+import random
+import sys
+from lorem_text import lorem
+
+ADJECTIVES = [
+    "boogie", "funky", "wobbly", "snazzy", "jazzy", "groovy", "zippy",
+    "bouncy", "fluffy", "crunchy", "sparkly", "fuzzy", "spiffy", "dandy",
+    "peppy", "snappy", "sassy", "zesty", "swanky", "nifty", "plucky",
+    "quirky", "wacky", "goofy", "dizzy", "breezy", "cheery", "perky",
+    "frisky", "chirpy", "feisty", "jolly", "lively", "merry", "spunky",
+    "zippy", "vivid", "brisk", "sunny", "witty", "kinky",
+]
+
+NOUNS = [
+    "woogie", "monkey", "noodle", "pickle", "muffin", "waffle", "pebble",
+    "wobble", "doodle", "tangle", "giggle", "wiggle", "jiggle", "sparkle",
+    "crinkle", "twinkle", "frizzle", "drizzle", "sizzle", "fizzle",
+    "puddle", "bubble", "muddle", "huddle", "cuddle", "juggle", "muggle",
+    "snuggle", "tuggle", "buggle", "nugget", "widget", "gadget", "gibbet",
+    "trinket", "bracket", "racket", "jacket", "ticket", "cricket", "thicket",
+    "biscuit", "circuit", "summit", "muppet", "trumpet", "basket", "casket",
+]
+
+TARGET_BYTES = 1024
+
+
+def random_attr_name(used: set) -> str:
+    for _ in range(1000):
+        name = f"user.{random.choice(ADJECTIVES)}-{random.choice(NOUNS)}"
+        if name not in used:
+            return name
+    base = f"user.{random.choice(ADJECTIVES)}-{random.choice(NOUNS)}"
+    i = 2
+    while f"{base}-{i}" in used:
+        i += 1
+    return f"{base}-{i}"
+
+
+def random_value(length: int) -> bytes:
+    # Pull words from lorem sentences and trim/pad to exact length
+    text = ""
+    while len(text) < length:
+        text += lorem.sentence() + " "
+    return text[:length].encode()
+
+
+def add_xattrs(path: str) -> int:
+    """Add xattrs to path until TARGET_BYTES added. Returns bytes added."""
+    used_names = set()
+    total = 0
+    while total < TARGET_BYTES:
+        remaining = TARGET_BYTES - total
+        if remaining < 40:
+            length = remaining
+        else:
+            length = random.randint(40, min(200, remaining))
+        name = random_attr_name(used_names)
+        used_names.add(name)
+        value = random_value(length)
+        os.setxattr(path, name, value)
+        total += len(value)
+    return total
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description=f"Add random xattrs to files until {TARGET_BYTES} bytes "
+                    "of xattr values are added."
+    )
+    parser.add_argument("files", nargs="+", help="Files to add xattrs to")
+    args = parser.parse_args()
+
+    errors = 0
+    for path in args.files:
+        try:
+            added = add_xattrs(path)
+            print(f"  {path}  ({added:,} bytes in xattrs)")
+        except OSError as e:
+            print(f"  {path}  error: {e}", file=sys.stderr)
+            errors += 1
+
+    if errors:
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()

tests/zfs-tests/tests/functional/zstream/test-stream-creation-scripts/gen-lorem-files.py

@@ -0,0 +1,109 @@
+#!/tmp/zstream-venv/bin/python3
+"""Generate randomly-named files with lorem ipsum paragraphs."""
+
+#
+# SPDX-License-Identifier: CDDL-1.0
+#
+# CDDL HEADER START
+#
+# This file and its contents are supplied under the terms of the Common
+# Development and Distribution License ("CDDL"), version 1.0. You may only use
+# this file in accordance with the terms of version 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this source. A
+# copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+# CDDL HEADER END
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+import argparse
+import random
+import sys
+from pathlib import Path
+from lorem_text import lorem
+
+ADJECTIVES = [
+    "boogie", "funky", "wobbly", "snazzy", "jazzy", "groovy", "zippy",
+    "bouncy", "fluffy", "crunchy", "sparkly", "fuzzy", "spiffy", "dandy",
+    "peppy", "snappy", "sassy", "zesty", "swanky", "nifty", "plucky",
+    "quirky", "wacky", "goofy", "dizzy", "breezy", "cheery", "perky",
+    "frisky", "chirpy", "feisty", "jolly", "lively", "merry", "spunky",
+    "frisky", "zippy", "vivid", "brisk", "sunny", "witty", "kinky",
+]
+
+NOUNS = [
+    "woogie", "monkey", "noodle", "pickle", "muffin", "waffle", "pebble",
+    "wobble", "doodle", "tangle", "giggle", "wiggle", "jiggle", "sparkle",
+    "crinkle", "twinkle", "frizzle", "drizzle", "sizzle", "fizzle",
+    "puddle", "bubble", "muddle", "huddle", "cuddle", "juggle", "muggle",
+    "snuggle", "tuggle", "buggle", "nugget", "widget", "gadget", "gibbet",
+    "trinket", "bracket", "racket", "jacket", "ticket", "cricket", "thicket",
+    "biscuit", "circuit", "summit", "muppet", "trumpet", "basket", "casket",
+]
+
+
+def random_name(used: set) -> str:
+    for _ in range(1000):
+        name = f"{random.choice(ADJECTIVES)}-{random.choice(NOUNS)}"
+        if name not in used:
+            return name
+    # Fallback: append a number
+    base = f"{random.choice(ADJECTIVES)}-{random.choice(NOUNS)}"
+    i = 2
+    while f"{base}-{i}" in used:
+        i += 1
+    return f"{base}-{i}"
+
+
+def fill_file(path: Path, target_size: int, repeat=False) -> None:
+    content_parts = []
+    total = 0
+    para = lorem.paragraph()
+    while total < target_size:
+        content_parts.append(para)
+        total += len(para) + 1  # +1 for newline
+        if not repeat:
+            para = lorem.paragraph()
+    path.write_text("\n\n".join(content_parts) + "\n")
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Generate files with random names and lorem ipsum content."
+    )
+    parser.add_argument("count", type=int, help="Number of files to create")
+    parser.add_argument("-d", "--directory", default=".",
+                        help="Target directory (default: .)")
+    parser.add_argument("-r", "--repeat", action="store_true",
+                        help="Fill files with reps of a single paragraph")
+    parser.add_argument("--min-size", type=int, default=16384,
+                        help="Minimum file size in bytes (default: 16384)")
+    parser.add_argument("--max-size", type=int, default=128000,
+                        help="Maximum file size in bytes (default: 128000)")
+    args = parser.parse_args()
+
+    if args.min_size >= args.max_size:
+        print(f"error: min-size ({args.min_size}) must be less than max-size "
+              f" ({args.max_size})", file=sys.stderr)
+        sys.exit(1)
+
+    directory = Path(args.directory)
+    directory.mkdir(parents=True, exist_ok=True)
+
+    used_names = set()
+    for i in range(args.count):
+        name = random_name(used_names)
+        used_names.add(name)
+        target_size = random.randint(args.min_size, args.max_size)
+        path = directory / name
+        fill_file(path, target_size, args.repeat)
+        print(f"  {path}  ({path.stat().st_size:,} bytes)")
+
+
+if __name__ == "__main__":
+    main()

tests/zfs-tests/tests/functional/zstream/test-stream-creation-scripts/make-all-records-streams.sh

@@ -0,0 +1,67 @@
+#!/bin/sh
+
+#
+# SPDX-License-Identifier: CDDL-1.0
+#
+# CDDL HEADER START
+#
+# This file and its contents are supplied under the terms of the Common
+# Development and Distribution License ("CDDL"), version 1.0. You may only use
+# this file in accordance with the terms of version 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this source. A
+# copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+# CDDL HEADER END
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+if [ $# -ne 1 ]; then
+	echo "Usage: $0 <device>" >&2
+	exit 1
+fi
+
+DEVICE="$1"
+SCRIPTDIR="$(cd "$(dirname "$0")" && pwd)"
+
+zpool create -o ashift=12 test "$DEVICE"
+zfs set compression=on xattr=sa test
+zfs create test/source
+
+"$SCRIPTDIR/gen-lorem-files.py" -r -d /test/source --min-size 2048 \
+    --max-size 32000 3
+"$SCRIPTDIR/add-xattrs.py" /test/source/*
+echo "very small" > /test/source/small
+echo "password" > /test/source/to-be-redacted
+chmod 400 /test/source/to-be-redacted
+
+zfs snapshot -r test/source@baseline
+zfs clone test/source@baseline test/redacted
+rm /test/redacted/to-be-redacted
+"$SCRIPTDIR/gen-lorem-files.py" -r -d /test/redacted --min-size 4096 \
+    --max-size 32000 3
+"$SCRIPTDIR/add-xattrs.py" /test/redacted/*
+cd /test/redacted || exit 1
+tar cf /tmp/dups.tar .
+mkdir copies
+cd copies || exit 1
+tar xvf /tmp/dups.tar
+
+echo "password" > /test/redacted/new-key
+zfs create -o encryption=on -o keylocation=file:///test/redacted/new-key \
+    -o keyformat=passphrase test/redacted/encrypted
+"$SCRIPTDIR/gen-lorem-files.py" -r -d /test/redacted/encrypted 3
+echo "very small" > /test/redacted/encrypted/small-encrypted
+# "$SCRIPTDIR/add-xattrs.py" /test/redacted/encrypted/*
+
+zfs snapshot -r test/redacted@clean
+
+zfs redact test/source@baseline redaction-bookmark test/redacted@clean
+zfs send -ce --redact redaction-bookmark test/source@baseline \
+    > /tmp/all-record-types-base.zsend
+zfs send -Rcew -i test/source@baseline test/redacted@clean \
+    > /tmp/all-record-types-incr.zsend

tests/zfs-tests/tests/functional/zstream/test-stream-creation-scripts/make-decompression-streams.sh

@@ -0,0 +1,49 @@
+#!/bin/sh
+
+#
+# SPDX-License-Identifier: CDDL-1.0
+#
+# CDDL HEADER START
+#
+# This file and its contents are supplied under the terms of the Common
+# Development and Distribution License ("CDDL"), version 1.0. You may only use
+# this file in accordance with the terms of version 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this source. A
+# copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+# CDDL HEADER END
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+if [ $# -ne 1 ]; then
+	echo "Usage: $0 <device>" >&2
+	exit 1
+fi
+
+set -e
+
+DEVICE="$1"
+SCRIPTDIR="$(cd "$(dirname "$0")" && pwd)"
+
+zpool create -o ashift=12 test "$DEVICE"
+echo "password" > /test/password
+
+zfs create -o compression=zstd-5 test/unencrypted
+"$SCRIPTDIR/gen-lorem-files.py" -r -d /test/unencrypted --min-size 12000 \
+    --max-size 40000 2
+"$SCRIPTDIR/gen-lorem-files.py" -r -d /test/unencrypted --min-size 140000 \
+    --max-size 160000 1
+
+zfs create -o compression=lz4 -o encryption=on \
+    -o keylocation=file:///test/password -o keyformat=passphrase test/encrypted
+"$SCRIPTDIR/gen-lorem-files.py" -r -d /test/encrypted --min-size 12000 \
+    --max-size 40000 3
+
+zfs snapshot -r test@decompression
+zfs send -cw test/unencrypted@decompression > /tmp/decompression.zsend
+zfs send -cw test/encrypted@decompression > /tmp/decompression-crypt.zsend

tests/zfs-tests/tests/functional/zstream/test-stream-creation-scripts/make-dump-files.py

@@ -0,0 +1,97 @@
+#!/usr/bin/env python3
+"""Run old and new zstream dump -v on streams, producing dump outputs."""
+
+#
+# SPDX-License-Identifier: CDDL-1.0
+#
+# CDDL HEADER START
+#
+# This file and its contents are supplied under the terms of the Common
+# Development and Distribution License ("CDDL"), version 1.0. You may only use
+# this file in accordance with the terms of version 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this source. A
+# copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+# CDDL HEADER END
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+import argparse
+import subprocess
+import sys
+from pathlib import Path
+
+
+def abbreviate(filename: str) -> str:
+    """Split filename at dashes, take first letter of each segment, lc."""
+    stem = Path(filename).stem
+    # Strip common compression suffixes to get the logical stem
+    for ext in (".zfs", ".gz", ".bz2", ".xz", ".zst", ".lz4"):
+        if stem.endswith(ext):
+            stem = stem[: -len(ext)]
+    return "".join(seg[0] for seg in stem.split("-") if seg).lower()
+
+
+def run_dump(zstream: Path, stream: Path, output: Path) -> bool:
+    """Run `zstream dump -v < stream > output`.  Returns True on success."""
+    try:
+        with open(stream, "rb") as inf, open(output, "w") as outf:
+            proc = subprocess.run(
+                [str(zstream), "dump", "-v"],
+                stdin=inf,
+                stdout=outf,
+                stderr=outf,
+            )
+        if proc.returncode != 0:
+            print(
+                f"  WARNING: {zstream} exited {proc.returncode} for "
+                f"{stream.name}", file=sys.stderr
+            )
+        return True
+    except Exception as e:
+        print(f"  ERROR: {e}", file=sys.stderr)
+        return False
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Run old and new zstream dump -v on stream files."
+    )
+    parser.add_argument("old_zstream", type=Path, help="Path to old zstream")
+    parser.add_argument("new_zstream", type=Path, help="Path to new zstream")
+    parser.add_argument(
+        "streams", nargs="+", type=Path, help="Streams to process"
+    )
+    args = parser.parse_args()
+
+    for zs in (args.old_zstream, args.new_zstream):
+        if not zs.is_file():
+            parser.error(f"zstream binary not found: {zs}")
+
+    for stream in args.streams:
+        if not stream.is_file():
+            print(f"Skipping missing file: {stream}", file=sys.stderr)
+            continue
+
+        abbrev = abbreviate(stream.name)
+        out_dir = stream.parent
+
+        old_out = out_dir / f"{abbrev}-old.dump"
+        new_out = out_dir / f"{abbrev}-new.dump"
+
+        print(f"{stream.name} -> {abbrev}")
+
+        print(f"  old: {old_out}")
+        run_dump(args.old_zstream, stream, old_out)
+
+        print(f"  new: {new_out}")
+        run_dump(args.new_zstream, stream, new_out)
+
+
+if __name__ == "__main__":
+    main()

tests/zfs-tests/tests/functional/zstream/test-stream-creation-scripts/make-long-payloads.sh

@@ -0,0 +1,48 @@
+#!/bin/sh
+
+#
+# SPDX-License-Identifier: CDDL-1.0
+#
+# CDDL HEADER START
+#
+# This file and its contents are supplied under the terms of the Common
+# Development and Distribution License ("CDDL"), version 1.0. You may only use
+# this file in accordance with the terms of version 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this source. A
+# copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+# CDDL HEADER END
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+if [ $# -ne 1 ]; then
+	echo "Usage: $0 <device>" >&2
+	exit 1
+fi
+
+set -e
+
+DEVICE="$1"
+SCRIPTDIR="$(cd "$(dirname "$0")" && pwd)"
+
+zpool create -o ashift=12 test "$DEVICE"
+
+zfs set compression=off recordsize=16MiB test
+
+# We are testing 8MB blocks, so write one short file, 8.5MB
+# file, and one 24.5MB file.
+
+"$SCRIPTDIR/gen-lorem-files.py" -d /test -r --min-size 20000 \
+    --max-size 24000 1
+"$SCRIPTDIR/gen-lorem-files.py" -d /test -r --min-size 8500000 \
+    --max-size 8510000 1
+"$SCRIPTDIR/gen-lorem-files.py" -d /test -r --min-size 24500000 \
+    --max-size 24510000 1
+
+zfs snapshot test@long-payloads
+zfs send -L test@long-payloads > /tmp/long-payloads.zsend

tests/zfs-tests/tests/functional/zstream/test-stream-creation-scripts/make-venv.sh

@@ -0,0 +1,27 @@
+#!/bin/sh
+
+#
+# SPDX-License-Identifier: CDDL-1.0
+#
+# CDDL HEADER START
+#
+# This file and its contents are supplied under the terms of the Common
+# Development and Distribution License ("CDDL"), version 1.0. You may only use
+# this file in accordance with the terms of version 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this source. A
+# copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+# CDDL HEADER END
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+set -e
+
+python3 -m venv /tmp/zstream-venv
+. /tmp/zstream-venv/bin/activate
+pip install lorem_text

tests/zfs-tests/tests/functional/zstream/zstream.cfg

@@ -0,0 +1,33 @@
+# SPDX-License-Identifier: CDDL-1.0
+#
+# CDDL HEADER START
+#
+# The contents of this file are subject to the terms of the
+# Common Development and Distribution License (the "License").
+# You may not use this file except in compliance with the License.
+#
+# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
+# or https://opensource.org/licenses/CDDL-1.0.
+# See the License for the specific language governing permissions
+# and limitations under the License.
+#
+# When distributing Covered Code, include this CDDL HEADER in each
+# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
+# If applicable, add the following below this CDDL HEADER, with the
+# fields enclosed by brackets "[]" replaced with your own identifying
+# information: Portions Copyright [yyyy] [name of copyright owner]
+#
+# CDDL HEADER END
+#
+
+#
+# Copyright (c) 2026 by ConnectWise. All rights reserved.
+#
+
+export BACKDIR=${TEST_BASE_DIR%%/}/backdir-zstream
+export PATH=${PATH}:/usr/local/sbin:/usr/local/bin:/usr/bin
+
+read -r DISK1 _ <<<"$DISKS"
+export DISK1
+
+export POOL=$TESTPOOL

tests/zfs-tests/tests/functional/zstream/zstream.kshlib

@@ -0,0 +1,90 @@
+# SPDX-License-Identifier: CDDL-1.0
+#
+# CDDL HEADER START
+#
+# The contents of this file are subject to the terms of the
+# Common Development and Distribution License (the "License").
+# You may not use this file except in compliance with the License.
+#
+# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
+# or https://opensource.org/licenses/CDDL-1.0.
+# See the License for the specific language governing permissions
+# and limitations under the License.
+#
+# When distributing Covered Code, include this CDDL HEADER in each
+# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
+# If applicable, add the following below this CDDL HEADER, with the
+# fields enclosed by brackets "[]" replaced with your own identifying
+# information: Portions Copyright [yyyy] [name of copyright owner]
+#
+# CDDL HEADER END
+#
+
+#
+# Copyright (c) 2026 by ConnectWise. All rights reserved.
+#
+
+. $STF_SUITE/include/libtest.shlib
+. $STF_SUITE/include/math.shlib
+. $STF_SUITE/tests/functional/zstream/zstream.cfg
+
+#
+# Several zstream tests need functions defined in rsend.kshlib
+# (stream_has_features, get_resume_token, cleanup_pool).  Rather than
+# duplicate them, source the rsend library.  Note that rsend.kshlib
+# also sources rsend.cfg, which will define its own $BACKDIR, $POOL,
+# $POOL2, etc.  Our zstream.cfg is sourced first, and we reassert our
+# values afterward so that our tests use the zstream-specific paths
+# and pool names.
+#
+. $STF_SUITE/tests/functional/rsend/rsend.kshlib
+
+# Reassert our config over rsend's
+. $STF_SUITE/tests/functional/zstream/zstream.cfg
+
+ZSTREAM_DATADIR=$STF_SUITE/tests/functional/zstream
+
+#
+# Return "little" or "big" depending on the system's byte order.
+#
+function get_system_endian
+{
+	python3 -c "import sys; print(sys.byteorder)"
+}
+
+#
+# Map a stream filename stem to its dump-file abbreviation.
+# e.g. "big-endian-all-drr-types-base-NATIVE" -> "beadtbn"
+#
+function get_stream_abbrev
+{
+	typeset stem=$1
+	typeset -l abbrev=""
+	typeset IFS="-"
+
+	for part in $stem; do
+		abbrev="${abbrev}${part%"${part#?}"}"
+	done
+
+	echo "$abbrev"
+}
+
+#
+# Receive a stream into $POOL/recv, compute sorted xxhsum -H2 of all
+# regular files under the mountpoint, and write the result to $1.
+# Destroys $POOL/recv afterward if $3 is set to "cleanup".
+#
+function recv_and_hash
+{
+	typeset hashfile=$1
+	typeset stream=$2
+	typeset cleanup=${3:-""}
+
+	log_must zfs receive -F $POOL/recv < "$stream"
+	typeset mnt=$(get_prop mountpoint $POOL/recv)
+	( cd "$mnt" && find . -type f -print0 | sort -z | \
+	    xargs -0 xxh128sum ) > "$hashfile"
+	if [[ -n $cleanup ]]; then
+		log_must zfs destroy -r $POOL/recv
+	fi
+}

tests/zfs-tests/tests/functional/zstream/zstream_checksum_001_pos.ksh

@@ -0,0 +1,64 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that very long payload records are checksummed correctly by
+# running them through zstream redup (which recalculates checksums but is
+# otherwise a no-op). This exercises the 8MiB Fletcher4 chunk boundary
+# handling. There are test streams of both endiannesses because opposite-
+# endian checksumming is a slightly separate path.
+#
+# Strategy:
+# 1. Decompress the long-payloads test streams
+# 2. Pipe through zstream redup
+# 3. Verify the output is byte-identical to the input
+#
+
+verify_runnable "both"
+
+log_assert "Verify long payload records are checksummed correctly."
+
+typeset -a streams=(
+	little-endian-long-payloads
+	big-endian-long-payloads
+)
+
+typeset failed=""
+
+for stem in "${streams[@]}"; do
+
+	typeset src="$ZSTREAM_DATADIR/${stem}.zsend.bz2"
+	typeset orig="$BACKDIR/${stem}.zsend.orig"
+	typeset redup_out="$BACKDIR/${stem}.zsend.redup"
+
+	bzcat "$src" > "$orig"
+	log_must eval "zstream redup '$orig' > '$redup_out'"
+
+	if ! cmp -s "$orig" "$redup_out" > /dev/null 2>&1; then
+		log_note "MISMATCH: $stem"
+		failed="$failed $stem"
+	fi
+
+done
+
+[[ -z $failed ]] || log_fail "Round-trip mismatch for: $failed"
+
+log_pass "Long-payload records are checksummed correctly."

tests/zfs-tests/tests/functional/zstream/zstream_decompress_001_pos.ksh

@@ -0,0 +1,98 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that zstream decompress actually decompresses selected WRITE
+# records. The input stream contains zstd-compressed writes.
+#
+# Strategy:
+# 1. Decompress selected records (2,0 3,0 128,131072) from the stream
+# 2. Run zstream dump -v on both original and decompressed streams
+# 3. Verify the selected records now show compression type = 0,
+#    compressed_size = 0, and payload_size = logical_size
+#
+
+verify_runnable "both"
+
+log_assert "Verify zstream decompress decompresses selected WRITE records."
+
+typeset src="$ZSTREAM_DATADIR/decompress.zsend.bz2"
+typeset orig="$BACKDIR/decompress.orig.zsend"
+typeset decompressed="$BACKDIR/decompress.out.zsend"
+typeset orig_dump="$BACKDIR/decompress.orig.dump"
+typeset decomp_dump="$BACKDIR/decompress.out.dump"
+
+# Selected records: object,offset
+typeset -a records=(2,0 3,0 128,131072)
+
+# Decompress the bz2 and run zstream decompress on selected records
+bzcat "$src" > "$orig"
+log_must eval "zstream decompress ${records[*]} < '$orig' > '$decompressed'"
+
+# Dump both streams
+log_must eval "zstream dump -v < '$orig' > '$orig_dump' 2>&1"
+log_must eval "zstream dump -v < '$decompressed' > '$decomp_dump' 2>&1"
+
+# For each selected record, verify it was decompressed in the output
+typeset failed=""
+for rec in "${records[@]}"; do
+	typeset obj=${rec%,*}
+	typeset off=${rec#*,}
+
+	# Find the WRITE line for this object/offset in the original dump
+	# and extract the logical_size
+	typeset orig_line=$(awk \
+	    "/^WRITE object = $obj .* offset = $off /" \
+	    "$orig_dump")
+	typeset lsize=$(echo "$orig_line" | \
+	    sed 's/.*logical_size = \([0-9]*\).*/\1/')
+
+	# Find the same record in the decompressed dump
+	typeset decomp_line=$(awk \
+	    "/^WRITE object = $obj .* offset = $off /" \
+	    "$decomp_dump")
+
+	# Verify compression type = 0
+	if ! echo "$decomp_line" | grep -q 'compression type = 0'; then
+		log_note "Record $rec: compression type not 0"
+		log_note "  got: $decomp_line"
+		failed="$failed ${rec}(comp)"
+	fi
+
+	# Verify compressed_size = 0 (indicates uncompressed)
+	if ! echo "$decomp_line" | grep -q 'compressed_size = 0'; then
+		log_note "Record $rec: compressed_size not 0"
+		failed="$failed ${rec}(csize)"
+	fi
+
+	# Verify payload_size = logical_size
+	typeset psize=$(echo "$decomp_line" | \
+	    sed 's/.*payload_size = \([0-9]*\).*/\1/')
+	if [[ "$psize" != "$lsize" ]]; then
+		log_note "Record $rec: payload_size ($psize) != logical_size ($lsize)"
+		failed="$failed ${rec}(psize)"
+	fi
+done
+
+[[ -z $failed ]] || \
+    log_fail "Decompression verification failed for:$failed"
+
+log_pass "zstream decompress decompresses selected WRITE records."

tests/zfs-tests/tests/functional/zstream/zstream_decompress_002_pos.ksh

@@ -0,0 +1,57 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that a decompressed stream produces identical filesystem contents
+# when received.
+#
+# Strategy:
+# 1. Receive the original decompress.zsend into a test pool and hash files
+# 2. Receive the decompressed stream (with records 2,0 3,0
+#      and 128,131072 decompressed)
+# 3. Verify file hashes are identical
+#
+
+verify_runnable "both"
+
+log_assert "Verify decompressed stream receives with identical file contents."
+log_onexit cleanup_pool $POOL
+
+typeset src="$ZSTREAM_DATADIR/decompress.zsend.bz2"
+typeset orig="$BACKDIR/decompress.orig.zsend"
+typeset decompressed="$BACKDIR/decompress.out.zsend"
+
+typeset -a records=(2,0 3,0 128,131072)
+
+# Prepare streams
+bzcat "$src" > "$orig"
+log_must eval "zstream decompress ${records[*]} < '$orig' > '$decompressed'"
+
+# Receive original and hash
+recv_and_hash "$BACKDIR/hash-orig.txt" "$orig" cleanup
+
+# Receive decompressed and hash
+recv_and_hash "$BACKDIR/hash-decomp.txt" "$decompressed" cleanup
+
+# Compare
+log_must diff "$BACKDIR/hash-orig.txt" "$BACKDIR/hash-decomp.txt"
+
+log_pass "Decompressed stream receives with identical file contents."

tests/zfs-tests/tests/functional/zstream/zstream_decompress_003_neg.ksh

@@ -0,0 +1,59 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that specifying the wrong compression type (lz4 for a zstd
+# stream) causes decompression to fail gracefully, leaving the output
+# stream identical to the input.
+#
+# Strategy:
+# 1. Run zstream decompress with lz4 type on zstd-compressed records
+# 2. Verify the output stream is byte-identical to the input
+# 3. Check stderr for failure messages
+#
+
+verify_runnable "both"
+
+log_assert "Verify wrong compression type leaves stream unchanged."
+
+typeset src="$ZSTREAM_DATADIR/decompress.zsend.bz2"
+typeset orig="$BACKDIR/decompress.orig"
+typeset output="$BACKDIR/decompress-lz4.out"
+typeset errfile="$BACKDIR/decompress-lz4.err"
+
+typeset -a records=(2,0,lz4 3,0,lz4 128,131072,lz4)
+
+bzcat "$src" > "$orig"
+
+# Attempt to decompress zstd records as lz4 — should fail for each
+zstream decompress ${records[*]} < "$orig" > "$output" 2>"$errfile"
+
+# Output stream must be identical to input (nothing decompressed)
+log_must cmp -s "$orig" "$output"
+
+# Stderr should contain messages about the failed decompressions
+typeset errcount=$(wc -l < "$errfile")
+if [[ $errcount -ne 3 ]]; then
+	log_fail "Did not receive 3 error messages on stderr, got $errcount"
+fi
+log_note "Got $errcount lines of error output (expected)"
+
+log_pass "Wrong compression type leaves stream unchanged."

tests/zfs-tests/tests/functional/zstream/zstream_decompress_004_pos.ksh

@@ -0,0 +1,55 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that specifying the correct compression type (zstd) produces
+# the same output as omitting the type.
+#
+# Strategy:
+# 1. Decompress records without specifying type
+# 2. Decompress records specifying zstd as type
+# 3. Verify both outputs are identical
+#
+
+verify_runnable "both"
+
+log_assert "Verify explicit correct compression type matches default."
+
+typeset src="$ZSTREAM_DATADIR/decompress.zsend.bz2"
+typeset orig="$BACKDIR/decompress.orig"
+typeset out_default="$BACKDIR/decompress-default.out"
+typeset out_zstd="$BACKDIR/decompress-zstd.out"
+
+typeset -a records=(2,0 3,0 128,131072)
+typeset -a zstd_records=(2,0,zstd 3,0,zstd 128,131072,zstd)
+
+bzcat "$src" > "$orig"
+
+# Decompress without specifying type
+log_must eval "zstream decompress ${records[*]} < '$orig' > '$out_default'"
+
+# Decompress specifying zstd
+log_must eval "zstream decompress ${zstd_records[*]} < '$orig' > '$out_zstd'"
+
+# Both outputs must be identical
+log_must cmp -s "$out_default" "$out_zstd"
+
+log_pass "Explicit correct compression type matches default."

tests/zfs-tests/tests/functional/zstream/zstream_decompress_005_pos.ksh

@@ -0,0 +1,115 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that zstream decompress with "off" as the compression type
+# changes record headers to mark them as uncompressed but leaves the
+# actual data payload untouched.
+#
+# Strategy:
+# 1. Decompress selected records with type "off"
+# 2. Verify via zstream dump that selected records now show
+#    compression type = 0 and logical_size equals the original
+#    compressed_size (i.e., the header now claims the record is
+#    uncompressed at the smaller, originally-compressed size)
+#
+# Note: we intentionally do not attempt to zfs receive the resulting
+# stream. The data payloads are still compressed despite the header's
+# claims otherwise, so the affected WRITE records are now inconsistent
+# with the dnodes in the corresponding OBJECT records. zfs receive will
+# fail with EINVAL.
+#
+# zstream decompress off is intended to correct a specific error case
+# in which these header adjustments bring the WRITE records into alignment
+# with their dnodes rather than disrupting that relationship.
+#
+
+verify_runnable "both"
+
+log_assert "Verify decompress with 'off' changes headers but not data."
+
+typeset src="$ZSTREAM_DATADIR/decompress.zsend.bz2"
+typeset orig="$BACKDIR/decompress.orig"
+typeset off_out="$BACKDIR/decompress-off.out"
+typeset orig_dump="$BACKDIR/decompress-orig.dump"
+typeset off_dump="$BACKDIR/decompress-off.dump"
+
+typeset -a records=(2,0 3,0 128,131072)
+typeset -a off_records=(2,0,off 3,0,off 128,131072,off)
+
+bzcat "$src" > "$orig"
+
+log_must eval "zstream decompress ${off_records[*]} < '$orig' > '$off_out'"
+
+# Dump both streams
+log_must eval "zstream dump -v < '$orig' > '$orig_dump' 2>&1"
+log_must eval "zstream dump -v < '$off_out' > '$off_dump' 2>&1"
+
+# Verify selected records show compression type = 0 with same logical_size
+typeset failed=""
+for rec in "${records[@]}"; do
+	typeset obj=${rec%,*}
+	typeset off=${rec#*,}
+
+	typeset orig_line=$(awk \
+	    "/^WRITE object = $obj .* offset = $off /" \
+	    "$orig_dump")
+	typeset orig_lsize=$(echo "$orig_line" | \
+	    sed 's/.*logical_size = \([0-9]*\).*/\1/')
+	typeset orig_csize=$(echo "$orig_line" | \
+	    sed 's/.*compressed_size = \([0-9]*\).*/\1/')
+	typeset orig_ctype=$(echo "$orig_line" | \
+	    sed 's/.*compression type = \([0-9]*\).*/\1/')
+
+	typeset off_line=$(awk \
+	    "/^WRITE object = $obj .* offset = $off /" \
+	    "$off_dump")
+	typeset off_lsize=$(echo "$off_line" | \
+	    sed 's/.*logical_size = \([0-9]*\).*/\1/')
+	typeset off_csize=$(echo "$off_line" | \
+	    sed 's/.*compressed_size = \([0-9]*\).*/\1/')
+	typeset off_ctype=$(echo "$off_line" | \
+	    sed 's/.*compression type = \([0-9]*\).*/\1/')
+
+	if [[ "$orig_ctype" == "0" ]]; then
+		log_note "Record $rec: original compression type is 0"
+		failed="$failed ${rec}(orig ctype)"
+	fi
+	if [[ "$off_ctype" != "0" ]]; then
+		log_note "Record $rec: modified compression type is not 0"
+		failed="$failed ${rec}(modified ctype)"
+	fi
+	if [[ "$off_lsize" != "$orig_csize" ]]; then
+		log_note "Record $rec: modified logical_size != original " \
+		    "compressed_size ($orig_lsize -> $off_lsize)"
+		failed="$failed ${rec}(lsize)"
+	fi
+	if [[ "$off_csize" != "0" ]]; then
+		log_note "Record $rec: modified compressed_size != 0 " \
+		    "($orig_csize -> $off_csize)"
+		failed="$failed ${rec}(csize)"
+	fi
+done
+
+[[ -z $failed ]] || \
+    log_fail "Header verification failed for:$failed"
+
+log_pass "Decompress with 'off' changes headers correctly."

tests/zfs-tests/tests/functional/zstream/zstream_decompress_006_neg.ksh

@@ -0,0 +1,61 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that zstream decompress prints a warning to stderr for encrypted
+# WRITES, attempts to decompress anyway, fails, and leaves stream unchanged.
+#
+# Strategy:
+# 1. Select compressed encrypted records from decompress-crypt.zsend
+# 2. Attempt to decompress them
+# 3. Verify stderr contains warnings for each record
+# 4. Verify output stream is byte-identical to input
+#
+
+verify_runnable "both"
+
+log_assert "Verify that zstream decompress handles encrypted records correctly."
+
+typeset src="$ZSTREAM_DATADIR/decompress-crypt.zsend.bz2"
+typeset orig="$BACKDIR/decompress-crypt.orig"
+typeset output="$BACKDIR/decompress-crypt.out"
+typeset errfile="$BACKDIR/decompress-crypt.err"
+
+
+
+typeset -a records=(2,0 3,0 36,0)
+
+bzcat "$src" > "$orig"
+
+# Attempt to decompress encrypted records
+zstream decompress ${records[*]} < "$orig" > "$output" 2>"$errfile"
+
+# Output stream must be identical to input
+log_must cmp -s "$orig" "$output"
+
+# Stderr should contain warnings about each record
+typeset errcount=$(wc -l < "$errfile")
+if [[ $errcount -ne 6 ]]; then
+	log_fail "Expected 6 messages on stderr, got $errcount"
+fi
+log_note "Got $errcount lines of warning output (expected)"
+
+log_pass "Encrypted records refuse to decompress."

tests/zfs-tests/tests/functional/zstream/zstream_drop_record_001_pos.ksh

@@ -16,8 +16,7 @@
 # Copyright (c) 2026 by ConnectWise. All rights reserved.
 #
 
-. $STF_SUITE/tests/functional/rsend/rsend.kshlib
-. $STF_SUITE/include/math.shlib
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
 
 #
 # Description:
@@ -32,10 +31,10 @@
 verify_runnable "both"
 
 log_assert "Verify zstream drop_record correctly drops records."
-log_onexit cleanup_pool $POOL2
+log_onexit cleanup_pool $POOL
 
-typeset sendfs=$POOL2/fs
-typeset recvfs=$POOL2/fs2
+typeset sendfs=$POOL/fs
+typeset recvfs=$POOL/fs2
 typeset stream=$BACKDIR/stream
 typeset filtered=$BACKDIR/filtered
 typeset dump=$BACKDIR/dump

tests/zfs-tests/tests/functional/zstream/zstream_dump_001_pos.ksh

@@ -17,12 +17,11 @@
 # Copyright (c) 2020 by Datto, Inc. All rights reserved.
 #
 
-. $STF_SUITE/tests/functional/rsend/rsend.kshlib
-. $STF_SUITE/include/math.shlib
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
 
 #
 # Description:
-# Verify compression features show up in zstream dump
+# Verify that compression features show up in zstream dump
 #
 # Strategy:
 # 1. Create a full compressed send stream
@@ -36,11 +35,11 @@
 verify_runnable "both"
 
 log_assert "Verify zstream dump correctly interprets compressed send streams."
-log_onexit cleanup_pool $POOL2
+log_onexit cleanup_pool $POOL
 
-typeset sendfs=$POOL2/fs
-typeset streamfs=$POOL2/fs2
-typeset recvfs=$POOL2/fs3
+typeset sendfs=$POOL/fs
+typeset streamfs=$POOL/fs2
+typeset recvfs=$POOL/fs3
 
 log_must zfs create -o compress=lz4 $sendfs
 log_must zfs create -o compress=lz4 $streamfs

tests/zfs-tests/tests/functional/zstream/zstream_dump_002_pos.ksh

@@ -0,0 +1,84 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that zstream dump -v output is as expected for pregenerated
+# same-endian, neutral (no BEGIN nvlists), and XDR-encoded test streams.
+# NV_ENCODE_NATIVE-encoded BEGIN records aren't readable on opposite-endian
+# systems, so for these the output of zstream dump varies according to
+# host endianness.
+#
+# Strategy:
+# 1. For each of the test streams, run zstream dump -v
+# 2. Compare stdout+stderr with the corresponding reference dump
+#
+
+verify_runnable "both"
+
+log_assert "Verify zstream dump -v output matches reference dump files."
+
+typeset sys_endian=$(get_system_endian)
+
+typeset -a streams=(
+	decompress
+	decompress-crypt
+	little-endian-long-payloads
+	big-endian-long-payloads
+	big-endian-all-drr-types-base-XDR
+	big-endian-all-drr-types-incr-XDR
+	little-endian-all-drr-types-base-XDR
+	little-endian-all-drr-types-incr-XDR
+)
+
+if [[ $sys_endian == "little" ]]; then
+	streams+=(
+		little-endian-all-drr-types-base-NATIVE
+		little-endian-all-drr-types-incr-NATIVE
+	)
+else
+	streams+=(
+		big-endian-all-drr-types-base-NATIVE
+		big-endian-all-drr-types-incr-NATIVE
+	)
+fi
+
+typeset failed=""
+
+for stem in "${streams[@]}"; do
+	typeset abbrev=$(get_stream_abbrev "$stem")
+	typeset ref_src="$ZSTREAM_DATADIR/${abbrev}-new.dump.bz2"
+	typeset send_src="$ZSTREAM_DATADIR/${stem}.zsend.bz2"
+	typeset ref="$BACKDIR/${abbrev}-new.dump"
+	typeset out="$BACKDIR/${abbrev}-out.dump"
+
+	bzcat "$send_src" | zstream dump -v > "$out" 2>&1
+	bzcat "$ref_src" > "$ref"
+
+	if ! diff -q "$ref" "$out" > /dev/null 2>&1; then
+		log_note "MISMATCH: $stem (abbrev $abbrev)"
+		log_note "$(diff "$ref" "$out")"
+		failed="$failed $stem"
+	fi
+done
+
+[[ -z $failed ]] || log_fail "Dump output mismatch for:$failed"
+
+log_pass "zstream dump -v output matches reference dump files."

tests/zfs-tests/tests/functional/zstream/zstream_dump_003_pos.ksh

@@ -0,0 +1,92 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that zstream dump -v output for all same-endian and XDR-encoded
+# test streams matches that of the previous version of zstream, with the
+# following exceptions:
+#
+# 1. Add a line that describes the nvlist packing format for BEGIN records
+# 2. Include DRR_OBJECT_RANGE and DRR_REDACT records in end summary
+#
+# The previous version of zstream does not dump opposite-endian streams
+# correctly, so these don't have a comparison basis.
+#
+
+verify_runnable "both"
+
+log_assert "Verify old-vs-new dump diff contains only expected additions."
+
+typeset sys_endian=$(get_system_endian)
+
+typeset -a streams=(
+	little-endian-long-payloads
+	big-endian-long-payloads
+	little-endian-all-drr-types-base-XDR
+	little-endian-all-drr-types-incr-XDR
+	big-endian-all-drr-types-base-XDR
+	big-endian-all-drr-types-incr-XDR
+)
+
+if [[ $sys_endian == "little" ]]; then
+	streams+=(
+		little-endian-all-drr-types-base-NATIVE
+		little-endian-all-drr-types-incr-NATIVE
+	)
+else
+	streams+=(
+		big-endian-all-drr-types-base-NATIVE
+		big-endian-all-drr-types-incr-NATIVE
+	)
+fi
+
+typeset failed=""
+
+for stem in "${streams[@]}"; do
+
+	typeset abbrev=$(get_stream_abbrev "$stem")
+	typeset send_src="$ZSTREAM_DATADIR/${stem}.zsend.bz2"
+	typeset old_src="$ZSTREAM_DATADIR/${abbrev}-old.dump.bz2"
+	typeset new_dump="$BACKDIR/${abbrev}-new.dump"
+	typeset old_dump="$BACKDIR/${abbrev}-old.dump"
+	typeset filtered="$BACKDIR/${abbrev}-filtered.dump"
+
+	bzcat "$send_src" | zstream dump -v > "$new_dump" 2>&1
+	bzcat "$old_src" > "$old_dump"
+
+	# Remove the lines that are new additions:
+	# 1. "nvlist encoding = ..." lines
+	# 2. Summary lines for DRR_OBJECT_RANGE and DRR_REDACT
+	grep -v '^nvlist encoding = ' "$new_dump" | \
+	    grep -v 'Total DRR_OBJECT_RANGE records' | \
+	    grep -v 'Total DRR_REDACT records' > "$filtered"
+
+	if ! diff -q "$old_dump" "$filtered" > /dev/null 2>&1; then
+		log_note "MISMATCH after filtering: $stem (abbrev $abbrev)"
+		log_note "$(diff "$old_dump" "$filtered")"
+		failed="$failed $stem"
+	fi
+done
+
+[[ -z $failed ]] || \
+    log_fail "Filtered new dump did not match old dump for:$failed"
+
+log_pass "Old-vs-new dump diff contains only expected additions."

tests/zfs-tests/tests/functional/zstream/zstream_dump_004_neg.ksh

@@ -0,0 +1,88 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that zstream dump on non-native-endian NATIVE-encoded streams
+# eventually exits with code ENOTSUP (95 on Linux, 45 on FreeBSD) but
+# still dumps the complete stream (minus nondecodable nvlists) and
+# prints the rollup summary.
+#
+# Strategy:
+# 1. Determine system endianness to identify non-native NATIVE streams
+# 2. Run zstream dump -v on each non-native NATIVE stream
+# 3. Verify exit code is 95 (ENOTSUP)
+# 4. Verify that SUMMARY section is present and complete
+#
+
+verify_runnable "both"
+
+log_assert "Non-native NATIVE-encoded streams exit with code 45 or 95."
+
+typeset sys_endian=$(get_system_endian)
+
+if [[ $sys_endian == "little" ]]; then
+	typeset -a streams=(
+		big-endian-all-drr-types-base-NATIVE
+		big-endian-all-drr-types-incr-NATIVE
+	)
+else
+	typeset -a streams=(
+		little-endian-all-drr-types-base-NATIVE
+		little-endian-all-drr-types-incr-NATIVE
+	)
+fi
+
+typeset failed=""
+
+for stem in "${streams[@]}"; do
+	typeset out="$BACKDIR/${stem}-out.dump"
+	typeset stream="$ZSTREAM_DATADIR/${stem}.zsend.bz2"
+
+	bzcat "$stream" | zstream dump -v > "$out" 2>&1
+	typeset rc=$?
+
+	if [[ $rc -ne 45 && $rc -ne 95 ]]; then
+		log_note "$stem: expected exit code 45 or 95, got $rc"
+		failed="$failed ${stem}(rc=$rc)"
+	fi
+
+	# Verify the SUMMARY section is present
+	if ! grep -q '^SUMMARY:' "$out"; then
+		log_note "$stem: missing SUMMARY section"
+		failed="$failed ${stem}(no-summary)"
+	fi
+
+	# Verify key summary lines are present
+	if ! grep -q 'Total DRR_BEGIN records' "$out"; then
+		log_note "$stem: missing DRR_BEGIN in summary"
+		failed="$failed ${stem}(incomplete-summary)"
+	fi
+
+	if ! grep -q 'Total stream length' "$out"; then
+		log_note "$stem: missing total stream length in summary"
+		failed="$failed ${stem}(no-stream-length)"
+	fi
+done
+
+[[ -z $failed ]] || \
+    log_fail "Non-native NATIVE stream check failed:$failed"
+
+log_pass "Non-native NATIVE-encoded streams exit with code 45 or 95."

tests/zfs-tests/tests/functional/zstream/zstream_recompress_001_pos.ksh

@@ -16,8 +16,7 @@
 # Copyright (c) 2022 by Delphix. All rights reserved.
 #
 
-. $STF_SUITE/tests/functional/rsend/rsend.kshlib
-. $STF_SUITE/include/math.shlib
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
 
 #
 # Description:
@@ -36,10 +35,10 @@
 verify_runnable "both"
 
 log_assert "Verify zstream recompress correctly modifies send streams."
-log_onexit cleanup_pool $POOL2
+log_onexit cleanup_pool $POOL
 
-typeset sendfs=$POOL2/fs
-typeset recvfs=$POOL2/fs2
+typeset sendfs=$POOL/fs
+typeset recvfs=$POOL/fs2
 
 log_must zfs create -o compress=lz4 $sendfs
 typeset dir=$(get_prop mountpoint $sendfs)

tests/zfs-tests/tests/functional/zstream/zstream_recompress_002_pos.ksh

@@ -0,0 +1,51 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2024 by the OpenZFS project. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that recompressing a send stream and then decompressing it
+# with zstream produces a stream identical to the original.
+#
+# Strategy:
+# 1. Create a filesystem with compressible data
+# 2. Generate a replication send stream
+# 3. Pipe the stream through zstream recompress lz4 | zstream recompress off
+# 4. Verify the result is byte-identical to the original stream
+#
+
+verify_runnable "both"
+
+log_assert "Verify zstream recompress round-trip produces identical stream."
+log_onexit cleanup_pool $POOL
+
+typeset sendfs=$POOL/fs
+
+log_must zfs create $sendfs
+typeset dir=$(get_prop mountpoint $sendfs)
+write_compressible $dir 16m
+log_must zfs snapshot $sendfs@snap
+
+log_must eval "zfs send -R $sendfs@snap >$BACKDIR/original"
+log_must eval "zstream recompress lz4 <$BACKDIR/original | \
+    zstream recompress off >$BACKDIR/roundtrip"
+
+log_must cmp $BACKDIR/original $BACKDIR/roundtrip
+
+log_pass "zstream recompress round-trip produces identical stream."

tests/zfs-tests/tests/functional/zstream/zstream_recompress_003_pos.ksh

@@ -0,0 +1,64 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that zstream recompress with zstd at level 10 produces a smaller
+# stream that receives with identical file contents.
+#
+# Strategy:
+# 1. Receive the original stream and compute file hashes as baseline
+# 2. Recompress the stream with zstd-10
+# 3. Verify the recompressed stream is smaller than the original
+# 4. Receive the recompressed stream and verify file hashes match
+#
+
+verify_runnable "both"
+
+log_assert "Verify zstream recompress with zstd-10 produces smaller stream."
+log_onexit cleanup_pool $POOL
+
+typeset src="$ZSTREAM_DATADIR/decompress.zsend.bz2"
+typeset orig="$BACKDIR/recompress.orig"
+typeset recompressed="$BACKDIR/recompress-zstd10.out"
+typeset orig_hash="$BACKDIR/hash-baseline.txt"
+typeset rc_hash="$BACKDIR/hash-rc.txt"
+
+bzcat "$src" > "$orig"
+
+# Baseline: receive original and hash
+recv_and_hash "$orig_hash" "$orig" cleanup
+
+# Recompress with zstd at level 10
+log_must eval "zstream recompress -l 10 zstd \
+    < '$orig' > '$recompressed'"
+
+# Verify size is smaller
+typeset orig_size=$(wc -c < "$orig")
+typeset recomp_size=$(wc -c < "$recompressed")
+log_note "Original size: $orig_size, recompressed size: $recomp_size"
+[[ $recomp_size -lt $orig_size ]] || \
+    log_fail "Recompressed stream ($recomp_size) not smaller than original ($orig_size)"
+
+# Receive recompressed and verify
+recv_and_hash "$rc_hash" "$recompressed" cleanup
+log_must diff "$orig_hash" "$rc_hash"
+
+log_pass "zstream recompress with zstd-10 produces smaller stream."

tests/zfs-tests/tests/functional/zstream/zstream_recompress_004_pos.ksh

@@ -0,0 +1,61 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that zstream recompress with "off" produces a larger (uncompressed)
+# stream that still receives with identical file contents.
+#
+# Strategy:
+# 1. Receive the original stream and compute file hashes as baseline
+# 2. Recompress with "off" (decompress)
+# 3. Verify the output stream is larger than the original
+# 4. Receive and verify file hashes match
+#
+
+verify_runnable "both"
+
+log_assert "Verify zstream recompress with 'off' produces larger stream."
+log_onexit cleanup_pool $POOL
+
+typeset src="$ZSTREAM_DATADIR/decompress.zsend.bz2"
+typeset orig="$BACKDIR/recompress.orig"
+typeset uncompressed="$BACKDIR/recompress-off.out"
+
+bzcat "$src" > "$orig"
+
+# Baseline
+recv_and_hash "$BACKDIR/hash-baseline.txt" "$orig" cleanup
+
+# Recompress with off
+log_must eval "zstream recompress off < '$orig' > '$uncompressed'"
+
+# Verify size is larger
+typeset orig_size=$(wc -c < "$orig")
+typeset uncomp_size=$(wc -c < "$uncompressed")
+log_note "Original size: $orig_size, uncompressed size: $uncomp_size"
+[[ $uncomp_size -gt $orig_size ]] || \
+    log_fail "Uncompressed stream ($uncomp_size) not larger than original ($orig_size)"
+
+# Receive and verify
+recv_and_hash "$BACKDIR/hash-off.txt" "$uncompressed" cleanup
+log_must diff "$BACKDIR/hash-baseline.txt" "$BACKDIR/hash-off.txt"
+
+log_pass "zstream recompress with 'off' produces larger stream."

tests/zfs-tests/tests/functional/zstream/zstream_recompress_005_pos.ksh

@@ -0,0 +1,62 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+# Verify that zstream recompress lz4 of a zstd-5-compressed input stream
+# yields a stream with a nonidentical size that zfs receives with identical
+# file contents.
+#
+# Strategy:
+# 1. Receive the original stream and compute file hashes as baseline
+# 2. Recompress with lz4
+# 3. Verify the output stream size differs from the original
+# 4. Receive and verify file hashes match
+#
+
+verify_runnable "both"
+
+log_assert "Verify zstream recompress with lz4 preserves data."
+log_onexit cleanup_pool $POOL
+
+typeset src="$ZSTREAM_DATADIR/decompress.zsend.bz2"
+typeset orig="$BACKDIR/recompress.orig"
+typeset lz4_out="$BACKDIR/recompress-lz4.out"
+
+bzcat "$src" > "$orig"
+
+# Baseline
+recv_and_hash "$BACKDIR/hash-baseline.txt" "$orig" cleanup
+
+# Recompress with lz4
+log_must eval "zstream recompress lz4 < '$orig' > '$lz4_out'"
+
+# Verify size is different
+typeset orig_size=$(wc -c < "$orig")
+typeset lz4_size=$(wc -c < "$lz4_out")
+log_note "Original size: $orig_size, lz4 size: $lz4_size"
+[[ $lz4_size -ne $orig_size ]] || \
+    log_fail "LZ4 stream size ($lz4_size) same as original ($orig_size)"
+
+# Receive and verify
+recv_and_hash "$BACKDIR/hash-lz4.txt" "$lz4_out" cleanup
+log_must diff "$BACKDIR/hash-baseline.txt" "$BACKDIR/hash-lz4.txt"
+
+log_pass "zstream recompress with lz4 preserves data."

tests/zfs-tests/tests/functional/zstream/zstream_redup_001_pos.ksh

@@ -0,0 +1,79 @@
+#!/bin/ksh -p
+# SPDX-License-Identifier: CDDL-1.0
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2026 by Garth Snyder. All rights reserved.
+#
+
+. $STF_SUITE/tests/functional/zstream/zstream.kshlib
+
+#
+# Description:
+#
+# Verify that zstream redup produces output identical to input for same-endian
+# test streams. These input files contain no dedup records. However, the round
+# trip does involve the full pipeline as well as validating and regenerating all
+# checksums, so this is a useful check.
+#
+# Strategy:
+# 1. For each of the same-endian test streams, decompress with bzcat
+# 2. Pipe through zstream redup
+# 3. Compare with cmp against the original decompressed stream
+#
+
+verify_runnable "both"
+
+log_assert "Verify zstream redup is an identity transform on non-dedup streams."
+
+typeset sys_endian=$(get_system_endian)
+
+if [[ $sys_endian == "little" ]]; then
+	typeset -a streams=(
+		decompress
+		decompress-crypt
+		little-endian-all-drr-types-base-NATIVE
+		little-endian-all-drr-types-base-XDR
+		little-endian-all-drr-types-incr-NATIVE
+		little-endian-all-drr-types-incr-XDR
+	)
+else
+	typeset -a streams=(
+		big-endian-all-drr-types-base-NATIVE
+		big-endian-all-drr-types-base-XDR
+		big-endian-all-drr-types-incr-NATIVE
+		big-endian-all-drr-types-incr-XDR
+	)
+fi
+
+typeset failed=""
+
+for stem in "${streams[@]}"; do
+	typeset src="$ZSTREAM_DATADIR/${stem}.zsend.bz2"
+	typeset orig="$BACKDIR/${stem}.orig"
+	typeset redup_out="$BACKDIR/${stem}.redup"
+
+	bzcat "$src" > "$orig"
+	zstream redup "$orig" > "$redup_out"
+
+	if ! cmp -s "$orig" "$redup_out"; then
+		log_note "MISMATCH: zstream redup output differs for $stem"
+		failed="$failed $stem"
+	fi
+
+	rm -f "$orig" "$redup_out"
+done
+
+[[ -z $failed ]] || log_fail "Redup identity check failed for:$failed"
+
+log_pass "zstream redup is an identity transform on non-dedup streams."