MLE-31461: Expose zip archive read limits in Flux import commands by jonmille · Pull Request #656 · marklogic/flux

jonmille · 2026-07-23T15:16:00Z

Description:

Exposes the zip bomb protection options added in MLE-31218 through all five Flux import commands that read zip content, via both the CLI and the Java API.

Two new options are available on import-files, import-aggregate-xml-files, import-rdf-files, import-archive-files, and import-mlcp-archive-files:

CLI flag	Java API method	Connector option
`--zip-max-entry-bytes <n>`	`zipMaxUncompressedEntryBytes(long)`	`spark.marklogic.read.zip.maxUncompressedEntryBytes`
`--zip-max-entry-count <n>`	`zipMaxEntryCount(int)`	`spark.marklogic.read.zip.maxEntryCount`

Set to a positive integer to enable protection. Any value less than 1 (including 0) disables the limit. When not set, the connector default of 0 (unlimited) applies — no existing behaviour changes. Any explicitly set value (including 0 and negatives) is forwarded to the connector so operators can override future connector defaults.

The options map directly to the underlying connector options via makeOptions() → AbstractImportFilesCommand.loadDataset() → SparkSession.read().options(map).

Note: The connector options default to 0 (disabled) in 3.1.2 to avoid a breaking change. A future release will switch to protective defaults.

Refactoring: A new ReadCompressibleFilesOptions interface and ReadCompressibleFilesParams abstract class were introduced to eliminate duplication — the --partitions, --zip-max-entry-bytes, and --zip-max-entry-count options are now declared once in the shared base rather than in all five command classes.

Running the new tests

These are pure unit tests — no MarkLogic connection or Spark session required.

Run all five modified options test classes:

./gradlew :flux-cli:test \
  --tests "com.marklogic.flux.impl.importdata.ImportFilesOptionsTest" \
  --tests "com.marklogic.flux.impl.importdata.ImportAggregateXmlFilesOptionsTest" \
  --tests "com.marklogic.flux.impl.importdata.ImportRdfFilesOptionsTest" \
  --tests "com.marklogic.flux.impl.importdata.ImportArchiveFilesOptionsTest" \
  --tests "com.marklogic.flux.impl.importdata.ImportMlcpArchiveFilesOptionsTest"

Each test class contains:

zipProtectionOptions() — verifies positive values are passed to the correct connector options
zipProtectionOptionsNotSetByDefault() — verifies no options are added to the map when the flags are omitted
zipZeroAndNegativeValuesAreForwarded() — verifies that 0 and negative values ARE forwarded to the connector (so the connector can explicitly disable, overriding future defaults)

26 tests total across the 5 classes, all passing.

Linked ticket: MLE-31461
Depends on: MLE-31218 (connector options must be merged first): marklogic/marklogic-spark-connector#680

Adds --zip-max-entry-bytes and --zip-max-entry-count to all five zip-capable import commands (import-files, import-aggregate-xml-files, import-rdf-files, import-archive-files, import-mlcp-archive-files) via CLI flags and Java API methods. Both map directly to the connector options added in MLE-31218. Values of 0 or any negative other than -1 throw a FluxException. Defaults to -1 (unlimited) when not set.

Copilot

Pull request overview

Exposes zip-archive read-limit (“zip bomb protection”) settings across Flux import commands by adding new CLI flags and Java API methods, and wiring them into the connector options map, with unit tests validating option mapping and input validation.

Changes:

Added --zip-max-entry-bytes / --zip-max-entry-count to multiple import commands and forwarded them into Spark connector read options.
Added Java API methods (zipMaxUncompressedEntryBytes, zipMaxEntryCount) to the corresponding importer option interfaces.
Added/expanded unit tests to verify option mapping and invalid-value handling.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
flux-cli/src/main/java/com/marklogic/flux/impl/importdata/ImportFilesCommand.java	Adds zip limit CLI flags, validates values, and maps them into read options.
flux-cli/src/main/java/com/marklogic/flux/impl/importdata/ImportAggregateXmlFilesCommand.java	Adds zip limit CLI flags, validates values, and maps them into read options.
flux-cli/src/main/java/com/marklogic/flux/impl/importdata/ImportRdfFilesCommand.java	Adds zip limit CLI flags, validates values, and maps them into read options.
flux-cli/src/main/java/com/marklogic/flux/impl/importdata/ImportArchiveFilesCommand.java	Adds zip limit CLI flags, validates values, and maps them into read options.
flux-cli/src/main/java/com/marklogic/flux/impl/importdata/ImportMlcpArchiveFilesCommand.java	Adds zip limit CLI flags, validates values, and maps them into read options.
flux-cli/src/main/java/com/marklogic/flux/api/GenericFilesImporter.java	Adds Java API methods for zip entry byte/count limits.
flux-cli/src/main/java/com/marklogic/flux/api/AggregateXmlFilesImporter.java	Adds Java API methods for zip entry byte/count limits.
flux-cli/src/main/java/com/marklogic/flux/api/RdfFilesImporter.java	Adds Java API methods for zip entry byte/count limits.
flux-cli/src/main/java/com/marklogic/flux/api/ArchiveFilesImporter.java	Adds Java API methods for zip entry byte/count limits.
flux-cli/src/main/java/com/marklogic/flux/api/MlcpArchiveFilesImporter.java	Adds Java API methods for zip entry byte/count limits.
flux-cli/src/test/java/com/marklogic/flux/impl/importdata/ImportFilesOptionsTest.java	Extends options tests for zip flags and invalid values.
flux-cli/src/test/java/com/marklogic/flux/impl/importdata/ImportAggregateXmlFilesOptionsTest.java	Adds options tests for zip flags and invalid values.
flux-cli/src/test/java/com/marklogic/flux/impl/importdata/ImportRdfFilesOptionsTest.java	Adds options tests for zip flags and invalid values.
flux-cli/src/test/java/com/marklogic/flux/impl/importdata/ImportArchiveFilesOptionsTest.java	Adds options tests for zip flags and invalid values.
flux-cli/src/test/java/com/marklogic/flux/impl/importdata/ImportMlcpArchiveFilesOptionsTest.java	Adds options tests for zip flags and invalid values.

Comments suppressed due to low confidence (4)

flux-cli/src/main/java/com/marklogic/flux/impl/importdata/ImportFilesCommand.java:140

--zip-max-entry-bytes/-count accepts -1 (validated above), but makeOptions() currently only forwards values > 0. This drops an explicit -1 and prevents callers from forcing "disabled" if the connector default ever changes.

            if (zipMaxEntryBytes != null && zipMaxEntryBytes > 0) {
                options.put(Options.READ_ZIP_MAX_UNCOMPRESSED_ENTRY_BYTES, String.valueOf(zipMaxEntryBytes));
            }
            if (zipMaxEntryCount != null && zipMaxEntryCount > 0) {
                options.put(Options.READ_ZIP_MAX_ENTRY_COUNT, String.valueOf(zipMaxEntryCount));
            }

flux-cli/src/main/java/com/marklogic/flux/impl/importdata/ImportArchiveFilesCommand.java:146

makeOptions() validates that -1 is allowed for zip limits, but it only adds connector options when values are > 0. That drops an explicit -1 and makes it impossible to force "disabled" if connector defaults become protective later.

            if (zipMaxEntryBytes != null && zipMaxEntryBytes > 0) {
                options.put(Options.READ_ZIP_MAX_UNCOMPRESSED_ENTRY_BYTES, String.valueOf(zipMaxEntryBytes));
            }
            if (zipMaxEntryCount != null && zipMaxEntryCount > 0) {
                options.put(Options.READ_ZIP_MAX_ENTRY_COUNT, String.valueOf(zipMaxEntryCount));
            }

flux-cli/src/main/java/com/marklogic/flux/impl/importdata/ImportAggregateXmlFilesCommand.java:122

--zip-max-entry-bytes/-count permits -1, but makeOptions() only forwards values > 0. This silently ignores an explicit -1 and prevents callers from overriding future connector defaults.

            if (zipMaxEntryBytes != null && zipMaxEntryBytes > 0) {
                options.put(Options.READ_ZIP_MAX_UNCOMPRESSED_ENTRY_BYTES, String.valueOf(zipMaxEntryBytes));
            }
            if (zipMaxEntryCount != null && zipMaxEntryCount > 0) {
                options.put(Options.READ_ZIP_MAX_ENTRY_COUNT, String.valueOf(zipMaxEntryCount));
            }

flux-cli/src/main/java/com/marklogic/flux/impl/importdata/ImportMlcpArchiveFilesCommand.java:97

Zip limit flags accept -1, but makeOptions() only forwards values > 0. An explicit -1 is therefore ignored, so users can’t reliably disable protection if connector defaults change.

            if (zipMaxEntryBytes != null && zipMaxEntryBytes > 0) {
                options.put(Options.READ_ZIP_MAX_UNCOMPRESSED_ENTRY_BYTES, String.valueOf(zipMaxEntryBytes));
            }
            if (zipMaxEntryCount != null && zipMaxEntryCount > 0) {
                options.put(Options.READ_ZIP_MAX_ENTRY_COUNT, String.valueOf(zipMaxEntryCount));
            }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

rjrudin · 2026-07-23T15:43:42Z

+            description = "Maximum number of uncompressed bytes to read from a single zip entry. " +
+                "Accepts a positive integer. Protection is not enabled when this option is not set."
+        )
+        private Long zipMaxEntryBytes;


We can have a subclass of ReadFilesParams that declares the 3 options - the 2 new zip ones and partitions - to avoid duplication on the 6 classes.

Created ReadCompressibleFilesParams that now declare the 3 options

Updated the default values for --zip-max-entry-bytes and --zip-max-entry-count from -1 to 0. Anything >1 now disables the zip limit protections

jonmille · 2026-07-23T19:39:53Z

@rjrudin I am not sure why I can't reply to this message "Wouldn't zero work for a default? I think it's semantically the same as negative one, because a max of zero is meaningless, just like a max of negative one. I think it's worth updating the connector use anything less than 1 as a way to disable the behavior.", but I agree with your assessment. Defaulting to 0 simplifies the solution and makes more sense as a default value. I have converted the defaults for the new options from -1 to 0. I have similarly changed the defaults for the related options in the Spark Connector: marklogic/marklogic-spark-connector#680. I'll work on the refactor next

RitaChen609 · 2026-07-24T18:16:16Z

+    private int partitions;
+
+    @CommandLine.Option(
+        names = "--zip-max-entry-bytes",


It would be nice to consider --zip-max-uncompressed-entry-bytes to be clear and match the Options constant READ_ZIP_MAX_UNCOMPRESSED_ENTRY_BYTES

RitaChen609 · 2026-07-24T18:47:28Z

No documentation update visible in the diff. Should the new --zip-max-entry-bytes and --zip-max-entry-count flags be added to the online docs if needed?

This is a tricky one because we have 5 different commands, each on their own page in the docs today. When faced with this before, I've opted for duplicating the section in each of the 5 separate pages. I think that's appropriate here. Just need a small section explaining the options that are available to prevent against a zip bomb attack. I'd toss it near the bottom of the page for each command.

Copilot AI review requested due to automatic review settings July 23, 2026 15:16

jonmille requested review from RitaChen609, ngodugu-marklogic, rjdew-progress and rjrudin as code owners July 23, 2026 15:16

Copilot started reviewing on behalf of jonmille July 23, 2026 15:16 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

rjrudin requested changes Jul 23, 2026

View reviewed changes

MLE-31461 Updated default values

c1ba7ed

Updated the default values for --zip-max-entry-bytes and --zip-max-entry-count from -1 to 0. Anything >1 now disables the zip limit protections

MLE-31361 Refactor

1a993cb

jonmille requested a review from rjrudin July 24, 2026 13:49

rjrudin previously approved these changes Jul 24, 2026

View reviewed changes

Comment thread flux-cli/src/main/java/com/marklogic/flux/api/ReadCompressibleFilesOptions.java

MLE-31461 Added partitions JavaDoc

506323f

jonmille dismissed rjrudin’s stale review via 506323f July 24, 2026 14:23

RitaChen609 reviewed Jul 24, 2026

View reviewed changes

Uh oh!

Conversation

jonmille commented Jul 23, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rjrudin Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

jonmille Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jonmille commented Jul 23, 2026

Uh oh!

Uh oh!

RitaChen609 Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

RitaChen609 Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

rjrudin Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jonmille commented Jul 23, 2026 •

edited by atlassian Bot

Loading