Fix special characters in .mvn/jvm.config (fix #11363, #11485 and #11486) #11365

gnodet · 2025-10-30T14:39:42Z

This PR fixes multiple related issues with special character handling in .mvn/jvm.config files by unifying the parsing approach to use JvmConfigParser.java on both Unix and Windows platforms.

Issues Fixed

MNG-11363: Pipe symbols (|) in jvm.config cause shell parsing errors
MNG-11485: Non-POSIX xargs -0 fails on AIX, FreeBSD, and other systems
MNG-11486: sed command fails when paths contain @ symbols

Related PRs

Supersedes Replace xargs -0 with POSIX-compliant alternative (fixes #11485) #11489 (MNG-11485: POSIX compliance)
Supersedes [#11486] replace '@' with pipe in sed #11499 (MNG-11486: @ symbols in paths)

Solution

Instead of using complex shell/batch script parsing with awk, sed, and xargs, this PR uses the existing JvmConfigParser.java on both platforms:

Unix (`mvn`)

Compiles JvmConfigParser.java to a temporary directory
Runs the parser to process jvm.config
Cleans up temporary files
Falls back gracefully if compilation fails

Windows (`mvn.cmd`)

Already used JvmConfigParser.java (no functional changes)
Removed debug output
Updated comments

Benefits

✅ Correctness: Handles all edge cases reliably (pipes, quotes, @, etc.)
✅ POSIX Compliance: No non-standard tools needed (xargs -0, GNU-specific awk/sed)
✅ Maintainability: Single implementation for both platforms
✅ Consistency: Same behavior on Windows and Unix

Test Coverage

✅ MavenITgh11363PipeSymbolsInJvmConfigTest - Tests pipe symbols in jvm.config
✅ MavenITgh11485AtSignInJvmConfigTest - Tests @ symbols in jvm.config (important for Jenkins workspaces like workspace/project_PR-350@2)

Performance Impact

Unix script is ~200ms slower (320ms → 520ms for mvn -v) due to Java compilation, but this is acceptable for the correctness and maintainability benefits.

Examples

Pipe symbols (MNG-11363)

# .mvn/jvm.config
-Dhttp.nonProxyHosts=de|*.de|my.company.mirror.de

✅ Now works correctly: de|*.de|my.company.mirror.de

Quoted pipes

# .mvn/jvm.config
-Dprop.with.pipes="value|with|pipes"

✅ Now works correctly: value|with|pipes

@ symbols in paths (MNG-11486)

# .mvn/jvm.config
-Dtest.path=${MAVEN_PROJECTBASEDIR}/workspace@2/test

✅ Now works correctly: /path/to/workspace@2/test

gnodet · 2025-11-01T07:39:42Z

Updated all commit messages to use [gh-11363] instead of [MNG-11363] to follow the correct issue reference format.

gnodet · 2025-11-01T08:15:54Z

Fixed the integration test failures. The issue was that the previous implementation was escaping pipes in concat_lines but then using eval which still interpreted them as shell operators.

The updated fix:

Uses awk to escape unquoted pipes with backslashes in the concat_lines function
Uses eval with the escaped MAVEN_OPTS to properly handle the pipes

This ensures that pipe symbols in JVM arguments (like -Dhttp.nonProxyHosts=de|*.de) are treated as literal characters rather than shell pipe operators.

All previously failing tests now pass:

✅ MavenITgh11363PipeSymbolsInJvmConfigTest
✅ MavenITmng4559MultipleJvmArgsTest
✅ MavenITmng5889FindBasedir
✅ MavenITmng6223FindBasedir
✅ MavenITmng8598JvmConfigSubstitutionTest

The concat_lines function in the Unix mvn script was using xargs -n 1 which split arguments by whitespace, causing pipe symbols (|) in JVM arguments to be interpreted as shell command separators. This fix replaces the problematic implementation with a line-by-line reader that preserves quoted arguments and special characters while maintaining all existing functionality: - Comments are still removed - Empty lines are still filtered out - Variable substitution (MAVEN_PROJECTBASEDIR) still works - Pipe symbols and other special characters are preserved The Windows mvn.cmd seems too limited, so we use a custom java class to parse the jvm.config file and process it. Added integration test to verify pipe symbols in jvm.config work correctly and don't cause shell parsing errors.

cstamas

One nit: why not use same Java helper on all platforms?

gnodet · 2025-11-05T13:34:33Z

One nit: why not use same Java helper on all platforms?

Answered in the description.
When doing perfs measurements, running mvn -v went from 320 ms (using awk) to 520 ms (using the java parser).
So I don't think it's better to keep two separate solutions at this point.

- Always initialize JVM_CONFIG_MAVEN_OPTS to empty to avoid inheriting stale values from environment - Add debug output to see actual values during IT runs - This prevents previous test runs from interfering with current test

Write JVM_CONFIG_MAVEN_OPTS and MAVEN_OPTS values to mvn-debug.txt in the project directory so we can check what values are being used during IT runs.

- Use random temp directory for each invocation to avoid conflicts - Capture stderr from Java parser to debug output file - Clean up temp directory after use - This ensures complete isolation between different Maven invocations

- Removed debug output to mvn-debug.txt - Cleaned up error capture logic - Ready for production use

This reverts commit 199b457.

gnodet · 2025-11-26T08:40:29Z

The ITs are unstable, especially on windows...

This commit unifies the .mvn/jvm.config parsing approach by using JvmConfigParser.java on both Unix and Windows platforms, solving multiple related issues with special character handling. Issues Fixed: - MNG-11363: Pipe symbols (|) in jvm.config cause shell parsing errors - MNG-11485: Non-POSIX xargs -0 fails on AIX, FreeBSD, and other systems - MNG-11486: sed command fails when paths contain @ symbols Changes: - Modified mvn (Unix): Replaced awk/sed pipeline with JvmConfigParser.java compilation and execution. This ensures POSIX compliance and handles all special characters (pipes, quotes, @, etc.) correctly. - Modified mvn.cmd (Windows): Removed debug output and updated comments to reflect all special characters now handled correctly. - Added integration test MavenITgh11485AtSignInJvmConfigTest to verify @ symbol handling in jvm.config files (important for Jenkins workspaces like workspace/project_PR-350@2). The Java-based parser approach: - Handles comment removal (both # at start and end-of-line) - Performs variable substitution (${MAVEN_PROJECTBASEDIR}) - Correctly parses quoted arguments with special characters - Works consistently across all platforms - Is POSIX-compliant (no xargs -0, awk, or complex sed needed) Performance Impact: Unix script is ~200ms slower (320ms → 520ms for mvn -v) due to Java compilation, but this is acceptable for the correctness, maintainability, and cross-platform consistency benefits. Related PRs: - Fixes apache#11365 (MNG-11363: pipe symbols) - Fixes apache#11489 (MNG-11485: POSIX compliance) - Fixes apache#11499 (MNG-11486: @ symbols in paths)

gnodet · 2025-12-02T12:46:46Z

@bmarwell this would solve all problems with parsing the jvm.config file I think. At least, make it easier to maintain.

apache-maven/src/assembly/maven/bin/mvn

apache-maven/src/assembly/maven/bin/JvmConfigParser.java

bmarwell · 2025-12-02T21:41:41Z

apache-maven/src/assembly/maven/bin/mvn.cmd

+set "JVM_CONFIG_PARSER_DIR=%TEMP%\mvn-jvm-parser-%RANDOM%-%RANDOM%"
+mkdir "%JVM_CONFIG_PARSER_DIR%"
+set "JVM_CONFIG_TEMP=%TEMP%\mvn-jvm-config-%RANDOM%.txt"
+"%JAVACMD:java.exe=javac.exe%" -d "%JVM_CONFIG_PARSER_DIR%" "%MAVEN_HOME%\bin\JvmConfigParser.java" >nul 2>&1


Same as on posix/unix: why run javac? We are on 17 and can easily run single-source files using java.

apache-maven/src/assembly/maven/bin/JvmConfigParser.java

- Add System.out.flush() after output to ensure data is written before exit (fixes non-deterministic test failures on Windows with JDK 25) - Extract parseJvmConfig() method (package-private) for testability - Keep System.exit(1) for IOException to prevent Maven from running with invalid jvm.config (fail-fast approach) - Add clear error messages with ERROR: prefix to stderr - Update Unix script to capture and propagate parser exit code - Update Windows script to check parser exit code and cleanup properly This addresses: - Non-deterministic Windows test failure in MavenITmng4559SpacesInJvmOptsTest - Testability concerns while maintaining fail-fast behavior - Clear error reporting for configuration issues

- Replace javac compilation + java execution with single 'java JvmConfigParser.java' command (source-launch mode available since JDK 11, Maven requires JDK 17+) - Simplifies scripts by eliminating temporary directories and compilation step - Add comprehensive error logging on both Unix and Windows for debugging non-deterministic issues (especially on Windows with JDK 25) - Error messages now include: * Exit code * jvm.config path * Maven basedir * Java command used * Full parser output (stdout and stderr) - Windows: Capture stdout and stderr separately for better diagnostics - Unix: Use sed to indent parser output for clarity Benefits: - Simpler, more maintainable scripts - Faster execution (no compilation step) - Better error diagnostics for troubleshooting - Consistent behavior across platforms

…symbols-jvm-config

Copilot

Pull request overview

This PR unifies the parsing of .mvn/jvm.config files across Unix and Windows platforms by introducing a Java-based parser (JvmConfigParser.java) that handles special characters correctly. Previously, shell/batch scripts used awk, sed, and xargs with platform-specific behavior that failed on special characters like pipes (|) and at signs (@).

Key Changes:

New JvmConfigParser.java parser handles quotes, pipes, @ symbols, and environment variable substitution
Unix mvn script now uses Java source-launch mode (JDK 11+) to run the parser
Windows mvn.cmd updated to use the same parser with improved error handling
Integration tests added for pipe symbols (MNG-11363) and @ signs (MNG-11485)

Reviewed changes

Copilot reviewed 8 out of 11 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
`apache-maven/src/assembly/maven/bin/JvmConfigParser.java`	New Java parser that correctly handles special characters in jvm.config
`apache-maven/src/assembly/maven/bin/mvn`	Unix script updated to use Java parser via source-launch mode
`apache-maven/src/assembly/maven/bin/mvn.cmd`	Windows batch script updated for consistency, improved error handling
`apache-maven/src/assembly/component.xml`	Assembly configuration updated to include .java files in distribution
`its/core-it-suite/src/test/java/org/apache/maven/it/MavenITgh11363PipeSymbolsInJvmConfigTest.java`	Integration test for pipe symbol handling
`its/core-it-suite/src/test/java/org/apache/maven/it/MavenITgh11485AtSignInJvmConfigTest.java`	Integration test for @ sign handling
`its/core-it-suite/src/test/resources/gh-11363-pipe-symbols-jvm-config/*`	Test resources for pipe symbol test
`its/core-it-suite/src/test/resources/gh-11485-at-sign/*`	Test resources for @ sign test

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…nflicts - Verifier: Save stdout/stderr to separate files (log-stdout.txt, log-stderr.txt) when not empty. This captures shell script debug output that occurs before Maven starts, which is useful for debugging Windows batch script issues on CI. - MavenITmng6255FixConcatLines: Use different log file names for each test method (log-lf.txt, log-crlf.txt) to avoid overwriting when all tests share the same resource directory.

Replace temp file approach with direct stdout capture using for /f. On Windows, temp files created by shell redirection may not be released immediately after the Java process exits, causing 'file in use' errors when the batch script tries to read them. Using 'for /f ... in (`command`)' captures stdout directly without intermediate temp files, avoiding the locking issue.

- mvn.cmd: Use 'cmd /c' to run JvmConfigParser in a subshell, ensuring file handles are properly released before reading the temp file. The previous 'for /f' with backticks approach did not correctly capture the Java output on Windows. - mvn, mvn.cmd: Add debug logging controlled by MAVEN_DEBUG_SCRIPT environment variable. When set, both scripts will output detailed information about jvm.config parsing, including: - Path to jvm.config file - Java command and arguments - Parser exit code and output - Final MAVEN_OPTS/JVM_CONFIG_MAVEN_OPTS value This helps diagnose jvm.config issues on CI systems where the log files are the primary debugging tool.

- mvn.cmd: Use 'cmd /c' to run JvmConfigParser in a subshell, ensuring file handles are properly released before reading the temp file. The previous 'for /f' with backticks approach did not correctly capture the Java output on Windows. - mvn, mvn.cmd: Add debug logging controlled by MAVEN_DEBUG_SCRIPT environment variable. When set, both scripts will output detailed information about jvm.config parsing, including: - Path to jvm.config file - Java command and arguments - Parser exit code and output - Final MAVEN_OPTS/JVM_CONFIG_MAVEN_OPTS value - Full JVM command line before launching Maven This helps diagnose jvm.config issues on CI systems where the log files are the primary debugging tool.

gnodet changed the title ~~[MNG-11363] Fix pipe symbol parsing in .mvn/jvm.config~~ Fix pipe symbol parsing in .mvn/jvm.config (fix #11363) Oct 30, 2025

gnodet added bug Something isn't working backport-to-4.0.x labels Oct 30, 2025

gnodet force-pushed the fix/mng-11363-pipe-symbols-jvm-config branch 2 times, most recently from 465eef5 to bdc5130 Compare November 1, 2025 07:39

gnodet force-pushed the fix/mng-11363-pipe-symbols-jvm-config branch 3 times, most recently from c2acb11 to c246af5 Compare November 3, 2025 19:20

gnodet force-pushed the fix/mng-11363-pipe-symbols-jvm-config branch from 4d00db1 to d1dc6f1 Compare November 5, 2025 11:13

gnodet requested a review from cstamas November 5, 2025 11:13

cstamas approved these changes Nov 5, 2025

View reviewed changes

gnodet force-pushed the fix/mng-11363-pipe-symbols-jvm-config branch from 4442e67 to d1dc6f1 Compare November 6, 2025 12:12

gnodet added 5 commits November 6, 2025 14:29

[apachegh-11363] Add debug output to file for IT verification

1c5a9c9

Write JVM_CONFIG_MAVEN_OPTS and MAVEN_OPTS values to mvn-debug.txt in the project directory so we can check what values are being used during IT runs.

[apachegh-11363] Remove debug output, ready for merge

199b457

- Removed debug output to mvn-debug.txt - Cleaned up error capture logic - Ready for production use

Revert "[apachegh-11363] Remove debug output, ready for merge"

178c6fa

This reverts commit 199b457.

gnodet marked this pull request as draft November 26, 2025 08:40

gnodet mentioned this pull request Dec 2, 2025

Replace xargs -0 with POSIX-compliant alternative (fixes #11485) #11489

Open

gnodet mentioned this pull request Dec 2, 2025

[#11486] replace '@' with pipe in sed #11499

Closed

gnodet changed the title ~~Fix pipe symbol parsing in .mvn/jvm.config (fix #11363)~~ [MNG-11363][MNG-11485][MNG-11486] Fix special characters in .mvn/jvm.config Dec 2, 2025

Fix possible issue

3abc5ce

bmarwell reviewed Dec 2, 2025

View reviewed changes

gnodet added 4 commits December 3, 2025 08:50

Fix description

ee17825

Merge remote-tracking branch 'origin/master' into fix/mng-11363-pipe-…

c0f1a94

…symbols-jvm-config

gnodet requested a review from Copilot December 3, 2025 12:33

Copilot started reviewing on behalf of gnodet December 3, 2025 12:33 View session

Copilot finished reviewing on behalf of gnodet December 3, 2025 12:34

Add debug info to windows batch file

36f56b6

Copilot AI reviewed Dec 3, 2025

View reviewed changes

gnodet added 4 commits December 3, 2025 23:23

gnodet changed the title ~~[MNG-11363][MNG-11485][MNG-11486] Fix special characters in .mvn/jvm.config~~ Fix special characters in .mvn/jvm.config (fix #11363, #11485 and #11486) Dec 4, 2025

gnodet marked this pull request as ready for review December 4, 2025 13:53

bmarwell approved these changes Dec 7, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix special characters in .mvn/jvm.config (fix #11363, #11485 and #11486) #11365

Fix special characters in .mvn/jvm.config (fix #11363, #11485 and #11486) #11365

gnodet commented Oct 30, 2025 •

edited

Loading

Uh oh!

gnodet commented Nov 1, 2025

Uh oh!

gnodet commented Nov 1, 2025

Uh oh!

cstamas left a comment

Uh oh!

gnodet commented Nov 5, 2025

Uh oh!

gnodet commented Nov 26, 2025

Uh oh!

gnodet commented Dec 2, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bmarwell Dec 2, 2025

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Fix special characters in .mvn/jvm.config (fix #11363, #11485 and #11486) #11365

Are you sure you want to change the base?

Fix special characters in .mvn/jvm.config (fix #11363, #11485 and #11486) #11365

Conversation

gnodet commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issues Fixed

Related PRs

Solution

Unix (mvn)

Windows (mvn.cmd)

Benefits

Test Coverage

Performance Impact

Examples

Pipe symbols (MNG-11363)

Quoted pipes

@ symbols in paths (MNG-11486)

Uh oh!

gnodet commented Nov 1, 2025

Uh oh!

gnodet commented Nov 1, 2025

Uh oh!

cstamas left a comment

Choose a reason for hiding this comment

Uh oh!

gnodet commented Nov 5, 2025

Uh oh!

gnodet commented Nov 26, 2025

Uh oh!

gnodet commented Dec 2, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bmarwell Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gnodet commented Oct 30, 2025 •

edited

Loading

Unix (`mvn`)

Windows (`mvn.cmd`)