Skip to content

feat: route real test program output to Test Results#1887

Merged
wenytang-ms merged 2 commits into
mainfrom
feat/test-output-to-test-results
Jul 16, 2026
Merged

feat: route real test program output to Test Results#1887
wenytang-ms merged 2 commits into
mainfrom
feat/test-output-to-test-results

Conversation

@wenytang-ms

@wenytang-ms wenytang-ms commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Problem

Running a test with the Java Test Runner splits output across two disconnected surfaces:

  • Program and debug output is delivered by java-debug as standard DAP output events, which normally appear only in the Debug Console.
  • Test results are driven by the runner's socket protocol and displayed in Test Results.

The runner also echoed raw JUnit/TestNG control frames into Test Results, producing protocol noise such as %TESTS, %TESTE, and TestNG JSON frames instead of useful user-facing output.

See eclipse-jdtls/eclipse.jdt.ls#3834 for the broader discussion.

Change

  • Mirror Debug Console output in Test Results. Attach a DebugAdapterTracker to the test debug session and forward all non-telemetry DAP output events through TestRun.appendOutput.
  • Keep output at the test-run level. DAP output is process-level and has no reliable test identifier, so it is not guessed onto individual test cases.
  • Correlate the exact debug session. Add a unique internal run ID to the launch configuration instead of matching sessions by display name.
  • Remove control protocol noise. JUnit no longer appends raw socket frames, and TestNG no longer echoes its JSON frames.
  • Preserve structured diagnostics. Test states, stack traces, and expected-vs-actual values continue to use the Testing API. TestNG runner-level error frames are surfaced as structured test errors.

This requires no changes to java-debug; it consumes the standard DAP events already emitted by the debug session.

Notes / limitations

  • TestRun.appendOutput is an uncategorized stream, so stdout, stderr, and console color separation is not preserved.
  • The Debug Console remains unchanged and continues to display the categorized output.

Validation

  • TypeScript compilation: clean
  • ESLint on changed source files: clean
  • Integration tests: 135 passing

The debuggee's stdout/stderr are delivered by java-debug as standard DAP
`output` events, which by default only surface in the Debug Console. This
splits program output (Debug Console) from test results (Test Results view)
across two separate surfaces, and the runner previously echoed the raw
JUnit/TestNG control protocol frames to Test Results as noise.

- Attach a DebugAdapterTracker to the test's own debug session and forward
  its DAP `output` events into the Test Results view (BaseRunner).
- Stop echoing the socket control protocol to Test Results: JUnit no longer
  appends every raw line, and TestNG no longer echoes its JSON frames.
- Attribute forwarded output to the running test when exactly one test is
  executing; fall back to run-level output when idle or when several tests
  run in parallel (attribution would only be a guess).
Correlate debug sessions with a unique launch marker, mirror non-telemetry DAP output at run level, and preserve structured TestNG runner errors while suppressing control protocol noise.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 8db89d6b-bcb8-42d2-9f14-78aab176fcd0

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the VS Code Java Test experience by routing real program/debug output into Test Results (in addition to the Debug Console) and by suppressing runner-protocol “noise” previously echoed into Test Results.

Changes:

  • Adds a DebugAdapterTracker for the specific Java test debug session (correlated via an internal run UUID) and forwards non-telemetry DAP output events to TestRun.appendOutput.
  • Stops echoing raw control/protocol frames into Test Results for JUnit and TestNG; preserves structured diagnostics via the Testing API.
  • Adds/updates tests to validate DAP-output forwarding and TestNG runner-error handling / control-frame ignoring.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
test/suite/TestNGAnalyzer.test.ts New unit tests for TestNG analyzer: structured runner error surfacing and ignoring control frames without test attributes.
test/suite/baseRunner.test.ts Adds unit tests covering BaseRunner.handleDebugAdapterMessage forwarding behavior and telemetry filtering.
src/runners/testngRunner/TestNGRunnerResultAnalyzer.ts Stops echoing raw TestNG frames; ignores control frames without test identifiers; surfaces runner-level error frames as structured test errors; makes duration parsing resilient.
src/runners/junitRunner/JUnitRunnerResultAnalyzer.ts Removes echoing of raw JUnit socket lines into Test Results (protocol-only stream), relying on structured messages instead.
src/runners/baseRunner/BaseRunner.ts Correlates the correct debug session via internal run ID, attaches a tracker, and mirrors non-telemetry DAP output events into Test Results with CRLF normalization.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@wenytang-ms wenytang-ms marked this pull request as ready for review July 15, 2026 08:18
@wenytang-ms

Copy link
Copy Markdown
Contributor Author
image image

@wenytang-ms wenytang-ms merged commit cc6a833 into main Jul 16, 2026
6 checks passed
@wenytang-ms wenytang-ms deleted the feat/test-output-to-test-results branch July 16, 2026 02:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants