Skip to content

CAMEL-26929: camel-cli-connector - the trace file starts again at 10 MB, and camel trace and the TUI follow the new file - #30028

Open
davsclaus wants to merge 1 commit into
mainfrom
CAMEL-26929-trace-file-cap
Open

davsclaus wants to merge 1 commit into
mainfrom
CAMEL-26929-trace-file-cap

Conversation

@davsclaus

Copy link
Copy Markdown
Contributor

JIRA: CAMEL-26929, item 6.

The problem

While tracing is on, FileCliConnectorTransport appends the new traces to ~/.camel/<pid>-trace.json, and nothing ever trims the file: 758 KB after ten minutes of a 2 s timer, and about 15 MB a minute at 100 exchanges/s. camel trace --tail reads it whole every time.

The change

  • Connector: when the next write would take the file past 10 MB (MAX_TRACE_FILE_SIZE), the file is deleted and the traces start a new one. A single snapshot larger than that is still written to an empty file.
  • camel trace (follow): a file that got smaller than when it was last read is a new file, so the reader reopens it and reads it from its start. Before, the open reader stayed on the deleted file.
  • TUI trace view: read from position 0 when the file is shorter than the position read so far. Before, length <= lastPos returned early, so the view never read the file again.

An older CLI or TUI that follows the file of a newer integration stops seeing new traces after the first new file, until it is started again. That only happens after 10 MB of traces.

Tests and docs

  • TraceFileSizeTest (connector): a full file starts again, a file with room is appended to, and an empty file takes data larger than the maximum.
  • CamelTraceActionTest.aTraceFileThatGotSmallerIsANewFile and DataRefreshMembershipTest.aTraceFileThatGotSmallerIsReadFromItsStart.
  • Connector suite green (127 tests).
  • cli-connector.adoc describes the trace file; the catalog copy is updated.

Not checked end to end: it needs 10 MB of traces.

Claude Code on behalf of davsclaus

🤖 Generated with Claude Code

…MB, and camel trace and the TUI follow the new file

While tracing is on, the connector appended every trace to ~/.camel/<pid>-trace.json without a limit: 758 KB after ten
minutes of a 2 s timer, about 15 MB a minute at 100 exchanges a second, and camel trace reads it whole for --tail. When
the file would grow past 10 MB, the connector now deletes it and the traces start a new file. camel trace (follow) and
the trace view of the TUI see the file become smaller and read the new file from its start; the TUI read nothing from
it again before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
@davsclaus davsclaus added this to the 4.24.0 milestone Nov 15, 2026
@davsclaus davsclaus added bug Something isn't working dsl labels Nov 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@gnodet

gnodet commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

🔄 Backport Bot

This bugfix targets main and may need porting to:

  • camel-4.22.x

Add a port/<branch> label to automatically create a backport PR for that branch.

Backport PRs will be created automatically on merge (once a port label is added). Comment /port to create them immediately.

ℹ️ If you push additional commits after /port, use /port again to update the port PRs.

@github-actions

Copy link
Copy Markdown
Contributor

🧪 CI tested the following changed modules:

  • catalog/camel-catalog
  • dsl/camel-cli-connector
  • dsl/camel-jbang/camel-jbang-core
  • dsl/camel-jbang/camel-jbang-plugin-tui

🔁 1 test passed only after a retry on JDK 21 (1 retried attempt)

Recovered flaky tests on JDK 21 (1)
Module Test Failed attempts First failure
dsl/camel-jbang/camel-jbang-plugin-tui AiPanelAcpTest.unknownSlashCommandIsForwardedToTheAgentWithoutThePreamble 1 Index 1 out of bounds for length 1

ℹ️ These tests did not fail the build. Surefire retried them and they passed.
Retries are enabled project-wide by surefire.rerunFailingTestsCount in parent/pom.xml.


🔬 Scalpel shadow comparison — Scalpel: 10 of 726 tested, 22 compile-only — current: 10 all tested

Maveniverse Scalpel detected 10 affected modules (current approach: 10).

Skip-tests mode would test 10 modules (4 direct + 8 downstream), skip tests for 22 (generated code, meta-modules)

Modules Scalpel would test (10)
  • camel-cli-connector ← dsl/camel-cli-connector/src/main/docs/cli-connector.adoc, dsl/camel-cli-connector/src/main/java/org/apache/camel/cli/connector/FileCliConnectorTransport.java, dsl/camel-cli-connector/src/test/java/org/apache/camel/cli/connector/TraceFileSizeTest.java
  • camel-cli-debug ← downstream of org.apache.camel:camel-cli-connector
  • camel-jbang-mcp ← downstream of org.apache.camel:camel-catalog
  • camel-jbang-plugin-mcp ← downstream of org.apache.camel:camel-jbang-core
  • camel-jbang-plugin-tui ← dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DataRefreshService.java, dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/DataRefreshMembershipTest.java
  • camel-jbang-plugin-validate ← downstream of org.apache.camel:camel-jbang-core
  • camel-jbang-run ← downstream of org.apache.camel:camel-catalog
  • camel-launcher-container ← downstream of org.apache.camel:camel-launcher
  • camel-yaml-dsl-validator ← downstream of org.apache.camel:camel-catalog
  • camel-yaml-dsl-validator-maven-plugin ← downstream of org.apache.camel:camel-yaml-dsl-validator
Modules with tests skipped (22)
  • apache-camel
  • camel-catalog-console
  • camel-catalog-maven
  • camel-catalog-suggest
  • camel-componentdsl
  • camel-endpointdsl
  • camel-endpointdsl-support
  • camel-itest
  • camel-jbang-it
  • camel-jbang-main
  • camel-jbang-plugin-edit
  • camel-jbang-plugin-generate
  • camel-jbang-plugin-kubernetes
  • camel-jbang-plugin-test
  • camel-kamelet-main
  • camel-launcher
  • camel-report-maven-plugin
  • camel-route-parser
  • camel-yaml-dsl
  • camel-yaml-dsl-deserializers
  • camel-yaml-dsl-maven-plugin
  • coverage

ℹ️ Shadow mode — Scalpel observes but does not affect test execution. Learn more

⚠️ Some tests are disabled on GitHub Actions (@DisabledIfSystemProperty(named = "ci.env.name")) and require manual verification:

  • dsl/camel-jbang/camel-jbang-core: 2 test(s) disabled on GitHub Actions

💡 Manual integration tests recommended:

You modified dsl/camel-jbang/camel-jbang-core. The related integration tests in dsl/camel-jbang/camel-jbang-it are excluded from CI. Consider running them manually:

mvn verify -f dsl/camel-jbang/camel-jbang-it -Djbang-it-test
All tested modules (34 modules, 5m 40s total)

Total reactor time: 5m 40s

Module Duration Status
Camel :: JBang :: Plugin :: TUI 49.2s SUCCESS
Camel :: JBang :: MCP 47.3s SUCCESS
Camel :: Component DSL 42.2s SUCCESS
Camel :: YAML DSL :: Validator 26.4s SUCCESS
Camel :: Catalog :: Camel Catalog 25.0s SUCCESS
Camel :: JBang :: Run 24.7s SUCCESS
Camel :: JBang :: Plugin :: Validate 23.6s SUCCESS
Camel :: YAML DSL 19.5s SUCCESS
Camel :: JBang :: Plugin :: Kubernetes 14.4s SUCCESS
Camel :: JBang :: Plugin :: Testing 11.8s SUCCESS
Camel :: Kamelet Main 9.3s SUCCESS
Camel :: JBang :: Plugin :: Generate 7.2s SUCCESS
Camel :: Catalog :: Camel Report Maven Plugin 7.1s SUCCESS
Camel :: YAML DSL :: Deserializers 4.9s SUCCESS
Camel :: Catalog :: Camel Route Parser 3.8s SUCCESS
Camel :: YAML DSL :: Validator Maven Plugin 3.8s SUCCESS
Camel :: Catalog :: Suggest (deprecated) 3.5s SUCCESS
Camel :: Catalog :: Maven 3.0s SUCCESS
Camel :: YAML DSL :: Maven Plugins 2.2s SUCCESS
Camel :: Coverage 1.6s SUCCESS
Camel :: Assembly 1.5s SUCCESS
Camel :: JBang :: Plugin :: MCP 1.4s SUCCESS
Camel :: JBang :: Plugin :: Edit 1.1s SUCCESS
Camel :: JBang :: Integration tests 1.1s SUCCESS
Camel :: Launcher :: Container 1.0s SUCCESS
Camel :: DSL :: CLI Debug 1.0s SUCCESS
Camel :: JBang :: Main 1.0s SUCCESS
Camel :: Endpoint DSL :: Support 0.7s SUCCESS
Camel :: Catalog :: Console 0.4s SUCCESS
Camel :: DSL :: CLI Connector n/a
Camel :: Endpoint DSL n/a
Camel :: Integration Tests n/a
Camel :: JBang :: Core n/a
Camel :: Launcher n/a

Top 20 slowest modules:

  • Camel :: JBang :: Plugin :: TUI (49.2s)
  • Camel :: JBang :: MCP (47.3s)
  • Camel :: Component DSL (42.2s)
  • Camel :: YAML DSL :: Validator (26.4s)
  • Camel :: Catalog :: Camel Catalog (25.0s)
  • Camel :: JBang :: Run (24.7s)
  • Camel :: JBang :: Plugin :: Validate (23.6s)
  • Camel :: YAML DSL (19.5s)
  • Camel :: JBang :: Plugin :: Kubernetes (14.4s)
  • Camel :: JBang :: Plugin :: Testing (11.8s)
  • Camel :: Kamelet Main (9.3s)
  • Camel :: JBang :: Plugin :: Generate (7.2s)
  • Camel :: Catalog :: Camel Report Maven Plugin (7.1s)
  • Camel :: YAML DSL :: Deserializers (4.9s)
  • Camel :: Catalog :: Camel Route Parser (3.8s)
  • Camel :: YAML DSL :: Validator Maven Plugin (3.8s)
  • Camel :: Catalog :: Suggest (deprecated) (3.5s)
  • Camel :: Catalog :: Maven (3.0s)
  • Camel :: YAML DSL :: Maven Plugins (2.2s)
  • Camel :: Coverage (1.6s)

⚙️ View full build and test results

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working catalog docs dsl

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants