Causeway 4068 v4

antora/components/userguide/modules/ROOT/pages/observability.adoc

@@ -29,53 +29,13 @@ An active observation profile does not require an agent or exporter to start suc
 == Java-agent-owned tracing
 
 An alternative configuration lets the OpenTelemetry Java agent own the SDK, automatic HTTP/JDBC instrumentation and export.
-Provide a registry whose Micrometer tracing handler bridges to the agent's global OpenTelemetry context.
-Add the BOM-managed `io.micrometer:micrometer-tracing-bridge-otel` dependency for this application configuration.
+Add the BOM-managed `io.micrometer:micrometer-tracing-bridge-otel` dependency and activate `observation,agent`.
+Causeway supplies the missing tracer and registry beans, bridges observations to the agent's global context, and automatically excludes competing Boot SDK/tracing and WebMVC observation configuration.
+No application Java bridge or manual exclusions are needed.
 
-[source,java]
-----
-@Configuration(proxyBeanMethods = false)
-@Profile("agent")
-class AgentObservationConfiguration {
-    @Bean
-    ObservationRegistry agentObservationRegistry() {
-        var currentContext = new OtelCurrentTraceContext();
-        var tracer = new OtelTracer(
-                GlobalOpenTelemetry.getTracer("org.apache.causeway"),
-                currentContext,
-                event -> {},
-                new OtelBaggageManager(currentContext, List.of(), List.of()));
-        var registry = ObservationRegistry.create();
-        registry.observationConfig()
-                .observationHandler(new DefaultTracingObservationHandler(tracer));
-        return registry;
-    }
-}
-----
-
-The types are from `io.micrometer.observation`, `io.micrometer.tracing.handler`, `io.micrometer.tracing.otel.bridge`, `io.opentelemetry.api`, `java.util`, and Spring's configuration annotations.
-Import this configuration into the application.
-For the validated Boot 4.2.0-M1 configuration, exclude the competing SDK/tracing auto-configurations:
-
-[source,yaml]
-----
-spring:
-  profiles:
-    active: observation,agent
-  autoconfigure:
-    exclude:
-      - org.springframework.boot.opentelemetry.autoconfigure.OpenTelemetrySdkAutoConfiguration
-      - org.springframework.boot.micrometer.tracing.opentelemetry.autoconfigure.OpenTelemetryTracingAutoConfiguration
-      - org.springframework.boot.micrometer.tracing.opentelemetry.autoconfigure.otlp.OtlpTracingAutoConfiguration
-      - org.springframework.boot.micrometer.tracing.autoconfigure.MicrometerTracingAutoConfiguration
-management:
-  tracing:
-    export:
-      enabled: false
-causeway:
-  observation:
-    duration-filtering-enabled: false
-----
+A supplied registry or tracer takes precedence over the corresponding default.
+A custom registry remains responsible for its handlers; those handlers and Causeway's entry-span classifier must share a single or primary tracer.
+Causeway does not construct an SDK or exporter, and the profiles do not attach an agent: attachment is still required for agent export.
 
 Launch the JVM with `-javaagent:/path/to/opentelemetry-javaagent.jar` and configure the agent's service name, sampler and OTLP endpoint through its normal settings.
 For example, a local tracing-only run can use:
@@ -88,41 +48,35 @@ export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
 export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4318/v1/traces
 export OTEL_METRICS_EXPORTER=none
 export OTEL_LOGS_EXPORTER=none
-java -javaagent:/path/to/opentelemetry-javaagent.jar -jar application.jar
+java -javaagent:/path/to/opentelemetry-javaagent.jar -jar application.jar --spring.profiles.active=observation,agent
 ----
 
+The registry and entry-span classifier share the exposed Micrometer tracer.
+The WebMVC exclusion avoids a second Boot HTTP observation inside the agent's entry span; Boot HTTP request observation meters are consequently absent, while other Micrometer metrics remain available.
+Causeway disables Boot environment-variable mapping for the `agent` profile, preventing the agent's `OTEL_METRICS_EXPORTER=none` from disabling a separately configured Micrometer metrics exporter.
 Do not also construct an independent SDK/export pipeline in this configuration.
 The agent configuration is an explicit application choice; it does not replace the Boot-managed default.
 With the agent profile but without the `observation` profile, automatic agent spans remain active and Causeway observations remain inactive.
 
-== Duration filtering and failures
+== Execution classification and interaction correlation
 
-Duration filtering is disabled by default, so short successful JPA parents remain available when their children are exported.
-This can increase span volume compared with the previous default.
-It does not override sampling, exporter failures or application-defined filtering.
+With the `observation` profile active, two attributes help you find relevant traces:
 
-To opt into the lossy Boot-managed duration filter:
+* `causeway.execution.mode` identifies the entry span as `foreground` for HTTP requests (including static resources), or `background` for `RunBackgroundCommandsJob` executions.
+Background classification requires an existing job span, supplied by instrumentation such as the Java agent's Quartz support.
+* `causeway.interaction.id` appears on *Causeway Root Interaction* and contains the interaction's UUID.
+It is distinct from the trace ID and retains the command's identifier during replay, allowing you to find related executions.
 
-[source,yaml]
-----
-causeway:
-  observation:
-    duration-filtering-enabled: true
-    jpa-duration-threshold: 2ms
-----
+User identity attributes are included automatically alongside interaction correlation; they are not added as metric labels.
+In Jaeger, select *All Span Names* and search separately by execution mode or interaction ID: they appear on different spans.
+Searches can only find traces that were sampled, exported and retained by the backend.
 
-The threshold defaults to `2ms` and accepts nonnegative durations; `0ms` suppresses no spans by duration.
-Malformed or negative thresholds fail startup, including when filtering is disabled.
-Successful JPA observations strictly below the configured threshold are marked for discard; observations at or above it and failed observations remain eligible for export.
-The Boot-managed export predicate honors that marker.
+== JPA span retention
 
-Filtering can leave children whose parent span was suppressed: discarding a JPA parent cannot retract an independently exported JDBC child.
-This option does not implement tail sampling or a per-request span budget.
-
-The agent-owned exporter does not consume Spring's span-export predicates.
-Keep `duration-filtering-enabled=false` in agent mode; the tested behavior retains those spans rather than claiming they were suppressed.
-The setting controls Causeway's JPA duration policy, not agent sampling or application-created observations.
-An explicitly discarded observation is still subject to the exporter's filtering capabilities.
+Causeway retains JPA observations regardless of duration in both Boot-managed and agent-managed tracing.
+The former `causeway.observation.jpa-duration-threshold` property has been removed; remove it and its environment override from existing configurations.
+Filtering a JPA parent independently could leave its JDBC children without an exported parent.
+Normal sampling, exporter failures and application-defined policies still determine which spans reach the backend.
 
 == Observation names and metadata migration (CAUSEWAY-4096)
 
@@ -153,21 +107,9 @@ Other existing fixed operation names are unchanged.
 Missing static member identifiers are omitted; runtime object data is never a fallback.
 Publishing counts are emitted as `causeway.execution.subscriber-count` high-cardinality data rather than a meter dimension or part of a name.
 
-Causeway no longer emits username and multitenancy token by default.
-Each can be enabled independently:
-
-[source,yaml]
-----
-causeway:
-  observation:
-    include-user-name: true
-    include-multi-tenancy-token: true
-----
-
-These options retain the existing `causeway.user.name` and `causeway.user.multiTenancyToken` attribute keys for nonempty values.
-High-cardinality attributes are still exported; the classification is not a redaction mechanism.
-Impersonation status and locale metadata remain available.
-These policies govern Causeway's instrumentation, not metadata emitted by application observations, agents, JDBC instrumentation or collectors.
+Causeway always includes nonempty `causeway.user.name` and `causeway.user.multiTenancyToken` on interaction spans.
+They describe the interaction's authenticated user and tenancy, and are high-cardinality span attributes rather than metric labels.
+Empty values are omitted; impersonation status and locale metadata remain available.
 
 == Validated baseline and regression fixture
 
@@ -176,8 +118,9 @@ These are the tested versions, not a requirement to override the application's C
 
 The child-process fixture uses production interaction and action services and the JPA facet with mocked non-telemetry collaborators, a local JDK HTTP server, H2, and a local OTLP receiver.
 Its JPA backend is controlled; agent mode instruments real H2 JDBC automatically, while Boot mode explicitly wraps the SQL call in a fixture observation.
-It proves agent HTTP/framework/JPA/JDBC ancestry, one framework span per invocation, a failure followed by a successful request on the same worker, Boot-managed export, duration filtering, inactive-profile behavior and operation without an agent/exporter.
-It does not prove every servlet container or viewer integration.
+It proves agent HTTP/framework/JPA/JDBC ancestry, one framework span per invocation, a failure followed by a successful request on the same worker, Boot-managed export, duration-independent JPA retention, inactive-profile behavior and operation without an agent/exporter.
+The additional entry-point fixture uses real Tomcat servlet and Quartz instrumentation with controlled business collaborators, verifying fixed-key classification and effective replay UUIDs in both trace ownership modes.
+It does not prove every servlet container or viewer integration, nor a full persisted-command replay.
 
 From the repository root, after building the required main artifacts:
 

core/config/pom.xml

@@ -97,6 +97,13 @@
             <artifactId>micrometer-tracing</artifactId>
         </dependency>
 
+        <!-- Agent bridge is opt-in for consuming applications. -->
+        <dependency>
+            <groupId>io.micrometer</groupId>
+            <artifactId>micrometer-tracing-bridge-otel</artifactId>
+            <optional>true</optional>
+        </dependency>
+
         <!--
         as per https://github.com/spring-projects/spring-boot/issues/30986, must be
         before the spring-boot-configuration-processor

core/config/src/main/java/module-info.java

@@ -35,6 +35,8 @@
     exports org.apache.causeway.core.config.viewer.web;
 
     requires static lombok;
+    requires static micrometer.tracing.bridge.otel;
+    requires static io.opentelemetry.api;
 
     requires transitive org.apache.causeway.applib;
     requires transitive org.apache.causeway.commons;

core/config/src/main/java/org/apache/causeway/core/config/observation/CausewayAgentEnvironmentPostProcessor.java

@@ -0,0 +1,75 @@
+/*
+ *  Licensed to the Apache Software Foundation (ASF) under one
+ *  or more contributor license agreements.  See the NOTICE file
+ *  distributed with this work for additional information
+ *  regarding copyright ownership.  The ASF licenses this file
+ *  to you under the Apache License, Version 2.0 (the
+ *  "License"); you may not use this file except in compliance
+ *  with the License.  You may obtain a copy of the License at
+ *
+ *        http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing,
+ *  software distributed under the License is distributed on an
+ *  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ *  KIND, either express or implied.  See the License for the
+ *  specific language governing permissions and limitations
+ *  under the License.
+ */
+package org.apache.causeway.core.config.observation;
+
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Map;
+
+import org.springframework.boot.EnvironmentPostProcessor;
+import org.springframework.boot.SpringApplication;
+import org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor;
+import org.springframework.boot.context.properties.bind.Binder;
+import org.springframework.core.Ordered;
+import org.springframework.core.env.ConfigurableEnvironment;
+import org.springframework.core.env.MapPropertySource;
+import org.springframework.core.env.Profiles;
+import org.springframework.util.ClassUtils;
+
+/** Selects agent trace ownership before Boot evaluates its auto-configurations. */
+public class CausewayAgentEnvironmentPostProcessor implements EnvironmentPostProcessor, Ordered {
+
+    private static final List<String> TRACE_AUTO_CONFIGURATIONS = List.of(
+            "org.springframework.boot.opentelemetry.autoconfigure.OpenTelemetrySdkAutoConfiguration",
+            "org.springframework.boot.micrometer.tracing.opentelemetry.autoconfigure.OpenTelemetryTracingAutoConfiguration",
+            "org.springframework.boot.micrometer.tracing.opentelemetry.autoconfigure.otlp.OtlpTracingAutoConfiguration",
+            "org.springframework.boot.micrometer.tracing.autoconfigure.MicrometerTracingAutoConfiguration",
+            "org.springframework.boot.webmvc.autoconfigure.WebMvcObservationAutoConfiguration");
+
+    @Override
+    public int getOrder() {
+        // Active profiles and application YAML must already have been loaded.
+        return ConfigDataEnvironmentPostProcessor.ORDER + 1;
+    }
+
+    @Override
+    public void postProcessEnvironment(final ConfigurableEnvironment environment,
+            final SpringApplication application) {
+        if (!environment.acceptsProfiles(Profiles.of("agent"))) {
+            return;
+        }
+        if (environment.acceptsProfiles(Profiles.of("observation"))
+                && !ClassUtils.isPresent("io.micrometer.tracing.otel.bridge.OtelTracer", application.getClassLoader())) {
+            throw new IllegalStateException("The observation,agent profiles require "
+                    + "io.micrometer:micrometer-tracing-bridge-otel. Add the BOM-managed dependency "
+                    + "and attach the OpenTelemetry Java agent to the application JVM.");
+        }
+        var exclusions = new LinkedHashSet<>(Binder.get(environment)
+                .bind("spring.autoconfigure.exclude", String[].class)
+                .map(List::of).orElseGet(List::of));
+        exclusions.addAll(TRACE_AUTO_CONFIGURATIONS);
+        // Ownership is a profile contract, so these values override command-line
+        // defaults too. Existing unrelated exclusions are retained above.
+        // Avoid mapping OTEL_METRICS_EXPORTER=none onto Micrometer metrics.
+        environment.getPropertySources().addFirst(new MapPropertySource("causewayAgentTracing", Map.of(
+                "spring.autoconfigure.exclude", String.join(",", exclusions),
+                "management.opentelemetry.map-environment-variables", "false",
+                "management.tracing.export.enabled", "false")));
+    }
+}

core/config/src/main/java/org/apache/causeway/core/config/observation/CausewayAgentObservationAutoConfiguration.java

@@ -0,0 +1,63 @@
+/*
+ *  Licensed to the Apache Software Foundation (ASF) under one
+ *  or more contributor license agreements.  See the NOTICE file
+ *  distributed with this work for additional information
+ *  regarding copyright ownership.  The ASF licenses this file
+ *  to you under the Apache License, Version 2.0 (the
+ *  "License"); you may not use this file except in compliance
+ *  with the License.  You may obtain a copy of the License at
+ *
+ *        http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing,
+ *  software distributed under the License is distributed on an
+ *  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ *  KIND, either express or implied.  See the License for the
+ *  specific language governing permissions and limitations
+ *  under the License.
+ */
+package org.apache.causeway.core.config.observation;
+
+import java.util.List;
+
+import org.springframework.boot.autoconfigure.AutoConfiguration;
+import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
+import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
+import org.springframework.context.annotation.Bean;
+import org.springframework.context.annotation.Profile;
+
+import io.micrometer.observation.ObservationRegistry;
+import io.micrometer.tracing.Tracer;
+import io.micrometer.tracing.handler.DefaultTracingObservationHandler;
+import io.micrometer.tracing.otel.bridge.OtelBaggageManager;
+import io.micrometer.tracing.otel.bridge.OtelCurrentTraceContext;
+import io.micrometer.tracing.otel.bridge.OtelTracer;
+import io.opentelemetry.api.GlobalOpenTelemetry;
+
+/**
+ * Joins Causeway observations to the attached Java agent's global context.
+ * This configuration creates neither an SDK nor an exporter. Application bean
+ * definitions take precedence; a custom registry owns its own handlers.
+ */
+@AutoConfiguration(beforeName = "org.springframework.boot.micrometer.observation.autoconfigure.ObservationAutoConfiguration")
+@Profile("observation & agent")
+@ConditionalOnClass(name = {"io.micrometer.tracing.otel.bridge.OtelTracer",
+        "io.opentelemetry.api.GlobalOpenTelemetry"})
+public class CausewayAgentObservationAutoConfiguration {
+
+    @Bean
+    @ConditionalOnMissingBean(Tracer.class)
+    public Tracer causewayAgentTracer() {
+        var currentContext = new OtelCurrentTraceContext();
+        return new OtelTracer(GlobalOpenTelemetry.getTracer("org.apache.causeway"), currentContext,
+                event -> {}, new OtelBaggageManager(currentContext, List.of(), List.of()));
+    }
+
+    @Bean
+    @ConditionalOnMissingBean(ObservationRegistry.class)
+    public ObservationRegistry causewayAgentObservationRegistry(final Tracer tracer) {
+        var registry = ObservationRegistry.create();
+        registry.observationConfig().observationHandler(new DefaultTracingObservationHandler(tracer));
+        return registry;
+    }
+}