Trace utilities no longer available to downstream packages after Maven migration

[alexeid]

Problem

Reported by Joelle Barido-Sottani:

I'm currently testing the new Maven build - seems the test classes are no longer exported? I was using some of the trace reading utilities (in the test.beast.beast2vs1.trace package) for my own tests and they can't be found anymore. Is there a way to put them back?

The old Ant build explicitly created a test.beast.jar containing all compiled test classes. When we migrated to Maven, this was not carried over — Maven only packages src/main/java classes by default.

Analysis

The classes in test.beast.beast2vs1.trace are not actually test classes. They are general-purpose trace-analysis tools that were misplaced in the test tree:

• LogAnalyser — a BEAST Runnable with @Description/@Citation and Input fields
• Expectation — a BEASTObject for comparing expected vs actual trace values
• LogFileTraces — log file parser
• TraceStatistics — computes mean, stdev, median, HPD, ESS, autocorrelation time, etc.
• NumberFormatter, TraceException, TraceType, TrimLineReader — supporting utilities
• DiscreteStatistics — identical duplicate of beast.base.util.DiscreteStatistics

Proposed solution

1. Move the 8 classes (excluding the duplicate DiscreteStatistics) from src/test/java/test/beast/beast2vs1/trace/ to src/main/java/beast/base/trace/ in beast-base.
2. Delete the test DiscreteStatistics and have TraceStatistics use the existing beast.base.util.DiscreteStatistics.
3. Export beast.base.trace in module-info.java.
4. Update internal test imports.

Future work

beastfx.app.tools.LogAnalyser duplicates much of this functionality (log parsing, ESS, HPD, etc.) but is coupled to beast-fx via a Swing file-chooser fallback — even though the class is otherwise entirely GUI-less. A follow-up refactor should extract the pure computation from the beast-fx LogAnalyser into beast.base.trace and have the beast-fx class delegate to it.

[alexeid]

Migration guide for downstream packages

The trace utilities have moved from the test tree to main sources. If your package was depending on these classes, update your imports:

// Old
import test.beast.beast2vs1.trace.LogAnalyser;
import test.beast.beast2vs1.trace.Expectation;
import test.beast.beast2vs1.trace.TraceStatistics;
// etc.

// New
import beast.base.trace.LogAnalyser;
import beast.base.trace.Expectation;
import beast.base.trace.TraceStatistics;
// etc.

The classes are now part of the beast-base main JAR (no longer require a separate test JAR). The DiscreteStatistics class that was in test.beast.beast2vs1.trace was an identical copy of beast.base.util.DiscreteStatistics and has been removed — use the latter directly.

@joellebs this should fix your issue once you update the imports.

[alexeid]

Refactoring plan: reconcile LogAnalyser implementations

Current state

There are two LogAnalyser classes with overlapping functionality:

	beast.base.trace.LogAnalyser	beastfx.app.tools.LogAnalyser
Extends	Runnable (BEAST framework)	nothing
File parsing	delegates to LogFileTraces	inline readLogFile()
Statistics	delegates to TraceStatistics	inline calcStats()
ESS/ACT	TraceStatistics (own autocorrelation code)	beast.base.spec.inference.util.ESS (static methods)
GUI deps	none	Utils.getLoadFile() in main() only
Used by	2 test frameworks	LogCombiner (extends it), LogComparator

Complication

LogCombiner extends beastfx.app.tools.LogAnalyser — it inherits the file reading and data storage. This can't be broken without reworking LogCombiner.

Proposed steps

Step 1 — Consolidate statistics computation

TraceStatistics and the inline calcStats() in beastfx LogAnalyser compute the same things (mean, stdev, median, HPD, ESS, ACT, geometric mean) with slightly different autocorrelation algorithms. Unify by having calcStats() delegate to TraceStatistics (or a shared static method). Reconcile the two ACT implementations — they use the same mathematical principle but different arithmetic; pick one and use it everywhere.

Step 2 — Extract core log reading into beast.base.trace

Move readLogFile() logic from beastfx LogAnalyser into LogFileTraces (or a new class in beast.base.trace). The beastfx LogAnalyser currently does its own tab-delimited parsing with type detection (REAL/INTEGER/BOOL/NOMINAL) — LogFileTraces does similar but simpler parsing. Enhance LogFileTraces to handle the type detection, then have beastfx LogAnalyser delegate to it.

Step 3 — Make beastfx LogAnalyser a thin wrapper

After steps 1-2, beastfx LogAnalyser becomes a thin shell providing:

• CLI main() with argument parsing
• GUI file-chooser fallback (the Utils.getLoadFile() call)
• Formatted console output (print(), printOneLine())
• The CoreRunnable for multi-threaded batch processing

All computation and data access delegate to beast.base.trace classes.

Step 4 — Rework LogCombiner inheritance

LogCombiner currently extends beastfx LogAnalyser to reuse file reading and data storage. After step 2, it should either:

• Extend the new beast.base.trace base class, or
• Switch to composition (hold a LogFileTraces instance) — preferred if the inheritance was only for code reuse

Step 5 — Update LogComparator

LogComparator only uses getLabels(), indexof(), getMean(), getStdError(), getLogFile(). These could be defined on an interface or on the beast.base.trace class directly.

Order of execution

Steps 1-2 can be done together. Step 3 follows naturally. Steps 4-5 are independent of each other but depend on 1-3.

Risk

The main risk is the ACT/ESS reconciliation (step 1) — the two algorithms may produce slightly different numerical results, which could affect existing test expectations. Compare outputs on a representative log file before switching.

[alexeid]

ACT algorithm analysis — which to keep

Both implementations use the same "fancy stopping criterion" (stop when the sum of two consecutive autocovariance values goes negative) and produce the same final quantities (ACT, ESS, std error of mean). The difference is how autocovariance is computed:

TraceStatistics — exact batch autocovariance (canonical)

mean = precomputed from full trace
for each lag:
    gammaStat[lag] = Σ (values[j] - mean)(values[j+lag] - mean) / (N - lag)

Standard textbook formula. Two nested loops, O(N × maxLag). Uses the exact mean and computes exact sample autocovariance at each lag. Also computes stdev of ACT.

ESS.ACT() — incremental/online approximation

for each sample i:
    sum += trace[i]
    mean = sum / (i+1)          ← running mean, recomputed each step
    for each lag:
        squareLaggedSums[lag] += trace[i-lag] * trace[i]
        autoCorrelation[lag] ≈ squareLaggedSums - (sum1+sum2)*mean + mean²*(N-lag)

Designed for the ESS.log() method where samples arrive one at a time during MCMC. The code comments acknowledge this is an approximation: "valid since mean*(samples-lag), sum1, and sum2 are approximately the same". The static ESS.ACT() method reuses this online logic even when the full array is available.

Recommendation

• Batch computation: use the TraceStatistics algorithm (exact). The static ESS.ACT() and ESS.stdErrorOfMean() methods should delegate to TraceStatistics rather than re-implementing an approximate version.
• Online/streaming computation (ESS during MCMC via ESS.log()): keep the incremental algorithm — it's correct for its use case where the full trace isn't available yet.
• Risk: the approximate and exact algorithms may produce slightly different numerical results, so existing test expectations should be checked after switching.