feat(berlinmod): streaming throughput benchmark on the canonical BerlinMOD corpus (stacks on #14)#15
Open
estebanzimanyi wants to merge 4 commits into
Conversation
This was referenced Jun 11, 2026
Closed
Closed
estebanzimanyi
force-pushed
the
consolidate/kafka-meos-integration
branch
from
c6f2016 to
6305403
Compare
estebanzimanyi
force-pushed
the
consolidate/kafka-benchmark
branch
from
c32df1c to
09bba60
Compare
… surface Rebuild the bundled JMEOS jar from the regenerated JMEOS surface: the hand-rolled functions.functions facade is gone, GeneratedFunctions carries the sret-bound by-value struct returns (the *Split family) and the collation-init fix, the org.mobilitydb.meos MeosOps* facade is regenerated against it, and the MeosSetSetJoin helper rides the set-set spatial-join surface. Verified: kafka-streams-app compiles and its test suite is green (11 tests, 0 failures) against the refreshed jar.
…inMOD corpus Adds the BerlinMOD streaming benchmark harness: BerlinMODBenchmark (TopologyTestDriver throughput per Q-form cell) and EmbeddedBrokerBenchmark (steady-state throughput against an in-process EmbeddedKafkaCluster), plus docs/benchmark.md. Both drivers run on the canonical BerlinMOD corpus only (--csv berlinmod_instants.csv); the invented synthetic fallback is gone. Both live in src/test since they use the test-scoped TopologyTestDriver / EmbeddedKafkaCluster.
estebanzimanyi
force-pushed
the
consolidate/kafka-benchmark
branch
from
09bba60 to
70cacea
Compare
Rebuild the union JMEOS.jar against pin 14l: functions.GeneratedFunctions from JMEOS-14l (4466 methods, 2-arg count accessors) + the org.mobilitydb.meos.MeosOps* streaming facades regenerated from the 14l catalog/signatures. Refresh the bundled libmeos.so to 14l. The jar is now PURE bindings (JMEOS + generated facades). MeosSetSetJoin — the NxN cross-stream set-set join operator (edwithin/adisjoint/tdwithin_tgeoarr_tgeoarr + spanset_as_hexwkb) — was previously bundled as a source-less .class; it is application logic the facade codegen deliberately leaves to the caller, so it now lives as tracked source under org/mobilitydb/meos/ and is compiled from source against the jar.
estebanzimanyi
force-pushed
the
consolidate/kafka-benchmark
branch
from
dd55558 to
dd85dab
Compare
Rebuild the union jar via streaming-codegen/build-union-jar.sh against the JMEOS regen off pin 15c (4466->4469 methods: + the eintersects_tpcpoint_geo / nad_tpcpoint_geo pointcloud spatial rels) + the 58 generated MeosOps* facades. Refresh bundled libmeos to 15c.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Path-B consolidation — PR 3 of 3 (scaffold #1 · MEOS integration #14 · benchmark).
Adds the BerlinMOD streaming benchmark harness on top of #14:
BerlinMODBenchmark(per-cell TopologyTestDriver throughput) andEmbeddedBrokerBenchmark(steady-state throughput against an in-processEmbeddedKafkaCluster), plusdocs/benchmark.md.Canonical / regularity
--csv berlinmod_instants.csv); the invented synthetic fallback is removed.src/test(they use the test-scopedTopologyTestDriver/EmbeddedKafkaCluster) — same placement fix as the scaffold's test.Verification
mvn test→ 11 tests, 0 failures (the benchmark drivers aremainentry points, compiled but not run in CI).