Skip to content

feat(wirings): capstone demo composing all 4 tier wirings in one pipeline (closes the #6→#9 stack)#10

Closed
estebanzimanyi wants to merge 1 commit into
MobilityDB:feat/flink-cross-stream-tier-wiringsfrom
estebanzimanyi:feat/flink-tier-wirings-capstone
Closed

feat(wirings): capstone demo composing all 4 tier wirings in one pipeline (closes the #6→#9 stack)#10
estebanzimanyi wants to merge 1 commit into
MobilityDB:feat/flink-cross-stream-tier-wiringsfrom
estebanzimanyi:feat/flink-tier-wirings-capstone

Conversation

@estebanzimanyi

Copy link
Copy Markdown
Member

Capstone follow-up on the PR #6#7#8#9 tier-wirings stack. Single Flink DataStream job that composes all four tier wirings in a coherent end-to-end pipeline — proves the wirings compose into a realistic pipeline shape, not just work in isolation.

Pipeline

Each stage uses one tier-wiring class:

Stream A (vehicles)                Stream B (region queries)
     │                                  │
① MeosStatelessFilter                   │   ← keep events in regions of interest
     │                                  │
② MeosBoundedStateMap                   │   ← per-vehicle running tbox union (byte[] state)
     │                                  │
③ MeosWindowedAggregate                 │   ← per-vehicle 30s tumbling tbox summary
     │                                  │
     └──────────────┐            ┌──────┘
                    ↓            ↓
④ MeosCrossStreamJoin                       ← interval-join on regionId, ±1m bound
                    ↓
              output

Answers: "for each region, which vehicles had an aggregate trajectory (running union) overlapping the region's query bbox during the latest 30-second window?"

Tier-per-stage breakdown

Stage Wiring Tier contract demonstrated
① Filter MeosStatelessFilter Per-event predicate, no state
② Running union MeosBoundedStateMap MEOS handle persisted across events as byte[] state (checkpoint-safe)
③ Window summary MeosWindowedAggregate Window-close-only aggregation; no handle persistence across windows
④ Region overlap MeosCrossStreamJoin Interval-join, same-key pairing, time-bounded match

File / compile

Stacking + closure

This PR closes the PR #6#9 wiring stack with the composite-demo capstone. The 5-PR stack collectively delivers:

PR What it adds
PR #6 MeosStatelessMap + MeosStatelessFilter (stateless tier — 804 methods)
PR #7 MeosBoundedStateMap (bounded-state tier — 797 methods)
PR #8 MeosWindowedAggregate (windowed tier — 161 methods)
PR #9 MeosCrossStreamJoin (cross-stream tier — 140 methods)
PR #10 (this) Capstone demo composing all 4 in one pipeline

Cumulative wirings-layer coverage = 2,097 of 2,097 emitted methods (100%) wirable through 5 generic classes; this PR proves they compose.

@estebanzimanyi estebanzimanyi force-pushed the feat/flink-tier-wirings-capstone branch from 6bddf45 to d322b8c Compare May 29, 2026 12:42
…peline

Adds MeosAllTiersCapstoneDemo — a single Flink DataStream job that
exercises all four tier-wiring classes from the PR MobilityDB#6MobilityDB#7MobilityDB#8MobilityDB#9 stack
in a coherent end-to-end pipeline.

Pipeline (each stage uses one tier-wiring class from the stack):

  ① MeosStatelessFilter    — drop events outside regions of interest
  ② MeosBoundedStateMap    — per-vehicle running tbox union (byte[] state)
  ③ MeosWindowedAggregate  — per-vehicle 30s tumbling tbox summary
  ④ MeosCrossStreamJoin    — interval-join vehicle aggregates against
                             region queries (±1m bound, regionId key)

The pipeline answers: 'for each region, which vehicles had an aggregate
trajectory (running union) overlapping the region's query bbox during
the latest 30-second window?'

Proves the wirings compose into a realistic pipeline shape (not just
work in isolation), each tier delivering its specific contract:
stateless filter is per-event, bounded-state persists handle state
across events as bytes, windowed aggregates window-close-only, cross-
stream interval-joins on shared key.

Stacks on PR MobilityDB#9; additive-only (1 new demo file). Locally compile-
verified: 146 .class files total (145 from PR MobilityDB#9 base + 1 new demo).

(cherry picked from commit d322b8c)
@estebanzimanyi estebanzimanyi force-pushed the feat/flink-tier-wirings-capstone branch from d322b8c to 173429b Compare May 31, 2026 07:49
@estebanzimanyi estebanzimanyi changed the base branch from main to feat/flink-cross-stream-tier-wirings June 11, 2026 16:46
@estebanzimanyi

Copy link
Copy Markdown
Member Author

Superseded by the Path-B consolidation: the former 18-deep stack is collapsed into two reviewable topical PRs on top of the merged scaffold — MEOS integration #30 → benchmark #31 — each one clean squashed commit with the generated-facade bulk, dead family-flag profiles, committed target/ artifacts, and invented synthetic corpus removed. Closing as folded into #30/#31.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant