feat - kafka support#243

Draft

msmygit wants to merge 49 commits into

feat/kafka_support

Commits on May 29, 2026

Initial plan for adding Kafka support
Madhavan
committed
Phase 1 - implemented
Madhavan
committed
Phase 2: Core Abstraction Layer
Madhavan
committed
Phase 3: Pulsar Implementation
Madhavan
committed
Phase 4: Kafka Implementation - week 1
Madhavan
committed
Phase 2: Core Abstraction Layer - Week 2-3 Deliverables. deleted ph 3 & 4
Madhavan
committed
Phase 3 - week1 - Pulsar Impl
Madhavan
committed
Phase 3 - weeks2&3 - Pulsar Impl
Madhavan
committed
Phase 4 - Kafka implementation initial
Madhavan
committed
Phase 4 - kafka impl basic docs updates
Madhavan
committed
Phase 5: Kafka Integration Tests & CI Workflows - Implementation
Madhavan
committed
Remove Made with Bob comment on files
Madhavan
committed
Phase 1: CI Stabilization - Disable Kafka tests and add resource limits
Madhavan
committed
ci failures
Madhavan
committed
backfill-ci failure fix attempt
Madhavan
committed
attempt to fix backfill-ci
Madhavan
committed
temp comment jdk17 from matrix
Madhavan
committed
attempt to remove kafka premature
Madhavan
committed
backfill cli and ci fix attempt
Madhavan
committed
bob created mess to fix backfill-ci job attempt 5
Madhavan
committed
phase 3 impl + refactor
Madhavan
committed
interim
Madhavan
committed
interim fixes;slf4j reverts
Madhavan
committed
interim phase3 fix attempts
Madhavan
committed
interim fixes backfill-ci failures
Madhavan
committed
interim backfill CI fixes
Madhavan
committed
s
Madhavan
committed
s
Madhavan
committed
interim backfill-ci fixes
Madhavan
committed
interim ci fixes
Madhavan
committed
Fix ClassCastException by reverting NativeSchemaWrapper and CassandraSource to original implementation
Madhavan
committed
Update BOB_CONTEXT_SUMMARY with ClassCastException fix details
Madhavan
committed
backfill-ci fail fix attempt
Madhavan
committed
Fix connector test failures: Revert to direct Pulsar API usage
Madhavan
committed
ci failures fixes
Madhavan
committed
CI Failure fix attempt
Madhavan
committed
fix(kafka_support): correct SSL keystore/truststore mapping and restore infinite producer timeout
Madhavan
committed

Commits on May 30, 2026

feat(kafka): complete & validate agent-side dual-provider (Pulsar+Kafka) CDC

Madhavan
and
claude
committed
feat(kafka): add Kafka Connect source connector (events -> Cassandra -> data)

Madhavan
and
claude
committed
fix(backfill): repair messaging-abstraction backfill (SPI discovery + sender construction)

Madhavan
and
claude
committed
fix(connector): make assertMapsEqual tolerant of shaded/non-shaded Avro Utf8

Madhavan
and
claude
committed
fix(backfill): forward Docker API version to the e2eTest task and backfill-ci

Madhavan
and
claude
committed
fix(backfill): discover dsbulk codec providers in the NAR CLI-extension path

Madhavan
and
claude
committed
ci: expand Pulsar/Kafka test matrices and restore 360m timeout

Madhavan
and
claude
committed
fix(agent): reject invalid messagingProvider and validate provider config

Madhavan
and
claude
committed
docs(kafka): publish Kafka/Confluent user docs and regenerate agent params

Madhavan
and
claude
committed
feat(backfill): support Kafka as a backfill destination + e2e + CI

Madhavan
and
claude
committed
docs(backfill): document Kafka as a backfill destination

Madhavan
and
claude
committed
fix(backfill-ci): pass cassandraFamily to the Kafka e2e so the right cassandra.yaml is mounted

Madhavan
and
claude
committed