Skip to content

feat(kafka): add opt-in workflow micro-batching - #721

Closed
xiami762 wants to merge 1 commit into
devfrom
feat/kafka-micro-batching
Closed

feat(kafka): add opt-in workflow micro-batching#721
xiami762 wants to merge 1 commit into
devfrom
feat/kafka-micro-batching

Conversation

@xiami762

Copy link
Copy Markdown
Contributor

Summary

  • add opt-in Kafka micro-batching with configurable max records and wait window
  • deliver decoded batches through the existing workflow input key without exposing Kafka metadata
  • use manual offset commits after successful workflow execution while preserving legacy single-record auto-commit defaults
  • persist batch settings across trigger compatibility paths and expose them in the workflow integration UI

Compatibility

  • batchMaxRecords defaults to 1, preserving the existing single-record behavior
  • workflows require no definition changes unless they opt in and consume list payloads

Verification

  • uv run ruff check on modified Python files
  • 67 related backend tests passed
  • npm run build passed

@xiami762 xiami762 closed this Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant