Fix/kafka connector build#696
Open
zhanglistar wants to merge 106 commits into
Open
Conversation
- 在StatefulSerialTask中添加notifyWatermark(long watermark)方法 - 在JNI层添加notifyWatermark和notifyIndexedWatermark两个接口 - 将原有的notifyWatermark(带index)重命名为notifyIndexedWatermark - 在SerialTask中添加不带index的notifyWatermark公共方法 - 最终调用velox的相应不带index的接口
feat: 添加不带index的notifyWatermark接口
Bump velox
[FLINK]Support rocksdb state
[FLINK]Fix state initialize
[FLINK]Support processing time window for nexmark q12
Revert "[FLINK]Support processing time window for nexmark q12"
Fix aarch64 build: add missing folly assembly sources
[FLINK] Support processing time window
Contributor
|
@zhanglistar is this submitted to the wrong repo? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix Kafka connector build in the Velox monolithic build.
The previous build failed after enabling the Kafka connector for two reasons:
Velox links against
CppKafka::cppkafka, but the bundled cppkafka build only defines thecppkafkatarget. This caused CMake target resolution to fail.
cppkafka exposes
include/cppkafkaas a public include directory. In the monolithic Velox build thisinclude path leaked into unrelated targets, so Folly's
#include <event.h>resolved tocppkafka/ event.hinstead of the system libevent header. That broke Folly/libevent compilation with missingevent,event_base, andlibevent_fd_ttypes.This PR adds a local cppkafka dependency resolver override that:
CppKafka::cppkafkaalias;include/cppkafka, avoiding theevent.hheader collision.
It also explicitly enables the Kafka connector and disables cppkafka examples/tests for the dependency
build.
Verified with:
JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 VELOX4J_BUILD_JOBS=16 mvn clean install -DskipTests
-Dgpg.skip -Dspotless.skip=true