Skip to content

Notebook on System Field ingestion#110

Draft
petermarshallio wants to merge 6 commits intomainfrom
202407-notebook-on-system-fields
Draft

Notebook on System Field ingestion#110
petermarshallio wants to merge 6 commits intomainfrom
202407-notebook-on-system-fields

Conversation

@petermarshallio
Copy link
Copy Markdown
Collaborator

@petermarshallio petermarshallio commented Jul 9, 2024

Notebook dedicated to incorporating system field data into tables.

Fix for #114

Brought forward content from the old batch ingestion intro.
Applied the template and added a transform to the field.
Reported an issue with the ingestion not completing.
Added some refs, added a bit to the query.
@petermarshallio
Copy link
Copy Markdown
Collaborator Author

Currently unable to complete this notebook due to apache/druid#16709 - this worked fine in Druid 29, hence passing tests in this environment.

@petermarshallio
Copy link
Copy Markdown
Collaborator Author

Adding a note that this notebook is waiting on a fix. It has not been resolved in Druid 31.

@petermarshallio petermarshallio changed the title Create 16-batch-systemfields.ipynb Notebook on System Field ingestion Dec 20, 2024
@petermarshallio
Copy link
Copy Markdown
Collaborator Author

Confirming that ingestion still fails in Druid 31 with:

java.lang.RuntimeException: java.lang.ClassCastException: class org.apache.druid.segment.transform.TransformedInputRow cannot be cast to class org.apache.druid.data.input.ListBasedInputRow (org.apache.druid.segment.transform.TransformedInputRow and org.apache.druid.data.input.ListBasedInputRow are in unnamed module of loader 'app')

Rename to avoid clash.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant