Skip to content

Conversation

@cj-zhukov
Copy link
Contributor

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@cj-zhukov
Copy link
Contributor Author

High Level Overview

This PR restructures and improves the organization of the README for datafusion-examples directory to make it significantly easier for users to discover, understand, and run examples. The main goals are consistency, clarity, and improved developer experience. The tables provide a quick and complete overview of the available examples.

Note: The ffi_xyz crate is not included, as it's not a runnable example. It also currently does not appear in the README. If needed, a follow-up PR can clarify its purpose and documentation.

@alamb
Copy link
Contributor

alamb commented Dec 10, 2025

For anyone else who is curious, you can see it rendered at this page: https://github.com/cj-zhukov/datafusion/tree/cj-zhukov/update-readme-occording-to-the-new-examples/datafusion-examples

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @cj-zhukov -- this look like a nice improvement to me

I wonder if there is any way to ensure this document stays in sync with the code. For example, if someone adds a new example, how do we ensure that an appropriate entry is added here 🤔

Maybe we could add a ci check similar to the config doc check

config-docs-check:
name: check configs.md and ***_functions.md is up-to-date
needs: linux-build-lib
runs-on: ubuntu-latest
container:
image: amd64/rust
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
submodules: true
fetch-depth: 1
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
rust-version: stable
- uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6.1.0
with:
node-version: "20"
- name: Check if configs.md has been modified
run: |
# If you encounter an error, run './dev/update_config_docs.sh' and commit
./dev/update_config_docs.sh
git diff --exit-code
- name: Check if any of the ***_functions.md has been modified
run: |
# If you encounter an error, run './dev/update_function_docs.sh' and commit
./dev/update_function_docs.sh
git diff --exit-code
🤔


| Group | Subcommand | Category | File Path | Description |
| ----------------- | ---------------- | -------------- | ------------------------------------------------ | ---------------------------------------------------------- |
| builtin_functions | date_time | Single Process | `examples/builtin_functions/date_time.rs` | Examples of date-time related functions and queries |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we make the file path a link so readers can easily navigate to the code with a single click?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point! Let's do it.

@cj-zhukov
Copy link
Contributor Author

@Jefffrey since you originally suggested using the tabular format for organizing the examples, you might be interested in taking a look at this PR.

- [`examples/flight/client.rs`](examples/flight/client.rs) and [`examples/flight/server.rs`](examples/flight/server.rs): Run DataFusion as a standalone process and execute SQL queries from a client using the Arrow Flight protocol.
## Builtin Functions Examples

| Group | Subcommand | Category | File Path | Description |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can probably remove the Group and Category columns as they seem constant per header; we can mention them upfront above the table & beneath the header, to reduce the table size

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point! I think reducing the table size is a good idea.

Copy link
Contributor

@Jefffrey Jefffrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be good to go once the CI checks pass

@alamb alamb added the documentation Improvements or additions to documentation label Dec 11, 2025
@alamb alamb added this pull request to the merge queue Dec 11, 2025
@alamb
Copy link
Contributor

alamb commented Dec 11, 2025

Thanks again @cj-zhukov and @Jefffrey

Merged via the queue into apache:main with commit c8add5c Dec 11, 2025
5 checks passed
@cj-zhukov
Copy link
Contributor Author

Thanks @cj-zhukov -- this look like a nice improvement to me

I wonder if there is any way to ensure this document stays in sync with the code. For example, if someone adds a new example, how do we ensure that an appropriate entry is added here 🤔

Maybe we could add a ci check similar to the config doc check

config-docs-check:
name: check configs.md and ***_functions.md is up-to-date
needs: linux-build-lib
runs-on: ubuntu-latest
container:
image: amd64/rust
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
submodules: true
fetch-depth: 1
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
rust-version: stable
- uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6.1.0
with:
node-version: "20"
- name: Check if configs.md has been modified
run: |
# If you encounter an error, run './dev/update_config_docs.sh' and commit
./dev/update_config_docs.sh
git diff --exit-code
- name: Check if any of the ***_functions.md has been modified
run: |
# If you encounter an error, run './dev/update_function_docs.sh' and commit
./dev/update_function_docs.sh
git diff --exit-code

🤔

This would be great if automated! I've opened a follow-up PR for that #19294

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants