Skip to content

Conversation

@2010YOUY01
Copy link
Contributor

Which issue does this PR close?

  • Closes #.

Rationale for this change

A dependency graph for workspace member crates are often needed when doing refactors, I want it to be included in the doc, and have a script to update it automatically.

Here is the preview:
image

What changes are included in this PR?

  • adds a script to generate the dependency graph deps.svg, and verify if the existing one is up to date.
  • adds a documentation page in Contributor Guide to show this graph
  • adds a CI job to check if the generated dependency graph is up to date with the code.

Are these changes tested?

I tested the dependency graph display locally, see above.
Is it possible to see the preview from this PR's change online?

I also included a dummy crate in the initial commit, to test if the CI can catch it and throw understandable error message.

Are there any user-facing changes?

No

@github-actions github-actions bot added documentation Improvements or additions to documentation development-process Related to development process of DataFusion labels Dec 11, 2025
## Dependency Graph for Workspace Crates

<!--
Below is an embedded .svg file, with interactive functionalities like drag/zoom-in/etc.
Copy link
Contributor Author

@2010YOUY01 2010YOUY01 Dec 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI disclosure:
This snippet is to embed the deps.svg inside the markdown, and be able to drag/zoom. The implementation is AI generated, I have skimmed through it and it looks good to me, though I don't have enough front-end knowledge to inspect it in details.

# under the License.

[package]
name = "datafusion-ci-dummy"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For CI testing:

  1. Generated the dep graph
  2. Added this new dummy workspace member crate
    Now the dep graph is obsolete, the CI should fail

If failed as expected in https://github.com/apache/datafusion/actions/runs/20129195011/job/57766134500?pr=19280

with:
submodules: true
fetch-depth: 1
- name: Setup Rust toolchain
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should just run this script as part of some other job and save having to check out the code again...

Copy link
Contributor Author

@2010YOUY01 2010YOUY01 Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, it's better to let it generate and deploy through CI automatically. Done in 307119c

Perhaps we should also do so for other auto-generated document sections, like configuration and function docs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-process Related to development process of DataFusion documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants