Skip to content

Commit 28874fe

Browse files
timsaucerclaude
andcommitted
docs: resolve mkdocs build warnings and tighten public API surface
Bring the strict mkdocs build to zero actionable warnings. Cross-reference fixes: - Replace ~20 unqualified `[X][X]` refs with fully-qualified targets (sort/sort_by, cube/rollup/grouping_sets, array_* aliases, register_*, Volatility, Serde, Producer, max_rows, metrics, etc.). - Fix typos: `Dataframe` -> `DataFrame`, `datafusion.Expr.to_bytes` -> `datafusion.expr.Expr.to_bytes`. - Drop links to non-Python symbols: `SessionState`, `CreateExternalTable`, `multiprocessing.Pool`, `cloudpickle`; point the `ObjectStore` reference at the module page; link `LogicalExtensionCodec` to `with_logical_extension_codec`. - Update relative link in `dataframe_formatter.md` for the subdirectory move. Public surface and reference page coverage: - Add `__all__` to `context`, `dataframe`, `dataframe_formatter`, `io`, `record_batch`, `user_defined`, `input/base`, `input/location` so the public surface is explicit. - Document the newly-declared public symbols on the corresponding reference pages (Compression, Volatility, *Exportable Protocols, ArrowStreamExportable, etc.). - Update `object_store.md` to render the PyO3 class aliases via explicit per-class directives (whole-module discovery skips re-assigned PyO3 bindings). - Allow `__next__`/`__anext__` through the formatter filter on `RecordBatchStream` so the iterator protocol is documented. Docstring fixes: - Convert overload-impl Args blocks (ScalarUDF.udf, AggregateUDF.udaf, WindowUDF.udwf) to free-form prose: their actual signatures are `*args, **kwargs`, which griffe was flagging. - Fix Args continuation indent in the formatter, the `idx::` typo, and a `with_extension` Args entry whose continuation lost its indent. - Re-wrap doctring lines that pushed past the 88-column limit after the cross-ref qualifications. mkdocstrings config: add `docstring_options: warn_unknown_params: false` so future overload-impl patterns don't trip the build. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 6c42adc commit 28874fe

23 files changed

Lines changed: 266 additions & 122 deletions

docs/source/reference/datafusion/context.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,11 @@
1111
::: datafusion.context.SQLOptions
1212

1313
::: datafusion.context.RuntimeEnvBuilder
14+
15+
::: datafusion.context.ArrowStreamExportable
16+
17+
::: datafusion.context.ArrowArrayExportable
18+
19+
::: datafusion.context.TableProviderExportable
20+
21+
::: datafusion.context.PhysicalOptimizerRuleExportable

docs/source/reference/datafusion/dataframe.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@
1616

1717
::: datafusion.dataframe.ExplainFormat
1818

19+
::: datafusion.dataframe.Compression
20+
1921
## DataFrame Formatter
2022

2123
See [DataFrame Formatter](dataframe_formatter.md) for the full formatter API

docs/source/reference/datafusion/dataframe_formatter.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
The `datafusion.dataframe_formatter` module controls how DataFrames render
44
in notebooks and HTML contexts. See the user-guide
5-
[Rendering](../user-guide/dataframe/rendering.md) page for worked examples.
5+
[Rendering](../../user-guide/dataframe/rendering.md) page for worked examples.
66

77
::: datafusion.dataframe_formatter.configure_formatter
88

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,15 @@
11
# Object Store
22

33
::: datafusion.object_store
4+
options:
5+
members: false
6+
7+
::: datafusion.object_store.AmazonS3
8+
9+
::: datafusion.object_store.GoogleCloud
10+
11+
::: datafusion.object_store.Http
12+
13+
::: datafusion.object_store.LocalFileSystem
14+
15+
::: datafusion.object_store.MicrosoftAzure

docs/source/reference/datafusion/record_batch.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,8 @@
33
::: datafusion.record_batch.RecordBatch
44

55
::: datafusion.record_batch.RecordBatchStream
6+
options:
7+
filters:
8+
- "!^_"
9+
- "^__next__$"
10+
- "^__anext__$"

docs/source/reference/datafusion/user_defined.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# User-Defined Functions
22

3+
::: datafusion.user_defined.Volatility
4+
35
::: datafusion.user_defined.ScalarUDF
46

57
::: datafusion.user_defined.AggregateUDF
@@ -19,3 +21,13 @@
1921
::: datafusion.user_defined.udwf
2022

2123
::: datafusion.user_defined.udtf
24+
25+
::: datafusion.user_defined.ScalarUDFExportable
26+
27+
::: datafusion.user_defined.AggregateUDFExportable
28+
29+
::: datafusion.user_defined.WindowUDFExportable
30+
31+
::: datafusion.user_defined.LogicalExtensionCodecExportable
32+
33+
::: datafusion.user_defined.PhysicalExtensionCodecExportable

docs/source/user-guide/common-operations/aggregations.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -453,8 +453,8 @@ The available aggregate functions are:
453453

454454
You can ship custom aggregations to the engine by subclassing
455455
[`Accumulator`][datafusion.user_defined.Accumulator] and registering it via
456-
[`udaf`][datafusion.user_defined.udaf]. See [`user_defined`][datafusion.user_defined] for
457-
the accumulator interface and worked examples.
456+
[`udaf`][datafusion.user_defined.udaf]. See [`user_defined`](../../reference/datafusion/user_defined.md)
457+
for the accumulator interface and worked examples.
458458

459459
<div class="admonition note">
460460
<p class="admonition-title">Note</p>

docs/source/user-guide/common-operations/windows.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -208,7 +208,7 @@ The possible window functions are:
208208

209209
You can ship custom window functions to the engine by subclassing
210210
[`WindowEvaluator`][datafusion.user_defined.WindowEvaluator] and registering it
211-
via [`udwf`][datafusion.user_defined.udwf]. See [`user_defined`](../../reference/user_defined.md)
211+
via [`udwf`][datafusion.user_defined.udwf]. See [`user_defined`](../../reference/datafusion/user_defined.md)
212212
for the evaluator interface and worked examples.
213213

214214
<div class="admonition note">

mkdocs.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,10 @@ plugins:
6868
- https://docs.pola.rs/api/python/stable/objects.inv
6969
options:
7070
docstring_style: google
71+
docstring_options:
72+
warn_unknown_params: false
73+
returns_named_value: false
74+
returns_multiple_items: false
7175
show_source: false
7276
members_order: source
7377
inherited_members: true

python/datafusion/catalog.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -239,7 +239,7 @@ class TableProviderFactory(ABC):
239239

240240
@abstractmethod
241241
def create(self, cmd: CreateExternalTable) -> Table:
242-
"""Create a table using the [`CreateExternalTable`][CreateExternalTable]."""
242+
"""Create a table using the `CreateExternalTable`."""
243243
...
244244

245245

0 commit comments

Comments
 (0)