Part of #599.
Populate planning-time statistics for IcebergDataSource from the Iceberg table and selected snapshot.
Essentially, implementing this TODO comment in the code:
|
// TODO: Implement planning time statistics for this DataSource. |
|
// At this point, we have information about the iceberg::table::Table which we are about |
|
// to read, so maybe there's something we can get from there. |
|
Ok(Arc::new(Statistics::new_unknown(&self.schema))) |
The snapshot summary and file metadata may provide row-count and byte-size estimates without planning all scan tasks.
Part of #599.
Populate planning-time statistics for
IcebergDataSourcefrom the Iceberg table and selected snapshot.Essentially, implementing this TODO comment in the code:
datafusion-distributed/iceberg/src/data_source.rs
Lines 252 to 255 in 72203a0
The snapshot summary and file metadata may provide row-count and byte-size estimates without planning all scan tasks.