Part of #3343.
What's the feature are you trying to implement?
FileScanTaskDeleteFile::file_size_in_bytes is required to open a Parquet delete file, because the reader uses it to locate the Parquet footer. Tasks planned by iceberg-rust from manifest entries always carry it, but execution engines that build FileScanTasks themselves (for example from tasks planned on the JVM) do not always have it. Today such an engine must stat every delete file while building tasks — one metadata request per delete file, made before it is known whether the data file will be read at all — and passing 0 instead makes the delete file fail to open.
Proposal: treat file_size_in_bytes == 0 as "size unavailable" for externally constructed tasks, and let the Parquet delete-file loader resolve the size with one metadata request when it first loads the file.
Expected behavior:
- Manifest-planned tasks continue using the recorded size; known sizes require no additional metadata request.
- Unknown sizes are resolved lazily from storage when the delete file is first loaded.
CachingDeleteFileLoader already loads each delete file once per reader, so the lookup happens once per delete file even when many data files reference it.
- A failed metadata request fails the read with delete-file context while preserving the storage error's kind and retryability.
- A size below the 8-byte Parquet footer minimum, whether resolved or supplied, fails explicitly rather than being read as "no deletes".
- Position, equality and encrypted Parquet delete files continue to work; deletion vectors do not use this size and are unaffected.
This is independently useful for execution engines constructing Iceberg scan tasks and is one of the reader foundations required by #3343.
Willingness to contribute
I can contribute to this feature independently; an implementation with tests is ready.
Part of #3343.
What's the feature are you trying to implement?
FileScanTaskDeleteFile::file_size_in_bytesis required to open a Parquet delete file, because the reader uses it to locate the Parquet footer. Tasks planned by iceberg-rust from manifest entries always carry it, but execution engines that buildFileScanTasks themselves (for example from tasks planned on the JVM) do not always have it. Today such an engine must stat every delete file while building tasks — one metadata request per delete file, made before it is known whether the data file will be read at all — and passing0instead makes the delete file fail to open.Proposal: treat
file_size_in_bytes == 0as "size unavailable" for externally constructed tasks, and let the Parquet delete-file loader resolve the size with one metadata request when it first loads the file.Expected behavior:
CachingDeleteFileLoaderalready loads each delete file once per reader, so the lookup happens once per delete file even when many data files reference it.This is independently useful for execution engines constructing Iceberg scan tasks and is one of the reader foundations required by #3343.
Willingness to contribute
I can contribute to this feature independently; an implementation with tests is ready.