Skip to content

GC download metadata locally should consider spark configuration #9738

@nopcoder

Description

@nopcoder

Currently, GC downloads metadata into the temporary directory. The expected behavior is to store the metadata in spark.local.dir instead, leveraging this directory as the designated Spark local storage, which may differ from the default temporary directory mainly in size.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions