HF_HUB_DISABLE_SYMLINKS_WARNING and mklink /D Link Target

[bmerkle]

Describe the bug
When running Olive on windows I get the warning below because of missing symlinks in Windows.
I can deactivate the warning via HF_HUB_DISABLE_SYMLINKS_WARNING which works.

However it would be nice if we could create symlinks in windows and acutally this is possible via mklink
mklink /D Link Target creates a symlink under Windows.

question: why don't we use mkinlik under Windows ?

To Reproduce

C:\work\microsoft\Olive\venv\Lib\site-packages\huggingface_hub\file_download.py:143: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\merklbe1\.cache\huggingface\hub\models--Qwen--Qwen2.5-Coder-0.5B-Instruct. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.
To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development
  warnings.warn(message)

Expected behavior
we should use mklink and avoid the warning

Olive config
Add Olive configurations here.

Olive logs
Add logs here.

Other information

• OS: Windows 11

Additional context

[bmerkle]

it is actually an upstream problem in the huggingface_hub library https://github.com/huggingface/huggingface_hub
which we can not really fix in olive. (I will report upstream though)

but we have 3 options:

1. Enable Developer Mode on Windows (one-time setup) https://learn.microsoft.com/en-us/windows/advanced-settings/developer-mode
2. Run Python as administrator when using Olive
3. Keep using HF_HUB_DISABLE_SYMLINKS_WARNING=1 to suppress the warning and accept the extra disk usage

I would vote that we include these options into the docs with a hint for windows users (as the problem only occurs there)

regarding the options, the user should decide what is possible in his environment (e.g. company policies) and which option fits best for him (albeit option 3 does not fix the problem at all)