Vulkan API backend

[Appricorn4]

Is a Vulkan backend possible? I use Vulkan on stable-diffusion.cpp and it's significantly faster than CLBlast, CUDA, and ROCm in my testing. I would really like to see it here as Vulkan support is universal and easy, and would help especially with AMD and Intel GPUs (and possibly Qualcomm).

[alshdavid]

+1 this should be a priority as it unlocks the use of virtually every device. Consider;

• AMD's "AI Max" APUs
  • Can have 100gb+ video RAM
  • Currently no support for ROCm
• Qualcomm/Rockchip/etc chips
• AMD 9000 series cards
  • Currently no (practical) support for ROCm
• Potentially even MacOS devices via MoltenVK.

[chew1976]

Please make Vulkan possible. I bought a Strix Halo 128gb just to use Comfy UI, only to find out that I am dying to use Comfy UI on it.

[alshdavid]

CC pytorch/pytorch#160230

[kamathln]

Using Vulkan made installing llama a breeze on my Linux distro that has trouble with ROCm. Wish every AI tool has fist class support for Vulkan first and then targets specific tools.

[dc3000]

Reply +1