Add automatic native AI routing across CPU, GPU, RAM, and media workloads

[johnnycoderbot-cloud]

What this delivers

• zero-configuration model routing for Colibri, ONNX Runtime, llama.cpp, optional IREE, and optional MAX
• native Windows DirectML/DirectX 12 and Vulkan paths with no CUDA or WSL requirement
• first-use calibration, model+hardware cache, correctness checks, recent-failure demotion, and automatic fallback
• RAM-first sparse-MoE placement; storage is used only for initial load/cache misses when the model fits physical RAM
• full lifecycle placement for inference, evaluation, training, fine-tuning, ML, quantization, image generation, and video generation
• native PowerShell and portable Unix installers
• strict OLMoE conversion completion markers, reconstruction thresholds, and token-oracle quality gates
• upstream llama.cpp GGUF support with Vulkan and current MTP/draft features; the Fable CUDA/Linux fork remains compatibility- and benchmark-gated

RTX 3070 native Vulkan optimization

• correctness-gated PowerShell tuner for portable and packed-INT8 subgroup kernels
• measured WG256 winner: +31.6% weighted S1 expert-mix throughput and +52.1% on the S16 batch fixture
• automatic profile is restricted to the measured RTX 3070 device; other cards retain the portable shader until calibrated
• validation layers pass FP32, INT8, INT4, and INT2/FP4 storage paths; stress passes 32/32 correctness cases, 25 lifecycle cycles, concurrency, and memory stability
• experimental OLMoE Vulkan expert tier passes the 12/12 greedy-token oracle and 6,144/6,144 calls with zero failures
• end-to-end warm OLMoE was tied with the exact dual-Xeon CPU path, so GPU expert routing remains opt-in instead of claiming a false automatic speedup
• complete method and raw interpretation are documented in docs/experiments/rtx3070-vulkan-tuning-2026-07-17.md

Validated on the native Windows host

• Python: 90 tests pass in both base and AI environments
• C: all precision, cache, grammar, compatibility, AVX-512 VNNI INT8/INT4, and FP4 tests pass
• Vulkan RTX 3070: 32/32 correctness cases, lifecycle, resident tensors, cold/warm concurrency, and sustained memory checks pass
• native Vulkan stress: 17.306 GFLOP/s at S=1 and 63.117 GFLOP/s at S=16, with zero failures and -0.20 MiB sustained RSS drift
• ONNX: DirectML output matches CPU; calibration correctly selected CPU for a small graph
• real OLMoE INT8 on D: strict token oracle passes; both the bounded 64-triplet LRU and high-residency profile were exercised
• DirectML training/fine-tuning/quantization and real image/video convolution/frame workloads pass
• IPC: 3,000/3,000 requests succeeded at 1,294 requests/s, p95 67.4 ms

Honest boundaries

• no NPU or TPU is claimed on this host because no compatible local provider was detected
• MAX/Mojo is optional and not installed natively on Windows
• IREE is discovered only when its compiler and runner are installed
• image/video model-specific request schemas remain in their framework runner; this project supplies measured device/resource placement
• the RTX 3070 kernel gain is not represented as an end-to-end token-rate gain