Skip to content

AI-31: Model Weight Quantization (FP16 & INT8) for Low-Latency GPU Inference #1069

Description

@chinweobtagaz

Context & Description

Running neural network models in full precision (FP32) limits batch throughput and increases VRAM usage. We need automated model quantization pipelines converting PyTorch model weights to FP16 and TensorRT / ONNX INT8 for high-throughput GPU inference.
Location: agent-engines/gpu_worker/models.py

Requirements

  1. Implement quantization script for evaluation and policy heads: FP32 -> FP16 and INT8 using TensorRT / ONNX Runtime quantization tools.
  2. Measure evaluation error: ensure < 0.5% degradation in Top-1 move accuracy compared to baseline FP32 model.
  3. Achieve at least 2.5x speedup in batch evaluation throughput on NVIDIA GPUs.
  4. Package quantized models into standard distribution artifacts.

Acceptance Criteria

  • Quantized INT8/FP16 models pass validation accuracy thresholds.
  • Benchmark proves >= 2.5x throughput improvement.
  • Memory footprint reduced by >= 50%.
  • Pytest test suite in agent-engines/tests/test_quantization.py.

What to be done

  1. Create quantize_models.py in agent-engines/gpu_worker/.
  2. Update models.py to support loading quantized TensorRT/ONNX engines.
  3. Add benchmark and accuracy validation test suite.

What NOT to be done

  • Do NOT sacrifice tactical move accuracy (must maintain >= 99.5% agreement with FP32).
  • Do NOT break CPU inference fallback support.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions