1
Transformers now runs llama.cpp quants
Transformers now supports GGUF quantized checkpoints (used by llama.cpp) via the `kernels` library. Load a GGUF model with `from_pretrained` on Apple Silicon, run generation with standard Transformers APIs, or serve it via `transformers serve`. Benchmarks on an M2 Max show token‑throughput comparable to llama.cpp, with fallback to dequantization if kernels are missing. The integration also enable…
Hugging Facehuggingface.co10 minHN2