The llama.cpp project merged pull request #25395 yesterday, adding native support for the Hy3 architecture. Within hours, developer satgeze published Hy3-1M GGUF quantizations on Hugging Face, making the 1-million-parameter model immediately runnable on local hardware.
Early testing shows the Q2_K quantization delivering coherent output at 10-11 tokens per second on an RTX 5090 paired with a Zen 4 CPU and 96 GB of DDR5 memory. The PR author, satindergrewal, implemented the necessary kernel changes to accommodate Hy3's attention structure, which differs from the Llama family that llama.cpp originally targeted.
For developers running models locally, this marks another step toward viable sub-7B parameter models that fit comfortably in VRAM without aggressive offloading. The 1M parameter scale positions Hy3 as a candidate for edge deployment, embedded assistants, and rapid iteration workflows where larger models introduce latency.
The speed of GGUF availability — less than 24 hours after the upstream merge — reflects how tightly the quantization ecosystem now tracks upstream changes. Quantization maintainers can package new architectures almost as fast as inference engines adopt them.
