Open-Source Llama-Modchip Runs 284B-Parameter DeepSeek Model on a 64GB PC Without a GPU

The GitHub repository, published under the name synchopate and licensed under Apache 2.0, is now at version 2.4.0. If its numbers hold up, a model of a size usually run on GPU servers could run locally on a high-end laptop or workstation. That matters to companies that want to keep data on their own premises or cut what they pay for inference.

According to the project's documentation, the model's weights are compressed to 56.56 GiB, about 1.71 bits per weight. The model uses 13.17 billion of its parameters for each token it generates. The author says memory use peaks at 57.61 GiB even with a 256,000-token context window, which leaves room on a 64GB machine without swapping to disk.

On the main test machine, a laptop with an AMD Ryzen 9 9955HX processor and 64GB of memory, the project reports reading prompts at 44.36 tokens a second and generating text at 9.1 to 14.7 tokens a second. That generation speed is usable for chat and coding help but far below what data-centre GPUs deliver. The project also supports Nvidia's Nemotron-3.5-30B-A3B model, aimed at machines with 16GB to 32GB of memory, and reports generation speeds of 25 to 111.6 tokens a second depending on the hardware and settings.

The speed comes from changes to the ggml and llama.cpp inference code. These include AVX-512 routines for processing prompts, a new attention kernel and memory hints that stop rarely used expert weights from pushing frequently used data out of the processor's cache. The project also adds a custom compression format called Fractal-64 and a self-speculative decoding scheme that guesses upcoming tokens with a small n-gram model. It includes a launcher for Anthropic's Claude Code tool.

The author lists several limits. Standard llama.cpp, Ollama and LM Studio cannot load files in the new format, so users need the project's own patched software. Ready-built binaries are offered only for Linux on x86 processors. The default mode runs all six routed experts for each token, and running fewer is faster but lowers quality. Most of the benchmark and quality results, including a perfect score on the author's own evaluation, are self-reported or submitted by community members and have not been independently checked.