27B Parameters in 4GB: Bonsai's 1-Bit Model Runs on an 8GB GPU (2026)

Last updated: November 12, 2026 · 3-minute read

A 27-billion-parameter model that needs roughly 4GB of VRAM. XDA's Nolen Jonker says Bonsai 27B runs on his 8GB GPU as smoothly as the 4B models he used before.

A 54GB model compressed to 4GB

Bonsai 27B comes from PrismML and is a compressed version of Qwen 3.6 27B, released in July 2026. The full model is 54GB. The 1-bit build stores each weight in roughly a single bit and brings the download down to about 4GB. PrismML's Hugging Face model card lists the 1-bit build at about 3.9GB, at 1.125 effective bits per weight, under an Apache 2.0 license.

Jonker's own download came in at just under 5GB with the vision file included. PrismML also offers a ternary build at about 5.9GB and a newer Bonsai 2, but both are larger and the ternary already fills most of an 8GB card.

These are PrismML's own figures, not independent tests.

Running Bonsai on an 8GB card

Bonsai is a Staff Pick in LM Studio's model search. Jonker offloads the whole model to his 8GB GPU and keeps context around 30,000 tokens, which holds longer conversations and lets him iterate on code and design for longer. Bonsai supports up to 262K tokens architecturally, but he says he does not need anywhere near that.

Thinking, vision, and tone

Bonsai thinks through a problem before it answers and supports vision. Jonker found that an old preset capped thinking at 1,024 tokens and cut replies off halfway. A fresh preset close to PrismML's suggested settings fixed the issue.

He describes the tone as surprisingly warm, similar to Gemma 4 and perhaps Claude's Sonnet. That is his own impression, not a measured fact. He also notes that Bonsai holds onto a system prompt well; one prompt requiring short paragraphs of plain prose stuck to that format across a whole thread.

Math, apps, and tool calling

PrismML reports a GSM8K score of 92.8 on the model card, a benchmark of everyday maths word problems. The vendor's own numbers also show that general knowledge is where compression cost the model the most, so Jonker verifies some facts.

In Cherry Studio, Bonsai works perfectly alongside cloud keys for Claude and GPT, letting him send one prompt to several models side by side. Open Design connects the same way through its bring-your-own-key option. Jonker says coding is not Bonsai's strong suit; a test in Open Design produced a sensible mobile app login screen prototype, but it took a little long.

Tool calling is hit or miss. LM Studio supports MCP, and Bonsai handles a filesystem MCP well, knowing when to call the tool and understanding file context. His Web MCP is weaker: Bonsai does not reliably know when to search the web, so he must tell it to use the Brave Search MCP every time, or it cannot even get the date right.

Where Bonsai fits in 2026

Slightly larger models are being optimized for mid-tier and lower-tier consumer hardware. Qwen 3.5 9B, Gemma 4 12B, and Qwythos 9B sit alongside Bonsai 27B in this lineup. Jonker says Bonsai already handles the majority of what he uses local AI for, from mundane questions to private health and finance documents kept on his own machine.

Read more local AI coverage on /blog, or explore work in /#projects.

--- Not affiliated with PrismML, Qwen, or XDA. Sources: XDA, PrismML (Hugging Face).