Why I am building a local coding agent
I use coding agents for all my development now. I haven't really written code by hand for almost a year, yet I still steer the agents very actively most of the time. At work, we have automated much of the bug fixing and code review, but whenever I'm working on something new, I'm the human in the loop. I'm in the business of shipping mission-critical systems, and I don't have the luxury of not reading the source code, because what I ship has to actually work. (I wrote more about how I work in another blog post.)
I've been on the Claude Code Ultra plan since forever, because I want access to frontier models such as Fable, and I burn through quite a lot of tokens. These tools are invaluable. But I don't want to be locked into what Anthropic, or any other company, allows me to do. Nor do I buy into the direction coding agents are taking, where they become more and more of a black box, working autonomously in their own sandboxes. That's the opposite of how I work.
What I want, alongside Claude Code, is a coding agent that:
- Does what I tell it to do. Sandboxing is critical, but it's something the operating system should enforce, not the coding agent.
- Runs locally, even offline. My code stays on my machine, there are no usage limits, and I can use it whenever I need it.
- Gets the job done. A local coding agent has to get real work done.
What is the smallest model I can run?
What held me back for a long time were the GPU and memory requirements of capable models. When Bonsai 2 came out — a ternary compression of Qwen3.8-27B by PrismML — back in September, I pulled the trigger and started building dwim, a coding agent that runs the model on your own machine. The agent, the model, and the inference engine are all one program.
I initially got it running on an old AMD GPU from 2019 with just 8 GiB of VRAM! Unfortunately, that didn't pan out, because the Linux machine kept crashing all the time. Either the GPU is broken, or the AMD drivers are terrible. (And yes, with my Linux kernel hacker hat on, I spent way too much time trying to figure that out, but I never found the root cause.)
I then moved to a MacBook Pro with an M5 and 32 GiB of memory. The coding agent made some progress, but I had a hard time getting the Bonsai model to actually make a decision and act. The agent would identify a bug fairly quickly, and even come up with a solution, but then it went into a weird thinking loop and never completed the task.
This turns out to be a known failure mode of reasoning models. Pipis et al. [1] show that smaller models loop more than larger ones in the same family, and that distilled models loop far more than the teachers they learned from. In agentic settings, the problem compounds. Cuadron et al. [2] call it the reasoning-action dilemma: reasoning models favor extended internal reasoning over interacting with their environment. One pattern they identify, analysis paralysis, is exactly what I saw: the model keeps planning instead of acting.
So I switched from the ternary Bonsai to the model it was compressed from, Qwen3.8-27B, at four bits. The trade-off is memory: Bonsai ran on a 16 GiB Mac, while the four-bit Qwen needs 32 GiB. But the coding agent now completes real tasks.
The "do what I mean" coding agent
The name dwim is short for "do what I mean." Today it runs Qwen3.8-27B on my MacBook Pro and gets work done, but slowly, at about 10 tokens per second. The reason is simple: to generate a single token, the model has to read all 16 GB of its weights from memory, and the M5 can do that roughly ten times per second. No amount of clever engineering in the inference engine gets around that. A MacBook Pro with an M5 Max has four times the memory bandwidth, so the same model should run at roughly 38 tokens per second. A MacBook Pro with an M5 Max and 64 GiB is probably the right machine for this, but it's on the expensive side, which puts it out of reach for many.
The other two things I wanted are already there. dwim works offline once the model is downloaded, and nothing leaves my machine. It also does what I tell it to: it runs commands without asking for permission each time. I run it in a container or a virtual machine, because that's where I think sandboxing belongs.
What's next?
Next, I need to fix the performance issue by either getting more powerful hardware or trying other smaller models to see if they perform better, because right now the coding agent on my machines definitely isn't fast enough for real work.
Then I need to make the evals much more systematic to measure the coding agent's efficiency, especially as I change things. For example, I want to understand if I can provide better tools for the coding agent to work with, to reduce token usage and, therefore, complete work faster. A local coding agent needs to strike the perfect balance between intelligence and speed. You want the agent to be intelligent enough to get the work done, but fast enough that you use it instead of offloading the work to Claude Code and others.
Finally, since dwim has no sandboxing, I want to look into how to integrate it with existing solutions, including multiplexers like Superlogical's Rex, which I started using recently.
Try it
dwim is open source, and you can find it at dwim.sh. If you have a Mac with 32 GiB of memory or more, or a Linux machine with a 24 GB GPU, you can install it with:
curl -fsSL https://dwim.sh/install.sh | sh
Be warned that it's experimental, and it runs commands without asking, so run it in a container or a virtual machine, on code you've committed.
References
[1] Charilaos Pipis, Shivam Garg, Vasilis Kontonis, Vaishnavi Shrivastava, Akshay Krishnamurthy, and Dimitris Papailiopoulos. 2026. Wait, Wait, Wait… Why Do Reasoning Models Loop? In Proceedings of the 43rd International Conference on Machine Learning (ICML).
[2] Alejandro Cuadron, Dacheng Li, Wenjie Ma, Xingyao Wang, Yichuan Wang, Siyuan Zhuang, Shu Liu, Luis Gaspar Schroeder, Tian Xia, Huanzhi Mao, Nicholas Thumiger, Aditya Desai, Ion Stoica, Ana Klimovic, Graham Neubig, and Joseph E. Gonzalez. 2025. The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks. arXiv:2502.08235.