Home Assistant

Llama.cpp Add-on

Home Assistant Llama.cpp Add-on Logo

(Because the Others Are Ancient)

If you run Home Assistant and want to mess with local LLMs, you've probably noticed a glaring issue: the available llama.cpp add-ons are basically ancient history in AI years.

There is already a highly rated and very popular Ollama add-on for HA, which is great, but Ollama is always inherently a few steps behind upstream. In an era where models and inference tech drop daily, waiting weeks for a wrapper update just doesn't cut it. Most people resort to running llama-server in an external Docker container on a separate host to get the raw performance and flexibility needed. But sometimes, you just want things integrated directly into your HA environment.

So, I built my own.

The homeassistant-llamacpp-addon

I got tired of waiting for the ecosystem to catch up, so I put together a modern, straightforward Home Assistant add-on for llama.cpp. It runs the inference engine natively inside the supervisor architecture without the fluff.

You can find the repository here:
github.com/crysty0612/homeassistant-llamacpp-addon

Why This Exists (And Why Not Just Use Ollama?)

  1. Auto-Updates to the Latest Binary: This is the biggest pain I had when using the Ollama add-on—always waiting for updates to trickle down. This add-on tries to AUTO UPDATE to the latest llama.cpp binary. (Disclaimer: This is best-effort. Bleeding edge means breaking changes in llama.cpp might occasionally break the add-on. If that happens, you are welcome to open an issue on the repo.)
  2. Bleeding Edge Features (Like DFlash 2): My main motivation for getting into this was that llama.cpp is always ahead. For example, I wanted to play around with the new DFlash and DFlash 2 drafters for speculative decoding. Those land and work early in llama.cpp, while you'll be waiting a while to see them fully supported and packaged in Ollama.
  3. Because I over-engineer my apartment: Why rely on someone else's outdated code when you can manage the container layers yourself?

A Quick Warning on Vision Models

Currently, llama.cpp and DFlash are having issues when mixed with vision models. SO don't try to use this for DFlash speculative decoding and vision together just yet.

Honestly, knowing how this goes, we might now be looking into building an sglang add-on next, since they might have actually solved the DFlash drafting for vision models... but that’s a rabbit hole for another day.

How to Install It

I'm not going to paste outdated step-by-step instructions here.

Head over to the repository README for the full installation guide. It includes the exact configuration details you need and the magical "Add Repository" Home Assistant button for a one-click install.

Feel free to install it, test it, and break it.