Self hosting AI can be cheaper

Wake-on-LAN LLM Routing with LiteLLM and Hermes

(Because Your GPU Doesn’t Need to Burn Money While You Sleep)

Running a high-performance local AI rig gives you absolute privacy and blazing-fast token generation, but it comes with a heavy utility tax.

While a beefy enthusiast desktop with a high-end multi-gigabyte VRAM GPU is unmatched for crunching large language models, leaving it idling 24/7 is a massive financial drain.

The Real Cost of an Idle AI Rig

Let’s talk numbers.

A modern high-end desktop (think Intel 13th/14th gen or AMD Ryzen 7000/9000 paired with an RTX 4090 or RX 7900 XTX) typically draws 50-100W+ at idle, depending on your configuration and optimization level. Some folks have managed to squeeze it down to 50W with aggressive power settings, but most systems hover around 80W without serious tweaking.

In my case? 80W idle, even with everything optimized. That’s my desktop sitting there doing absolutely nothing, just waiting for me to ask it to summarize a document or help me debug some code.

Here’s the math:

Idle Draw: ~80W continuous Active Usage: Let’s say I use my local LLM for 8 hours a day (and honestly, some days it’s less, some days it’s more) Wasted Energy: 80W × 16 hours = 1.28 kWh per day, or roughly 38 kWh per month burned purely on empty air

At my local electricity rates, that’s basically the cost of a standard GPT subscription. So for every now and then usage, my “free” local LLM can actually cost more than just paying for a subscription if I ignore the idle power usage.

Plus you get:

  • Continuous component wear and tear
  • Unwanted ambient heat in your living space
  • Unnecessary grid load

  • The guilt of knowing your PC is burning money while you’re asleep

    That’s why I built this setup.

    The Solution: Smart Wake-on-LAN Routing

The idea is simple: split the architecture.

The Low-Power Brain: A lightweight mini-PC running Hermes Agent and LiteLLM 24/7, idling around 15W. This is your always-on orchestrator.

The Heavy Iron: Your main desktop PC hosting the heavy LLM weights via an OpenAI-compatible backend (Lemonade, Ollama, vLLM, llama.cpp—whatever rocks your boat).

Instead of keeping the desktop awake, it goes to sleep (S3/Modern Standby), drawing a mere 5W while listening exclusively for network magic packets.

When an automation or agent needs heavy reasoning, LiteLLM intercepts the request, fires a Wake-on-LAN (WOL) unicast packet to the desktop, intelligently polls the model endpoint with a set timeout, and seamlessly routes the prompt the second the server boots.

Architecture Overview

[Hermes Agent] ---> [LiteLLM Proxy (24/7 Mini-PC ~15W)]
                           │
             (Interception & Target IP Match)
                           ├──> Sends Unicast WOL Packet (Port 9)
                           └──> Polls /models Endpoint (10s TTL)
                                        │
                         [Desktop PC Wakes Up (~80W -> Active)]
  1. Configuring LiteLLM (config.yaml) Your LiteLLM proxy needs to register the target local model pointing to your desktop’s static LAN IP, alongside our custom callback handler.
model_list:
  - model_name: pc
    litellm_params:
      model: openai/Qwen3.8-27B
      api_base: http://192.168.0.10:13305/api/v1
      api_key: lemonade
      rpm: 100
    model_info:
      disable_background_health_check: true

litellm_settings:
  detailed_debug: true
  callbacks: custom_callbacks.proxy_handler_instance

Key points:

  • disable_background_health_check: true prevents LiteLLM from waking up your PC just to check if it’s alive (trust me, you don’t want that at 3 AM)
  • The api_base points to your desktop’s static IP and your inference server’s port
  • The callbacks line is where the magic happens
  1. The Custom Callback Script (custom_callbacks.py)

Place this script in your LiteLLM app directory. It traps outgoing calls targeted at your desktop’s IP, fires a directed unicast magic packet, and polls the local inference engine until it comes online.

import socket
import asyncio
import aiohttp
import traceback
import sys
from litellm.integrations.custom_logger import CustomLogger

def force_log(msg):
    print(msg, flush=True)
    sys.stdout.flush()

class WOLHandler(CustomLogger):
    def __init__(self):
        self.target_ip = "192.168.0.10" # Replace with your target desktop IP
        self.target_mac = "A1:B2:C3:D4:E5:F6"  # Replace with your target desktop MAC
        self.port = 9
        self.timeout = 10
        force_log("\n[WOL Hook] Initialized and registered in LiteLLM proxy!\n")

    async def async_pre_call_hook(self, user_api_key_dict, cache, data, call_type):
        try:
            model = data.get("model", "")
            api_base = str(data.get("litellm_params", {}).get("api_base", ""))

            if self.target_ip not in api_base and model != "pc":
                return data

            metadata = data.get("metadata") or {}
            messages = data.get("messages") or []
            if metadata.get("is_health_check", False) or (len(messages) == 1 and messages[0].get("content") == "hello"):
                return data

            force_log(f"[WOL Hook] Target invocation detected! Sending unicast WOL to {self.target_ip}...")
            self._send_wol_unicast()
            await self._wait_for_server()

        except Exception as e:
            force_log(f"[WOL Hook] CRITICAL ERROR: {e}")
            traceback.print_exc()

        return data

    def _send_wol_unicast(self):
        try:
            mac_bytes = bytes.fromhex(self.target_mac.replace(":", "").replace("-", ""))
            magic_packet = b'\xff' * 6 + mac_bytes * 16

            with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:
                s.sendto(magic_packet, (self.target_ip, self.port))
            force_log(f"[WOL Hook] UDP Magic Packet sent to {self.target_ip}:{self.port}")
        except Exception as e:
            force_log(f"[WOL Hook] ERROR sending magic packet: {e}")

    async def _wait_for_server(self):
        url = f"http://{self.target_ip}:13305/api/v1/models"
        start_time = asyncio.get_event_loop().time()
        force_log(f"[WOL Hook] Polling inference server at {url} (TTL: {self.timeout}s)...")

        client_timeout = aiohttp.ClientTimeout(total=1)
        async with aiohttp.ClientSession() as session:
            attempt = 1
            while (asyncio.get_event_loop().time() - start_time) < self.timeout:
                try:
                    async with session.get(url, timeout=client_timeout) as resp:
                        if resp.status == 200:
                            force_log(f"[WOL Hook] SUCCESS: Desktop server online and ready!")
                            return
                except Exception:
                    pass

                attempt += 1
                await asyncio.sleep(1)

            force_log(f"[WOL Hook] WARNING: TTL expired. Forwarding request to LiteLLM anyway.")

proxy_handler_instance = WOLHandler()

What this does:

  • Intercepts all LiteLLM calls
  • Checks if the target is your desktop (by IP or model name)
  • Ignores health checks and simple pings (so your PC doesn’t wake up for no reason)
  • Sends a unicast WOL magic packet to your desktop
  • Polls your inference server’s /models endpoint until it responds
  • Once ready, lets LiteLLM proceed with the actual LLM call
  1. Essential Host Prerequisites

To make this architecture function reliably, a few system-level configurations are required:

Static IP & ARP Binding Your desktop PC must have a permanent static IP address on your LAN (or a fixed DHCP reservation). This ensures the LiteLLM callback script always hits the correct address.

Without this, your WOL packets might go to the wrong machine or get lost in the broadcast domain.

Motherboard BIOS Settings Depending on your motherboard and BIOS, you’ll need to:

Disable ErP Ready or deep power-saving sleep states that cut auxiliary power to PCIe lanes and the motherboard chipset Enable Power On By PCI-E or Wake on LAN Ensure the network interface card (NIC) maintains standby power

These settings vary wildly between manufacturers, so check your specific motherboard’s documentation.

Operating System NIC Settings (Windows) In your OS network adapter properties:

Enable Wake on Magic Packet Disable power-saving features like Energy Efficient Ethernet or Green Ethernet that might cut link states during sleep Disable Fast Startup (this is a hybrid hibernation mode that can completely drop network standby handshakes)

You can find these settings in Device Manager > Network Adapter Properties > Power Management.

Windows Sleep Control Tuning Here’s where it gets interesting. You need to tell Windows when it’s okay to sleep and when it’s not.

The goal:

During active LLM inference: PC should NOT sleep (it’s working, let it work) During LLM idle time: PC SHOULD be allowed to sleep, even though the model is still loaded in VRAM

How It All Fits Together

Here’s the beauty of this setup:

  • Your OpenAI-compatible server (Lemonade, Ollama, whatever) is set to start on boot in Windows
  • When your PC wakes from sleep, the LLM provider is immediately available
  • LiteLLM can call it, Hermes gets the response, everything just works
  • When you’re done and the LLM is idle, your PC goes back to sleep, drawing just 5W
  • Repeat

The result? Instant local intelligence on demand, with zero electricity wasted standing still.

Final Thoughts

This setup is only valuable if Hermes isn’t doing constant 24/7 work. If your agent is running continuous background tasks that require heavy LLM inference, you might still be keeping your desktop awake a lot.

But for my use case—intermittent LLM usage, maybe 8 hours a day, sometimes less, sometimes more—this setup saves me significant power costs and keeps my local LLM setup both private AND efficient.

No more feeling like “I would be better off with a subscription” just because my idle power bill is higher than the subscription cost.

Feel free to adapt this to your own setup. The principles are the same, even if your hardware configuration is different.