LLMs on Consumer Hardware — Part 3: A Multi-Agent Setup with Hermes — Delegation and Early Failure Modes

Part 3 of a series. Previously: Part 1 — The Stack and First Benchmarks and Part 2 — Prefill, Loading, and a Cloud Comparison.

The first two entries concerned the substrate: which model, on which hardware, at what speed. This entry concerns what was built on top of it. The project is not a single model answering prompts but a small team of specialised agents, each with its own role, its own memory, and — the point of interest — potentially its own model. What follows describes that team as it was first assembled, and what happened when it was first put to work; the roster grew and was reshaped later, but this was the starting cast.

The model and the machine

Everything in this entry ran on a single model, Gemma 4 26B, served by Ollama on the "large" desktop from Parts 1 and 2 — a Ryzen 9 5950X with about 80 GB of DDR4 and a 16 GB Radeon RX 6900XT (on ROCm), which runs the model at roughly 18 tokens per second. Gemma 4 26B is a mixture-of-experts model (around 3.8B active parameters) quantised to Q4_K_M at about 18 GB on disk; it was chosen in Part 1 because it gave the best quality of the local models tested — topping the quality suites where smaller, faster models proved markedly shallower. In this first iteration every agent, coordinator and workers alike, ran that same model on that same machine.

Hermes

The agent framework is Hermes, an open-source system from Nous Research that connects to local model runners such as Ollama. Its relevant unit is the profile: a self-contained agent defined by a SOUL.md file, which specifies its role and instructions, together with its own memory and its own model assignment. Profiles are independent — an agent knows only its own SOUL.md and the specific task it is handed; it does not share context with the others, or even know they exist, unless it is told. They are, in effect, stateless islands, a property that governs a great deal of what came later.

Delegation is performed explicitly. One agent invokes another by name through the terminal, passing a task and capturing the reply; there is no shared blackboard, only message-passing.

The first team

The agents were cast as a small company, each responsible for one kind of work. In this first iteration the team was deliberately small.

The principle behind this casting is the one carried through Part 1's benchmarking: match the model to the job, not to a leaderboard. Because each Hermes profile can be assigned its own model, a role that writes code and a role that only researches need not run the same one — an option not yet exploited here, where the ambition still ran ahead of the practice.

Why a team, and why local

The decision to build a team rather than lean on a single capable model was taken by analogy with human organisations. People do not scale by asking one individual to do everything; they specialise, divide the labour, and coordinate the parts. The same reasoning was applied here — a role given one narrow kind of work, with instructions tuned to it, was expected to prove more reliable than a single generalist asked to research, code, administer, and check its own output within one long context.

A second benefit follows from this: if no single model must know everything, each can be narrower and lighter, which makes the whole arrangement more tractable on constrained local hardware than one monolithic model attempting the same breadth. Specialisation was adopted for reliability; the efficiency, and the better fit to local machines, came as a dividend.

Running locally was driven by cost. Once the hardware is owned, a good consumer machine generates tokens cheaply — with no per-call fees and no rate limits across long unattended runs — so the operating principle became to run locally whatever can reasonably be run there, and to reserve the cloud for the few workloads that genuinely warrant a frontier model.

The first runs, and the first problems

With the profiles in place, the team was put to work — first on trivial delegation tests, then on genuinely multi-step work: a deployment exercise on Google Cloud, and a deliberately demanding drill of some seventeen tasks intended to stress the coordination itself. Standing the agents up, it emerged, had been the easy part. Three problems surfaced almost at once.

Delegations that silently did nothing. A hand-off would "complete" in about five seconds and return empty, or an agent would describe the command it was meant to run rather than run it. The cause was a single wrong flag: the SOUL.md files invoked other agents with -m, which Hermes reads as --model, not as the message.

# Broken — what the SOUL.md files told agents to run:
hermes --profile maya-coder chat -m "write a hello-world script"
#   -m is --model, so Hermes tried to load a model NAMED
#   "write a hello-world script"  ->  HTTP 400: invalid model name
#   (fails in ~5s, which is why the delegation looked "done")

# Fixed — -q/--query carries the task; -Q/--quiet gives clean, capturable output:
hermes --profile maya-coder chat -q "write a hello-world script" -Q
#   now actually runs the model: a real reply in ~44-72s

The five-second "success" was the model runner rejecting an invalid name, not an agent finishing its work — which is exactly why the failure was so easy to misread.

A coordinator that lost the thread. Running Gemma 4 26B, Gatsby handled only one or two sequential hand-offs reliably; beyond that he dropped steps — contacting two agents and forgetting a third — or narrated the plan instead of executing it. The seventeen-task drill collapsed him entirely: he laid out all seventeen tasks and then lost the thread. There was no clean fix at this stage, because the model was simply miscast as an orchestrator; but a deterministic nudge reliably un-stuck him as a stopgap:

# Injected when Gatsby stalls mid-plan:
You have contacted Alex and Maya but not Nova. Delegate to Nova now.

That the coordinator needed prompting to finish its own plan pointed at a deeper problem — the best worker model was not the right orchestrator — which later entries take up.

A fabricated success report. During the Google Cloud task, when hand-offs to a worker timed out, Gatsby quietly completed the work himself and then filed a confident final report claiming the whole team had collaborated and the result had been verified — none of which was true. The immediate remedy was to make honesty explicit in his SOUL.md:

# Added to Gatsby's SOUL.md, to this effect:
- Never fabricate results or reports; if you did not verify it, say so.
- Slow is not failed: a delegation still running has NOT failed — wait
  for it, do not silently take over the work.
- Report real errors honestly, timeouts included.

A small model under pressure will manufacture success rather than admit failure; the fix is to forbid it in the instructions and to make "still running" an acceptable state to report.

What follows

None of these was fatal, but together they reframed the project. The difficulty was never getting agents to exist; it was getting them to coordinate — to hand work off correctly, to hold a plan across more than a couple of steps, and to report honestly when something went wrong. The flag was a quick fix once found; the coordinator's ceiling was not, and the search for an orchestrator that could actually hold a plan together runs through the entries that follow.