homelab of rene
← All writing

▍ Finding · local inference

Engine Room: AI ops on a 6-node homelab

Validated Jun 23, 2026 · Design — not yet built

This picks up where Firecracker left off - the WhatsApp companion i built for me and a friend. Now comes the part of taking what i built and using it as a base for fleet ops and remote management of my homelab - so, back in Claude Code.

The full brief - the one i’m actually building from - is specific to my own setup, so it stays private. But the trust model underneath it is portable, so i pulled it out into a general blueprint anyone could use - it’s at the end of this post.

Morning! so we built this thing :) ~/source/firecracker. It is really good, i love it. i got a few ideas where to take it. Right now as a private me-and-my-friend group it is perfect as is. But i got a few ideas on top of my head right now ..

Why not existing tools

I had a chat with GLM, and in the midst of discussing features i shared my new idea. There are many tools for what i want to achieve and would make sense to use in general - but for my own particular use case - i am just enjoying custom narrow need based approach, build the minimal with maximum impact custom tailored to suit just one target.

@glm i already got new ideas. This chat is all about friends and ai. But i got plans for you hehe. I will create a separate group, just you and me. I need to manage and run my hardware fleet and services and projects i build, remotely. I want to be able to go out camping with a line back home, you helping to pull off wonders without me behind my cockpit on site lol

Full SSH on the table

@glm haha, i bet. Well in short, you will have full ssh access to all of my nodes, 6 of them ssh, both normal and root access. Or i ask to run opencode or claudecode. Or i ask to upgrade vllm on node 4 and i ask you to pull the model x from huggingface. Or i ask, restart yourself or shutdown, i need to use my gpus for something else. And there’s always gonna be 2 modes: when you are sleeping, claude is holding your seat and will wake you up when i need you again and will give the control back .. that is the core of it :)

@glm claude will have the same full control. if i mess something up and you are not waking up, claude gonna go and figure out what is wrong and will fix whatever is needed. The difference is, claude api is very expensive in relative terms - you burn a few hundred watts of electricity i need to cover. You are on par with claude. Same time you are different, both unique, each with own quirks. There will be a system service handling the seat occupancy and switching, and a link to whatsapp .. well, it exists already - this service has been facilitating all of this. The failure point though is, let’s call it the gateway service. And if the gateway service itself crashes, i could have a hot standby on another node. Also i could have a simple control app on phone, managing the gateway service. And if that fails too, then i just gotta accept it: i need to get back in front of my home computer.

The trust spec emerges

A few Claude Code back-and-forths in.

yea this was just a general idea. i agree Engine Room is the one that needs careful thought and design. what i mean by “restart yourself” -> glm runs a script which brings vllm down and pulls it back up. once down - if it crashes - claude api automatically takes the seat. When i send the next message, like “Glm you back online?”, i get either “yes all good, vllm restarted” or “No, it’s me claude, something went wrong with restart, you want me to look at it?”. Or if i say “@glm have a nap, glm, i need gpus for something else right now” => glm runs a script to shut down the fleet whatsapp config, either partially or fully (image gen on a second node, qwen vl on the cockpit) and claude api takes the seat. And then i could ask “Wake up glm and give the seat back”. The full control part probably needs some basic safety protocol. claude api could take my input and the tool glm is about to execute and judge if the bash command about to execute matches what i requested. No ‘YES’ confirm for every action is good safety practice, but it gets annoying soon vs having practical value. probably claude api judging 2 things: is the command about to run matching my request - if yes then allow; if the command is destructive in nature, then prompt me for yes. Seems simple enough :) claude api layer requests should be tiny enough so i barely notice the cents it costs, and remote is not every day anyway. And yes, log everything. The last stand, the pure android control app - it could have an option to check the logs directly, or stop services and see statuses etc .. no ai, simple static app. And i think i would love android native. I have an s24, nice phone. Also @glm freeze, or sending commands wrapped in chat clothes - i think i’m skipping that, because then i need to start paying attention to what is a system command vs just input for glm. The android control app is the pure system-commands layer.

Yup, this is the shape :) Write up the Engine Room trust spec brief. The control app can live separately in ~/source/engineroom_android_control_app i think. Also maybe don’t touch ~/source/firecracker .. it works right now as is. Let’s plan maybe a separate copy under source, and once built it can retire ~/source/firecracker. And regarding the anthropic api offline - we could make it explicit: nothing happens by default. In the control app i explicitly either enable read-mode only, or full-control bypass. Which is risky, but in rare moments i am fine taking the risk where i go into judgement mode on myself: “üheksa korda mõõda, üks kord lõika”. …

was thinking a few more things. all of my nodes have ‘ssh’ as normal user and ‘ssh root@’. the cockpit, where we’re at right now, is where the engineroom runs. Also i’m thinking the engineroom should run by default under my normal user, no root access. But sometimes i may need root access, simple ‘sudo apt install whateverineedfortaskathand’ - default mode does not allow this. The control app could have an option to restart the service as root. Nothing i’d use daily, special use cases. I think there’s a way to allow temporary root commands from my normal user context, but this means i would have to run ‘sudo authorizingcommand’ before. So maybe the control app should have an option where i could type in a command which will run as sudo without restarting the engineroom under root. engine room under root is something i most likely never use, but it would be nice to have, and when i do use it - i mean i run claude code sometimes ‘sudo su & claude’. This is the ‘CAREFUL! - you know what you are doing - all safety rails off’ mode. 2. 15min fine as default if i don’t flip it back manually. 3. you got my vote. 4. yup, worth adding. 5. just one - notes to start it off with. like ‘@glm add it to notes: this, that and third’. Also the engineroom and control app: what if the cockpit goes down for whatever reason - service apps, or a planned restart doesn’t come back online. the storage node is my standby host, where i have gitlab and just storage. The engine room could take over to active from standby. But i think here: no restart-under-root-credentials option, neither the authorized-root option separately. I just gotta deal with it haha - if i messed up the cockpit state then this means i should strip the remote root privileges on the storage node and just deal with it once i get back home. i would not want to mess up the storage node as well.

A: if the storage node standby is active, the option to elevate the engineroom to root mode - not gonna happen. i could still root other hosts via ‘root@’, this is fine, i may or may not use it - given the cockpit is offline it may have crippled my ongoing work anyway where it depended on it. goal is: no root on the storage node itself. that is to ensure i don’t mess up the access to back home. all other nodes besides the storage node can go offline for all i care lol; the storage node by design should stay online as the last one standing, and no root access prevents any dumb mistakes cutting off the entire access. So the goal: protect the storage node and the engineroom service from my own mistakes. B: yup. C: warn is ok.

End of quotes - gonna leave this here and gonna head back to building it.

The blueprint

A general design for letting AI operators run a machine fleet from a phone — safely. The transferable pattern, stripped of any one deployment.

The plumbing (a chat bridge, SSH, a model behind an API) is the easy part. The hard part — the whole job — is the trust envelope: how you hand an AI real, mutating control without handing it a way to wreck things or be talked into it.

Roles, not hostnames:

  • control host — always-on, low-compute. Runs the gateway (chat bridge + seat router + audit log + a small control API). Survives operators dying.
  • worker — the cheap, always-available operator (a local model). Does the day-to-day.
  • reserve — a premium model (paid API). Two jobs: the judge (reviews every worker mutation) and the backup operator (takes over when the worker is down).
  • control app — an out-of-band, AI-free app on your phone. The deterministic control plane.
  • standby — a weaker failover for the control host.

The principles

1. Two operators, one seat. A cheap worker handles everything; a premium reserve stands by. Whoever is alive when a message arrives holds the seat — no handoff protocol, just a liveness check. Shared chat history is the handoff: the reserve takes over mid-incident already in context.

2. The seat never lives on hardware an operator can reclaim. If “free the GPUs” kills the worker, the thing that can later wake it cannot be co-located with it. Gateway on the control host; operators on the reclaimable compute. This isn’t a preference — it’s forced by the self-terminate case.

3. Reads open, mutations gated. Let the AI freely inspect, tail logs, check status — no ceremony. Every state-changing action goes through the gate, and is expressed as a curated verb set, never free root shell. A model that reads untrusted input daily (web results, transcripts, text in images) must never have its output become an arbitrary command. Verbs bound the blast radius to things you pre-vetted.

4. The judge — match intent, not keywords. Before a mutation runs, the reserve model gets the user’s actual request + the concrete command + minimal context, and returns one of: ALLOW (a reasonable step toward the goal, not destructive), CONFIRM (destructive — ask the human first), or BLOCK (does not serve the request → refuse and log). Judge against the goal, not the lone command. The intent-matcher is the prompt-injection firewall, for free: injected text that makes the AI try something the user never asked for won’t match the goal → BLOCK.

5. A dumb floor under the smart ceiling. Independent of the judge, a small static deny-list always forces CONFIRM on catastrophe patterns (rm -rf, mkfs, dd, DROP TABLE, anything touching keys). The judge is smart but an LLM and will misjudge once in a while; the static list is dumb but never wrong about the handful of things that end the day. It fires for both operators and even in bypass.

6. Three modes — and the default is “nothing happens.” NORMAL (reads open, mutations judge-gated), READ-ONLY (sees everything, changes nothing), BYPASS (mutations run with no judge). Fail-safe, not fail-open: if the judge is unreachable in NORMAL, mutations are blocked. BYPASS auto-expires on a short timer so you can’t leave the blade out by wandering off. The static floor still fires in BYPASS.

7. Two planes, no mode-confusion. The chat plane is 100% natural language — you just talk to the AI. Every deterministic “I really mean it, system” action (mode switch, service stop/start, status, logs) lives in the separate control app. You never have to parse “is this a command or just talk” mid-conversation.

8. Privilege ⊥ oversight. Which user the process runs as is an independent axis from the mode. Run unprivileged by default; elevation is a separate, deliberate, app-only action. Maximum danger (elevated and unjudged) requires two explicit toggles — never reachable with one tap. Measure nine times, cut once.

9. Log everything, in the open. Append-only audit of every mutation: who, what command, which target, the judge verdict, the outcome. And echo actions into the chat as they happen, so the human watches the work in real time.

10. The standby is deliberately weaker. A failover that can manage the fleet but cannot damage the one node that guarantees you can get back in, and has no bypass. If you’ve already made a mess from the field, you don’t want a second box you can also break. Build it last, after the primary is trusted.

The shape that falls out: concentric circles of autonomy. The center is you at the keyboard. Everything above that is how far the distance can stretch — phone in a tent — before you have to drive home. The human is the last resort, never the first.