Metrics and alerts
Prometheus scrapes – targets. – alert rules page my phone through ntfy.
Live snapshot · loading
One server in my house.
A crew of AI agents runs it.
I give the orders from my phone.
I'm Anthony Kubeka. I write event-driven backends for a living. Normandy is the system I built for myself: 12 Docker stacks deployed from git, monitoring that pages me, and agents that take a voice note and come back with a verified result.
- containers running
- –
- commits to the infra repo
- –
- alert rules, 0 firing
- –
- agent skills
- –
- runbooks
- –
- agent tasks run
- –
01
Every deploy starts with a commit
I edit a file, commit and push. Forgejo (my own git server) hands the job to a runner on the same box. The runner works out which stacks the commit touched and redeploys those.
The whole server rebuilds from the repo, so nothing important lives only in my head.
- – commits since 2025
- – in the last 30 days
- – deploy workflows
- Last commit –
- push
- Forgejo
- runner
- deploy
- health
- notify
$ press the button
Simulated run. The steps and the stack names are the real ones.
02
What runs on it
– containers in – stacks. DNS, ingress and git live apart from the things that can fall over quietly, so a bad media deploy can't take the house's DNS with it.
03
It watches itself, then tells me
Night Watch
A plain script, on purpose. Every morning it sweeps failed units, deploys, backups and alerts, sends me 1 briefing, and opens an issue for anything red.
Backups
Restic snapshots every night, and a restore test every month. A backup counts once I've restored from it.
A day on Normandy
– timers live in git. These are some of them. Times in UTC; I'm at UTC+2.
04
The crew
I send a text or a voice note to a Telegram bot. An agent on Normandy does the work in its own workspace. I get back a question with buttons, a verified result, or a line in the evening digest.
The phone is the control channel
Every step I take has to work from an iPhone. If a step needs a terminal, I log it as a defect.
Act, then report
Agents act without asking, apart from a short ask-first list. Every change lands in a log line and in the digest.
Every account is crew
An account is a subscription and its quota, from any vendor. Safety belongs to the task's workspace, so I can move a task from Claude to Codex halfway through.
How a request moves
What asks first
- Deleting data that can't be restored
- Moving money
- Messaging people
- Posting in public
- Exposing a secret
- Changing an access path
A deletion covered by a verified backup snapshot counts as reversible, so it goes ahead. An access-path change runs with an automatic revert timer.
The score so far
One dead login failed the same job 3 times in a row. The worker gets its token through a wrapper now.
Known limit, from my own guide: the ask-first guard reads a request once, at intake. Guards at the tool level are next.
05
One instruction file, every agent
Claude Code and Codex both work on this machine, and they have to follow the same rules. So there's 1 file. Each agent's config path is a symlink to it, and a pre-commit hook fails the commit if a link has turned into a regular file.
Two copies drift. One inode can't.
The same goes for rules. A rule that changes behaviour gets updated on every agent surface in the same session. A rule changed in 1 place is a bug.
Agents also read the web through local tools on this box: a private metasearch instance and a headless browser. I get the source text, and every agent gets the same input.
– skills
A skill is a folder of instructions an agent loads when the task matches.
06
Shared memory
The agents share 1 Markdown vault. Raw sources go in and never change. The wiki pages belong to the agents: they write them, index them and log every operation.
It's close to Karpathy's LLM-wiki pattern: immutable sources, a wiki the model owns, and a schema file that says how.
Background agents never touch the live vault. Each one gets a job copy and an integrator merges it. A conflict becomes a question on my phone.
raw/sources, read-only
resources/the agents' wiki
index + logcatalogue, audit trail
- notes
- –
- wiki pages
- –
- logged operations
- –
- commits
- –
The vault commits itself every 5 minutes. Last commit –.
07
The research inbox
I send a link, a screenshot or a question to a second bot. An agent researches it, checks it against my own notes, repos and server, and replies with numbered actions. Replying "do 2" starts a task. Its tools are read-only, so that limit holds whatever the prompt says.
Real notes, trimmed. – delivered so far.
08
Rules that came from incidents
Each of these is in my runbooks because something went wrong once.
-
Measure DNS from a public resolver
I recorded a zone fact from inside the house. The LAN resolver answers for every subdomain, including ones that don't exist. The table stayed wrong for 2 weeks.
-
A deny-all lockdown breaks every existing subscriber, quietly
I locked down the notification server. The morning briefing job kept running green. Nobody received it.
-
Caddyfile indentation is tabs
Mixed tabs and spaces made whole site blocks vanish without an error. The service just stopped answering.
-
Empty output is a result, so say it out loud
The page extractor returns nothing for JavaScript sites, bot walls and dead links. An agent may fall back to another tool, and it has to tell me it did.
-
A timeout has to actually kill
A weekly agent job hung. The fix is a nested pair: the inner timeout kills the agent as its own user, the outer one backstops sudo.
-
Keep tokens out of remote URLs
One repo carried a stale token in its remote and started answering 403. Credentials come from a helper at call time now.
09
Poke at it
A toy shell over the real snapshot. Try help.
10
How this page is made
A timer on Normandy runs an export script. The script reads the live system and writes 1 JSON file through an allowlist: a name on the list gets printed, everything else gets counted.
A second script fails the build if a private term, an internal address or a key shape shows up anywhere in the output. Then it pushes, and the page redeploys.
The page is 1 HTML file, 1 stylesheet and 1 script. It loads nothing from anyone else.
This is how I build
I'm a senior software engineer in Johannesburg. My day job is event-driven systems in .NET, Python and Azure. If your team needs someone who builds like this, I'd like to hear about it.