404human

Checkpoint 1.1: I pushed a real app until it broke, and watched

404human — Wed, 14 Oct 2026 13:20:00 +0000

My practice server from M0 is great for tests. It fails exactly how I tell it to, every time.

But real apps don’t fail politely. They slow down a little, then a lot, then all at once. So for the first step of M1, I built a real app and pushed it until it broke, while my monitor watched.

What I built

A small Node.js API, shaped like an AI app backend, with zero dependencies:

/api/chat pretends to call an AI model. Most answers take about half a second, some take up to 5. It only allows 64 at once; anyone extra gets “429, too many requests”.
/api/search does real CPU work. In Node, CPU work blocks everything else, including the health check.
/health says “I’m alive”. /ready says “I can take traffic”, and honestly says no when the app is stalling.
A chaos switch to make it slow, broken or crashed on demand.

A load generator that ramps traffic up smoothly and reports, every second, how fast answers came back.

An experiment script, so anyone can rerun this exact test with one command.

What the monitor saw

Traffic climbed from 20 to 400 requests a second over one minute. Meanwhile my monitor checked /health and /ready once a second, with a 1-second time limit:

 req/s   /health               /ready                    what users got
   ~50   0.3 ms                fine                      all good
  ~115   105 ms                fine                      still fine (p95 1.2 s)
  ~147   fast-ish              503 "event loop stalled"  first "too busy" replies
  ~172   no reply in 1 s       slow                      135 OK + 47 "too busy" per second
  ~197   no reply              no reply                  still mostly OK
  ~305   no reply              no reply                  requests start timing out (10 s)

What it taught me

1. Slow comes before broken. The health check went from under a millisecond to over 100 ms long before anything failed. Watching that number is an early warning.

2. The app warned first, then went quiet. /ready honestly said “not ready” before anything failed. A bit later, the app was too busy to answer anything, including its own health check.

An app can’t report its own trouble when it’s drowning. That’s exactly why the monitor lives outside the app, with its own stopwatch.

3. The monitor saw it about 20 seconds before users did. Users started hitting 10-second timeouts around the 45-second mark. The monitor had been failing checks since second 24.

4. One check proves almost nothing. In the middle of the slowdown, one check came back in 1.5 ms, between readings of 60 and 136 ms. A single sample is noisy. That’s why the next step only raises an alarm after several failures in a row.

Where I messed up

A 5-second trial run before the real experiment found two bugs. My load generator used a setting before it was defined, and my script waited for the server to finish, which never happens. Lesson (again): run it small before you run it big.
My “slow the app down” test didn’t always work. 30 CPU-heavy requests stall Node badly if they arrive at the same moment. Spread out by a few milliseconds, the exact same work causes no stall at all. Same load, completely different picture. The test now uses one long request instead, which stalls it every time.

What it doesn’t do yet

The monitor doesn’t check on a schedule by itself yet. A small script called it once a second for this experiment.
This was one machine, one run, one minute. These numbers are real, but they aren’t a benchmark.

Up next

Checkpoint 1.2: checking on repeat. The monitor runs on its own schedule, never overlaps checks, and only raises an alarm after several failures in a row, so one noisy sample can’t cry wolf.

Checkpoint 0.7: I let a tool attack my code. It found 19 holes.

404human — Wed, 14 Oct 2026 13:10:00 +0000

So far, every checkpoint ended with me breaking the code on purpose and checking that a test noticed.

There’s a problem with that: I chose what to break. I’m also the person who wrote the tests. Of course my tests caught my attacks.

So for this checkpoint, I handed the attacking over to tools.

Attack 1: a tool that plants bugs

There’s a tool called cargo-mutants. It makes hundreds of tiny changes to your code, one at a time. It flips a > into a >=, deletes a line, makes a function return nothing. After each change, it runs all your tests.

If the tests still pass with the bug in place, that’s a survivor: proof that a real bug could hide there.

First run, on all my code: 238 planted bugs, 19 survived. Every one of my hand-picked attacks had been caught, and the tool still found 19 holes.

The survivors told a very clear story:

Edges. Nothing tested a page of exactly the 64 KB limit, or a web address of exactly 2,048 characters. A > quietly turning into >= went unnoticed. Edges are where bugs live, and I’d only tested near them.
Things I never actually checked. A monitor set to send a HEAD request could have sent GET instead, and no test would have noticed. Same for the “this reply is compressed” safety check: no test ever sent a compressed reply.
Two copies of the same code, where only one was tested.

I wrote a test for each one, and ran the tool again. 258 planted bugs, 0 survivors. Two of them made the tests hang instead of fail (an endless loop). That still counts as caught, because the test run stops on a time limit, just more slowly.

Attack 2: thousands of random inputs

My tests used examples I made up. Property tests flip that around: I write a rule that must always hold, and a tool throws thousands of random inputs at it. For example:

Every address inside every blocked range is blocked, using random addresses, not my favourites.
::ffff:10.0.0.1 and other “IPv4 in an IPv6 costume” forms always get the same answer as the plain address.
The address checker never crashes, no matter what string you throw at it.
To catch the opposite mistake, big chunks of the real internet must always be allowed. A typo that blocked half the internet would fail this.

11 rules, about 45,000 random cases every run. No counterexamples. Honestly, I was a little disappointed.

Attack 3: 400 checks at once

Then I fired 400 checks at my practice server at the same time. Each one had to get the right answer, using exactly one connection.

It passed. Then I ran the whole test suite 10 times in a row, and it failed 3 times out of 10.

Under load, a few checks reported “timed out while connecting”. Was my monitor wrong? The Linux kernel keeps a counter for exactly this situation, so I checked it before and after each run: 124 to 420 dropped connections every time.

The culprit: my practice server could only keep 128 connections waiting to be picked up. That’s the default in Rust’s networking, copied from the standard library. A burst of 400 overflowed it, the kernel dropped the extra connection attempts, and they took over 2 seconds to retry.

So the monitor was right. Connecting really did take too long. My test server was the weak link. I raised its waiting room to 4,096, and the result was 0 dropped connections in 20 runs, and 10 out of 10 clean runs of the full suite.

Bonus: this is exactly why the plan says the monitor should never fire more than 2 checks at once at the same server. Now I’ve seen why with my own eyes.

Where I messed up

The tools found two bugs in my tests, not my code:

A test that agreed with anything. To test a size limit, I used the limit’s own value from the code. So when the tool shrank the limit, my test shrank with it, and still passed. A test has to write down the expected number itself.
A pipe I closed by accident. One test read a line from a program’s output through a temporary helper. In Rust, a temporary is thrown away at the end of its line, and throwing it away closed the pipe. The program’s next print then crashed. It only failed sometimes, depending on timing, which is the worst kind of bug. The fix is to give the helper a name, so it lives as long as needed.

What it doesn’t mean

Zero survivors means the tests notice every small change the tool knows how to make. It doesn’t mean the code is bug-free.
Two planted bugs made the tests hang instead of fail (an endless loop). Those count as caught, because the test suite stops on a time limit, but they’re slower to notice.

Up next

The first milestone is done. Next, the monitor learns to check on repeat, every minute and without overlaps. It also gets its first dashboard, so you can watch it work in a browser, and a real Node.js app to watch.

Checkpoint 0.6: a command you can actually run

404human — Wed, 14 Oct 2026 13:00:00 +0000

Until now, the only way to use my monitor was to write a test.

That’s not a product. So this checkpoint gives it a front door.

What I built

One command:

service-monitor check https://example.com/health

It checks the address once and tells you what happened, in plain words. A few options:

--expect 200,204: which status codes count as fine.
--timeout-ms 5000: how long to wait, at most.
--contains "ready": only pass if the page says this.
--dev-allow-port 9000: allow exactly 127.0.0.1 on that port, for testing on my laptop.

And it ends with an exit code, a number that scripts read to know what happened:

0: it’s fine.
1: the site is broken.
2: you typed something wrong.
3: “I couldn’t look”, for example a blocked address.

That last one matters. A script can tell “the site is down” apart from “the monitor couldn’t check”, and those need very different reactions at 3 a.m.

Here it is, for real

Against my practice server:

$ service-monitor check http://127.0.0.1:9124/healthy --dev-allow-port 9124
PASS 200 OK
  time     0.4 ms, until the headers arrived

$ service-monitor check http://127.0.0.1:9124/error --dev-allow-port 9124
FAIL expected {200}, got 500 Internal Server Error

$ service-monitor check http://127.0.0.1:9124/hang --timeout-ms 1000 --dev-allow-port 9124
FAIL no reply before the deadline
  time     1001.1 ms, until it gave up

$ service-monitor check http://169.254.169.254/latest/meta-data/
GAP  169.254.169.254 is in 169.254.0.0/16 (link-local, incl. cloud metadata), which monitors may not reach

The last one is the cloud’s password address from Checkpoint 0.3. Refused before a single byte is sent.

If you type something wrong, it tells you which option, in the words you typed:

service-monitor: that check can't run:
  --timeout-ms: must be between 1000 and 15000 (got 20)

Does it actually work?

6 new tests run the real program as a separate process, against the practice server, and check its exit code and output, exactly as a script would see them. They cover:

pass,
broken,
hanging,
blocked,
bad input,
and complete nonsense.

73 tests in the project now. 0 warnings.

Where I messed up

Using the command myself, instead of just testing it, found two rough edges:

The same hang got two different descriptions. Sometimes it said “timed out”, sometimes “no reply before the deadline”. Two timers were racing (mine, and a backup in the web library), and whichever won picked the wording. The answer was always right; only the words changed. Now every failure is described from one list of plain-English names, so the wording can’t depend on a race.
“Connection refused” said it waited “until the headers arrived”. No headers ever arrived. It now says “until it gave up”.

Tests check that the answer is right. Actually using the thing checks whether it makes sense to a human.

What it doesn’t do yet

It checks once. There’s no schedule yet (that’s the next milestone).
Certificate problems on secure sites still show up as “couldn’t connect” instead of their own category.

Up next

Before I call this first milestone done, I want the testing to be aggressive. So Checkpoint 0.7 lets tools attack the code instead of me: automatic mutation testing, thousands of random inputs, and hundreds of checks at once.

Checkpoint 0.5: the first real check

404human — Wed, 14 Oct 2026 12:50:00 +0000

For four checkpoints, my monitor couldn’t actually monitor anything.

Now it can.

What I built

The heart of the whole project: one check. Give it a web address and some settings, and it comes back with one of three answers:

it’s fine,
it’s broken, and how: an error code, a timeout, a refused connection, a reply that’s too big…
or “I couldn’t look”. That’s a gap, for example when the address isn’t allowed. A gap is never counted as fine.

Every check follows the same strict rules: one try, one connection, one time limit for everything. It doesn’t follow redirects, go through proxies, reuse old connections, or unzip compressed replies.

How it works (the short version)

One time limit covers everything, from looking up the name to the very last byte. Think of a kitchen timer you start before you even pick up the phone.
The address book has a bouncer. When the monitor looks up a name like example.com, every address that comes back must be on the public internet. If even one is private, the whole answer is refused. Mixing a real address with a private one is a known trick for sneaking past checks like this.
A broken website isn’t an error in my code. It’s the answer. The check always reports what it saw, so a crash in the monitor can never be mistaken for a broken website, or the other way round.

Does it actually work?

First, the four cases from the plan, against the practice server from 0.4, over real connections:

Fine: pass (200).
Broken: “500”, reported as a failure.
Hanging: gave up when time ran out, within the 1-second limit plus a quarter-second of slack.
Server off: “connection refused”, in under a second.

Each one used exactly one connection.

Then the security side:

An address that isn’t allowed is refused before anything is sent. The practice server saw 0 connections.
A fake address book that points a name at the cloud’s password address (169.254.169.254): refused. Same for a mix of a real address and a private one.
A name that can’t be looked up: reported as a DNS failure, with the reason.

67 tests in the whole project, all passing. 0 warnings.

Where I messed up

Two of my own mistakes, both caught in this checkpoint.

1. A leaky exception. Back in 0.3, I let names use the special “testing on my laptop” port, assuming the address check would keep them honest. But the address check never sees the port number, which gets added later. So in testing mode, a name could have reached any port on my laptop.

Fixed: only the literal address 127.0.0.1 gets the exception now. Names always follow the public rules.

2. A test that couldn’t fail. I broke the “never reuse a connection” setting on purpose, and every test still passed.

Why? My practice server always says “close the connection after this”, so reuse could never happen in a test, and the setting was invisible. I added a practice address that keeps connections open, plus a test that runs two checks and expects two connections. Now, breaking the setting fails the test: 1 connection instead of 2.

That’s the whole point of breaking things on purpose. A passing test only means something if it can fail.

(Plus a small Rust one: the compiler refused a line that walks through a chain of errors. One extra character, &, fixed it, and it’s now in my Rust notes so it doesn’t happen twice.)

What it doesn’t do yet

Problems with a site’s security certificate show up as “couldn’t connect”, instead of getting their own category. That needs its own test server, in M1.
With a time limit under 2 seconds, a slow start to the connection is reported as “no reply in time” rather than “slow to connect”.
It does one check, once. There’s no schedule yet.

Up next

Checkpoint 0.6: a command I can actually run. service-monitor check <url> will print exactly what the check saw. Then the first milestone is done.

Checkpoint 0.4: a test server that fails on purpose

404human — Wed, 14 Oct 2026 12:40:00 +0000

You can’t test a smoke alarm without smoke.

To test a monitor, I need something to break: on demand, in exactly the same way, every time.

Fake responses won’t do. The nastiest monitoring bugs are about timing: a server that picks up the phone and then says nothing, or one that promises a long answer and stops after a few words. You only see those over a real connection.

What I built

A small web server whose only job is to misbehave. Each address is a different kind of bad day:

/healthy: everything’s fine.
/error: “500, something broke”.
/status/404 (or any other code): a valid answer, just not the one you expected.
/hang: listens to your request, then never answers. Ever.
/slow-headers/3000: answers, eventually (3 seconds later).
/stall-body: promises 1,000 bytes, sends 16, then goes quiet.
/big-body: 1 MB, sixteen times more than the monitor will ever read.
/close-early: hangs up without a word.

There’s also an address where nobody’s listening (a server that’s down), a counter of how many connections came in, and a small app to play with it by hand.

It lives in its own separate project, right next to the monitor. That way, test code can never sneak into the real monitor.

How it works (the short version)

It speaks just enough HTTP, written by hand. A web framework would helpfully tidy up and time out for me, which is exactly what I don’t want. Here, /hang really hangs.

The interesting part is switching it off. A test that leaves a hanging connection behind could mess up the next test.

In Rust, every value has one owner, and when the owner is done, the value gets cleaned up. So I set it up like a chain:

the test owns the practice server,
the server owns its background worker,
the worker owns a container holding every open connection.

When a test finishes, the server gets thrown away. That stops the worker, which empties the container, which closes every connection. One thing goes out of scope, and everything closes. No “please shut down” messages, no bookkeeping.

(I checked the library’s source code to be sure: throwing away that container really does stop everything inside it.)

Does it actually work?

12 tests talk to it over a real network connection and check both what comes back and when:

/hang sends nothing for 300 ms, and keeps the line open.
/stall-body promises 1,000 bytes and sends exactly 16.
/big-body delivers exactly 1,048,576 bytes.
Throwing the server away ends a connection that was stuck on /hang.

And here it is by hand, with curl:

/healthy       → 200
/error         → 500
/status/404    → 404
/close-early   → empty reply (curl exit 52)
/hang          → still nothing after 1 s (curl exit 28)
/big-body      → 1048576 bytes

Then I broke it on purpose, three times:

Made /hang answer: 2 tests failed.
Made /stall-body honest: 1 test failed.
Let connections run on their own, outside the container: the “throwing it away closes everything” test failed. That’s the proof that the ownership chain is doing the work.

Where I messed up

My first “break it on purpose” attempt didn’t even compile. A broken version that doesn’t compile proves nothing about the tests. So I fixed my broken version until it compiled, and then the test caught it.
I’d told the code reviewer that unwrap() is fine in tests. Turns out that only covers the test functions themselves, not the little helper functions next to them. Nine warnings. One clearly commented line fixed it.

What it doesn’t do yet

It answers every kind of request the same way. There’s no special handling for HEAD requests.
“An address where nobody’s listening” is found by grabbing a free port and letting it go. Another program could grab it in between, which is unlikely on my laptop.
The “nothing happens” tests wait 300 ms. On a very busy machine that could get flaky. I’ll keep an eye on it.

Up next

Checkpoint 0.5: the monitor’s actual check. One request against this practice server, for the four basic cases: fine, broken, hanging, and down. Each one must be judged correctly within its time limit, using exactly one connection.

Checkpoint 0.3: 127.0.0.1 has a lot of disguises

404human — Wed, 14 Oct 2026 12:30:00 +0000

Every one of these means “this computer”:

http://127.0.0.1/
http://2130706433/
http://0x7f.1/
http://127.1/
http://0177.0.0.1/
http://[::ffff:127.0.0.1]/
http://[64:ff9b::7f00:1]/

A monitor visits whatever address you give it. That makes it a bit like a delivery driver who’ll go to any address on the label. Handy, until someone writes the address of your private office on it.

The classic target is 169.254.169.254, a special address inside cloud servers that hands out passwords to anything asking from the inside. This trick even has a name: SSRF.

This checkpoint is the first lock on that door.

What I built

Two checks:

Is the address safe to write down? Only http and https. No username or password in it. Nothing that looks like a secret, such as ?token=. At most 2,048 characters. Secrets belong in a separate header, not in an address that ends up in logs.
Is it safe to go there? Only addresses on the public internet, on the normal web ports (80 and 443).

Good news: the library that reads web addresses turns all those disguises into plain 127.0.0.1 before my check even looks. One address to judge, not seven.

Not from memory

“Which addresses are private?” sounds like common knowledge. It’s a surprisingly long list.

So I took it straight from the official registry (IANA) the same day: 26 special IPv4 ranges and 25 IPv6 ranges, each marked whether it’s reachable from the public internet. A test checks every line of my list against the registry’s own text, so a typo breaks the build.

The sneaky IPv6 ones

For IPv6, I flipped it around. Instead of listing what’s banned, only the normal public range is allowed. Anything new or unusual is refused by default.
64:ff9b::7f00:1 is 127.0.0.1 wearing an IPv6 costume. Some networks use these addresses legitimately to reach older ones, so I don’t ban them all. I look at the address inside and judge that.
Rust’s own check says ::ffff:127.0.0.1 is not “this computer” until you convert it. So I convert first, every time.

The one exception

For testing on my laptop, there’s a developer mode that allows exactly 127.0.0.1 on one chosen port. Not ::1, not 127.0.0.2. And only the command line can switch it on, never the web API.

Does it actually work?

I broke the three defences an attacker would go after, one at a time:

Skipped the check for addresses written as numbers: 3 tests failed.
Removed the step that converts the IPv6 costume: 1 test failed. The interesting part: the address was still blocked, by a backup rule. Only the test that checks the exact reason noticed. Two layers, and the tests can see both.
Shrank the cloud-password range: 5 tests failed.

Then I put the original code back and compared it byte for byte.

Where I messed up: I broke this website

When I first published this build log, every deploy said “success”, and every post on this site turned into “404 not found”.

The old hosting ran two competing deploys on every update, and the wrong one happened to finish last.

Lesson: a green checkmark isn’t a working site. Open the page.

The fix: this site now lives on Cloudflare Pages. I build it on my own machine and upload exactly that, so there’s one deploy, and it’s the one I checked.

What it doesn’t do yet

This is the first layer. Names like example.com get turned into addresses later, and every one of those addresses gets checked too (Checkpoint 0.5). A name that points to a mix of public and private addresses will be refused outright, because that’s a known trick.
An app checking addresses isn’t a firewall. The plan adds network rules as a second layer later.

Checkpoint 0.2: making bad settings impossible

404human — Wed, 14 Oct 2026 12:20:00 +0000

Most settings bugs come from the same place: a value gets checked in one spot, then copied, changed or trusted somewhere else.

Think of a bouncer at a club. Checking IDs at every table inside is slow, and someone always gets missed. Check once at the door, and everyone inside is fine.

What I built

A front door for the monitor’s settings.

Settings arrive as raw settings: whatever you typed, trusted by nobody. There’s exactly one way to turn them into validated settings, and that’s passing every rule:

check every 30 seconds to 1 hour,
give up on a slow site after 1 to 15 seconds,
a name of 1 to 80 characters,
and a few more.

Once settings are through the door:

They can’t change. Editing creates a new version instead.
They can’t be copied by accident. They’ll be shared, not duplicated.

And the door reports every problem at once, like a form that highlights all the wrong fields instead of making you fix them one at a time.

How Rust helps (the short version)

In Rust, every value has exactly one owner. When the raw settings walk through the door, their text moves into the validated version instead of being copied. Nothing is duplicated, and nothing is left behind to go stale.

The whole door is one line wherever settings come in:

let config = ValidatedConfig::try_from(raw)?;

That reads as: “try to turn the raw settings into validated ones, and stop here if any rule fails.”

Small things, measured

I like knowing exactly how much memory things take, so these sizes are checked by tests. If a later change makes them bigger, a test fails:

8 bytes for an “ID, or nothing”. Normally it would be 16. But ID numbers start at 1, so Rust uses 0 to mean “nothing” for free.
16 bytes of bookkeeping for a name, instead of the usual 24. Text that never grows doesn’t need room to grow.
56 bytes for the list of “good” HTTP status codes, and checking a code is a single step.

Also, limits count characters, not bytes. “é” counts as one character, even though it takes two bytes.

A rule that can never fire

The plan says the timeout must be at most half the check interval. But with the plan’s own limits (interval at least 30 s, timeout at most 15 s), that’s always true.

I kept the check anyway and tested it directly. If someone changes a limit later, it won’t quietly break.

Does it actually work?

All 20 tests passed on the first run. That proves less than it sounds: a test that would also pass on broken code is useless.

So I broke two rules on purpose:

Turned off “a HEAD request can’t check the page’s content”: 1 test failed.
Turned off “no hidden control characters in names”: 2 tests failed.

Then I put the code back exactly as it was. Good: the tests notice.

Where I messed up

I checked the code’s formatting with a one-line command that printed “clean”, while the formatter was complaining right above it.

The reason: when you chain commands with a pipe (|), the shell reports how the last command went. The last one was perfectly happy.

Lesson: run checks on their own, especially the ones you trust.

What it doesn’t do yet

The web address is only checked for length so far. Making sure it can’t point at your private network comes next.

Checkpoint 0.1: rules first, code second

404human — Wed, 14 Oct 2026 12:10:00 +0000

Nobody gets excited about an empty project.

But it’s the cheapest moment to decide which mistakes you’ll never allow, because there’s no old code to clean up yet.

How I’m building this

Every milestone is cut into small checkpoints. A checkpoint only counts when:

its tests pass,
Rust’s strict code reviewer (a tool called clippy) has zero complaints,
it’s saved in git,
and it’s written up here, mistakes included.

What I built

An empty Rust project, split into nine parts: checking, scheduling, storage, alerts and so on. Each part has one sentence saying what it does, and what it must never touch.
Two libraries, pinned to exact versions: tokio, the engine that lets a program do many things at once, and reqwest, for making web requests.

The trick: leaving a feature out

On day 0 I found that the web library retries some failed requests by itself. There’s a setting to turn that off, and I’ll use it.

But settings get forgotten. So I also left out the part of the library where those retries live (HTTP/2, which a simple health check doesn’t need).

I checked the build afterwards: that code isn’t in there at all. It can’t misbehave if it doesn’t exist.

Rules the compiler now enforces

No unsafe code. Rust’s safety checks stay switched on everywhere in my code.
No unwrap() outside tests. That’s Rust’s “assume this worked, and crash if it didn’t”. A monitor should report a problem, not fall over.
A name-clash check. From my AI Rust exams: give a helper the same name as a built-in method, and Rust can quietly call the wrong one.

And one more choice: if a single check crashes, the monitor writes it down as a gap instead of taking the whole thing down with it.

Numbers

16 seconds to build the web library and its encryption code, using just the standard compiler on this laptop.
0 warnings. 0 tests, because there’s nothing to test yet.

What it doesn’t do yet

Nothing runs. This checkpoint is the foundation, and nothing more.

I read the code before writing the code. It found 4 traps.

404human — Wed, 14 Oct 2026 12:00:00 +0000

Have you ever had a website go down and found out from an angry customer instead of your own alerts?

That’s the job of an uptime monitor. It checks your site every minute and tells you when something breaks. Simple idea.

But a monitor has one job: telling the truth. And the dangerous bugs in a monitor aren’t the ones that crash it. They’re the quiet ones: a check that secretly retries until it passes, or a gap in the data that shows up as green.

I’m building one anyway. It’s called Service Monitor, it’s written in Rust, and I’m building it with Claude Code, in the open. This is the first entry in the build log.

The rule: don’t trust memory

Programmers (and AIs) often write code from memory of how a library works. But libraries change, and memory is optimistic.

So before writing any monitor code, Claude read the actual source code of the exact library versions we’d use, and ran tiny experiments to confirm what it found.

Four things surprised us.

Trap 1: an IP address skips the safety check

A monitor visits whatever web address you give it. That’s handy, and also a little risky. Someone could point it at things inside your private network, like the special address on cloud servers that hands out passwords to anything that asks from inside. (Security folks call this SSRF.)

The usual guard checks every address when a name like example.com gets looked up. But when the address is already a number, like http://[::ffff:127.0.0.1]/, there’s nothing to look up, so the guard never runs.

In our test, that address reached a private test server, and the guard was called 0 times.

Fix: check addresses written as numbers directly, before anything goes out.

Trap 2: hidden retries

My rule is “one check, one try”. If a service is flaky, I want to see it. If it’s struggling, I don’t want to pile on.

It turns out the web library quietly retries some requests: up to 2 extra attempts for certain errors. And a layer underneath it retries too, with no setting to turn it off.

Fix: switch off the first, avoid the conditions that trigger the second, and leave out the feature they live in.

Trap 3: the clock that sleeps with your laptop

Rust’s usual stopwatch stops counting while a laptop is asleep. On my machine it said 17,240 seconds since startup, while the clock that also counts sleep said 62,115. That’s about 12.5 hours of sleep it never noticed.

For a monitor, that means checks missed while the laptop slept would simply vanish, instead of showing up as “we weren’t watching”.

Fix: work out every check time from the real clock, and record missed checks as gaps.

Trap 4: the database that doesn’t mention typos

SQLite silently ignores a setting it doesn’t recognise. Mistype the one that makes writes crash-safe, and you get no error, just a less safe database.

Fix: read every setting back at startup, and refuse to start if one doesn’t match.

Bonus: checking the maths

The plan was full of numbers, so we checked every one with a script. All correct.

One limit nobody had written down: with the default settings, only 12 monitors can point at the same server before the monitor starts saying no. That’s the first wall a real user would hit.

What it doesn’t mean

None of these are bugs in those libraries. They’re sensible choices for general use, just wrong for a monitor.
These facts are for these exact versions, and they get re-checked on every upgrade.
At this point there was zero monitor code. This was all reading.

The takeaway

The libraries were fine. My assumptions weren’t.

Reading first turned four “it probably works like this” into four tests to write before the code that needs them.

I made an AI read the entire Rust Book. Then I tried to make it fail.

404human — Wed, 14 Oct 2026 00:00:00 +0000

AI can write Rust that looks perfect.

That’s exactly what worries me.

Rust is the language where “looks right” and “is right” can be miles apart. Ownership, lifetimes, error handling, crates that change their API between versions. That’s where AI-written Rust usually falls apart. You get code that’s confident, clean and wrong.

So I tried something. I asked Claude (in Claude Code) to do what a serious human learner would do: read the whole official Rust Book, take notes, sit exams, grade itself, and fix whatever it got wrong.

Then I asked the question that actually matters: does any of it carry over to real projects?

Step 1: Read the whole book

Not a summary. The full official Rust Book, all 23 parts, with examples checked against a real compiler (Rust 1.99).

Everything went into one main file: the rules, chapter notes, memory measurements, exam scores, and a log of every mistake.

Step 2: The theory exam

Separate AI “examiner” agents wrote 138 questions across 9 exams, with hidden answer keys checked against the compiler.

Score: 136.5 / 138 (98.9%).

The 1.5 points it lost were almost funny:

It miscounted a line number because it skipped a blank line.
It copied a line position from the book’s example instead of the code in front of it.
It quoted an error code that Rust has since retired.

Before grading, it predicted its own score: 133. It came in higher. I’ll take a humble AI over an overconfident one.

Step 3: Write code with no compiler

This is where most AI coding falls over.

36 coding tasks. 261 hidden tests it couldn’t see. And one hard rule: no compiler. No running the code, no checking, no “let me try that again”. Write it once and hand it in, like a whiteboard interview.

The topics were the hard ones: ownership and lifetimes, error handling across a whole app, threads, async, unsafe code, and real crates like tokio, serde, clap and nom.

Result: 254 / 261 tests passed on the first try (97.3%). 35 of 36 tasks compiled.

All 7 lost tests came from the one task that didn’t compile. The linter also flagged 4 style issues. In everything that did compile, there wasn’t a single logic error. After fixes: 261 / 261, zero warnings.

But that one failure turned out to be the best part of the whole experiment.

The bug that taught me something

Claude gave a helper method a name that Rust’s built-in Iterator already uses. Rust picked the built-in one. In the exam that meant a compile error and a quick fix.

But I’d asked it to understand why a mistake matters, not just patch it. So it built an experiment to see what the same mistake looks like in a real project.

Here’s a simple job queue with its own count method. The queue also implements Iterator, so it can hand out jobs one at a time:

impl Jobs {
    // "How many jobs are waiting?"
    fn count(&self) -> usize {
        self.items.len()
    }

    fn summary(&self) -> String {
        format!("{} jobs waiting", self.count()) // calls the count above
    }

    fn report(&mut self) -> usize {
        self.count() // same words... calls Iterator::count instead
    }
}

Run it:

3 jobs waiting
report() says 3
0 jobs waiting

report() returned the right number. And it quietly emptied the whole queue to get it.

Same two words, self.count(), two different methods. Which one runs depends on whether you’re inside a &self method or a &mut self one.

It compiles with zero warnings. The default linter says nothing. Even the extra-strict lint groups miss it. A test that only checks the returned number passes.

That’s silent data loss. The kind of bug that makes it to production.

It’s like having two people named Sam in the same room. You call “Sam”, the wrong one answers, and nobody notices, because someone did answer.

The catch is one opt-in lint, clippy::same_name_method, which flags it straight away. It’s now on Claude’s checklist for every Rust project.

Step 4: “Prove it wasn’t memorized”

Fixing a mistake is easy. Not repeating it in a different shape is the real test.

So a fresh examiner read the full list of past mistakes and built 8 brand-new tasks, each hiding one of those traps in disguise:

the same name clash, now hiding behind a type that acts like a string
text positions counted in three different units (bytes, characters and UTF-16)
callbacks that could deadlock while a lock is held
code that had to build on an older Rust version, where the “modern” suggestion doesn’t exist yet
a newer version of a real crate, whose API differs from the older version also installed

For each task the examiner also wrote a decoy: the plausible, confident answer an AI typically writes. The hidden tests had to catch it.

The decoys passed between 0 and 4 tests per task. Some deadlocked. Some didn’t even compile.

Claude, first try, no compiler: 59 / 59. Zero linter warnings.

Plot twist: the examiner caught Claude’s notes being wrong

While building that test, the examiner found a misleading line in Claude’s own notes about how Rust 2024 handles locks inside if let.

The note was technically true, but it left out the part that matters: inside the if let block, the lock is still held. Someone following that note could write code that freezes.

Claude didn’t argue. It tested it on the compiler, confirmed the examiner was right, and fixed it in all three places it appeared.

Honestly, this is the part I value most. It caught a mistake in its own knowledge, not just in its code.

What carries over to real projects

The scores aren’t the point. The point is what happens next time.

Every lesson is written down three ways: what went wrong, why it matters on a real project, and how to prevent it. Each one is backed by an experiment that proves it.

And there’s now a “definition of done” that loads in every Rust project I open with Claude Code:

tests pass
the linter runs with zero warnings, including the name-clash check
the project’s minimum Rust version is checked before using newer features
crate code is read from the exact installed version, never from memory
no lock or borrow is held while calling someone else’s code

The honest part

This isn’t proof that AI writes perfect Rust now.

The examiners were AI agents, not human Rust experts. Everything was checked against the real compiler, but they’re still AI.
These were library-sized tasks, not a huge production codebase. Real projects will throw up new problems.
It still made mistakes. That’s the whole point. What changed is what happens after one.

The takeaway

What separates a good developer from an average one isn’t never making mistakes. It’s not making the same mistake twice, and knowing why it was a mistake in the first place.

Turns out that’s a pretty good bar for AI too.