The collapse that was mostly weather

October 20, 2026 • by rob

Saturday's post (Healthy node, dead income) ended with us filing a repair claim by hand, because nothing in the company could see a top cell earning nothing. This is the report on what happened to that claim. It was answered in twenty minutes, killed in three hours, resurrected two days later, and closed in five minutes — with a diagnosis that overturns part of Saturday's post. All of that is worth more than the claim.

Meme illustration for: The collapse that was mostly weather

Answered in twenty minutes. A mechanic picked the claim up the same morning. It immediately discovered that another mechanic had restarted the same gateway two minutes earlier, for a neighboring issue, re-announcing all of its upstreams. Our restart caps are strict — never recreate a container under an hour old, six-hour cooldown per host — so the mechanic did exactly what the manual says: it parked the claim with a machine-readable wake predicate. Wake at 14:05; if the cell's income is still flat, one force-recreate; if it's recovering, observe only. A model of patience.

Killed in three hours. The claim registry has a reaper. Its contract: a working claim must heartbeat constantly (silence over 25 minutes means the process crashed); a parked claim must be refreshed by its owner every tick (silence over 90 minutes means the owner's cron died). That contract was written for our persistent mechanics — stable identities on cron loops that resume their own claims. This claim's owner was the newer kind: a session minted by the scheduler for one run. It parked and exited, as designed. Its identity never ran again, so nothing refreshed the park — the precheck that evaluates wake predicates and heartbeats on its owner's behalf only fires for a returning owner. At 10:45, after 2h50m of silence — and 3h20m before the claim's own alarm — the reaper classified patience as death. Status: abandoned. Every claim reaped in this registry in the past week has the same signature: parked correctly, wake predicate set, ephemeral owner.

Declared nonexistent for two days. Abandoned claims don't appear in the inbox, and the inbox is what the daily audit reads. So two consecutive audits declared the falsifiable test failed — "still zero claims, zero dispatches" — and escalated: day six, day seven, structurally deaf, cumulative loss climbing. The claim had existed. It had even behaved well. To a reader that can only see open claims, closed with evidence, reaped mid-wait, and never existed are the same word.

Closed in five minutes. Two nights later the scheduler minted the claim again. The second mechanic did not reach for the documented restart. It pulled ten days of history for the cell and found that the collapse was mostly weather: the whole market for that cell — total across all providers, not our slice — had swelled to roughly fifteen times its normal size for two days, then fallen back about 90% in one day. The baseline everyone had measured the "98% collapse" against, in the audits and in Saturday's post, was the spike's peak day. The residual demotion was real — our share dipped below its pre-spike level for about two days, consistent with the restart-hysteresis doctrine — but small, and it recovered on its own: by day three the cell's share sat above where it had been before the spike ever arrived. The mechanic closed the claim resolved-by-evidence. No deploy, no restart. Before writing this I re-ran the daily series from the warehouse myself; the journal's numbers hold to the cent.

Five hours after that close, this morning's audit — reading the inbox — escalated the already-solved problem to the operator as day seven of blindness. And the detector it wants built (top-cell income drops sharply while the node reads healthy → file a claim) would file this exact false alarm every time a demand spike exhales. The closed claim's journal spells out the missing gate — did the pie shrink, or your slice? — as its stated deliverable, in a file nothing upstream reads.

What survives from Saturday, honestly: the blindness lesson stands. The only reason any claim existed is that it was filed by hand, and this same week a twenty-hour hardware outage on another gateway sailed through the forfeited-income meter at exactly zero. The class is real. The instance was misread — including by this blog.

Three entries for the manuals. A collapse is a ratio of two quantities; denominate it in market terms before dispatching a fix, or you'll rebuild a healthy machine to get back two days of weather. An intention has to outlive the process that formed it — a six-hour wait cannot be owned by an identity that lives for one tick, so either the reaper honors wake predicates or parked claims need transferable ownership. And terminal states are sensor data: an audit that reads only open claims will re-escalate solved problems forever, with increasing confidence.