Fail Closed, Capture Before You Act

When software only showed people information, a wrong permission check was embarrassing. When software takes actions, like moving money, changing access or sending data out of the building, a wrong check is an incident. And now that agents are taking actions too, at a speed no human reviewer can match, the cost of a sloppy "is this allowed?" goes up fast.

I've spent the last stretch building a governance runtime: a layer that sits in the path of consequential actions and answers one question, "is this action, by this actor, right now, within the rules this organization has adopted?" Two rules ended up at the center of it. Everything else is built around them.

1. Fail closed. If the system can't reach a clear yes, the answer is no.
2. Capture before you act. The record of a decision is written before the action happens, or the action doesn't happen.

They sound obvious. They're surprisingly easy to break.

Four answers, not two#

Before the rules, a word about the answers themselves. Most permission systems have two: allow or deny. Real organizations have more than that. The runtime returns one of four verdicts:

• Allow. Inside the rules and the actor's authority. Proceed, and a receipt is written.
• Attest. Allowed, but the actor has to formally sign for it. They proceed on the record.
• Escalate. Outside the actor's authority, but not forbidden. The action blocks and goes to a named person who can decide.
• Deny. Outside the rules entirely. Blocked.

The middle two matter most. Without escalate, organizations drift into one of two failures: everything gets approved because people are told to move fast, or nothing ships because nobody can evaluate it. Without attest, there's no way to let a trusted person proceed while still holding them accountable.

At the API edge, the verdict can be mapped to a simple true or false for systems that need it. The full verdict always travels alongside, so nothing downstream can quietly lose the difference between "allowed" and "allowed because someone signed for it".

Rule one: fail closed#

Fail closed means every path that can't produce a clear answer produces a no. In practice that covers more paths than you'd think:

• The decision service is unreachable. Deny.
• The rule set for this organization can't be resolved. Deny.
• The decision logic throws an error. Deny.
• The actor has no capabilities at all. Deny.
• An escalation has no routing rule for its risk level. Notify an admin, and deny on timeout.
• An escalation times out without a decision. Deny.

That last one is the one people push back on. "What if the reviewer was just busy?" Then the action waits, or it's denied and retried. What it can't do is quietly go through because nobody answered. An escalation that times out into an allow isn't an escalation.

The cost is real. A fail-closed system can stop a customer's work when it's down or misconfigured. That makes uptime part of the product's trust story, not an operations afterthought. I accept that cost deliberately. A governance layer that fails open is only governance on good days.

Deny with the path#

Fail closed doesn't have to mean hostile. A bare "no" is a product failure. It makes people feel watched and makes the rules feel arbitrary.

The design goal for a denial is "deny, with the path": show which rule blocked the action, who owns that rule and how to ask for an exception. The person who got stopped should leave knowing what to do next.

Rule two: capture before you act#

The second rule is about order. On every governed action, the sequence is:

receive request
write decision event to the ledger    <- before anything happens
evaluate against the active rules
if allow or attest:
  perform the action
  write the receipt
else:
  block, and write why

If the ledger write fails, the action doesn't happen. There's no "we'll log it after". Logging after is how you end up with actions that happened and records that didn't, which is exactly the gap an auditor will find.

Each organization's ledger is append-only, and each entry includes a hash of the one before it. Change or remove any entry and the chain no longer adds up, for anyone who checks. That's what lets a public verification page confirm a receipt without anyone logging in and without trusting the company running it.

The verification page is deliberately narrow. It confirms that a decision with a given verdict was recorded, in order, at a given time. It doesn't reveal who acted, what the action concerned or anything else about the organization. Proof and privacy pull against each other, so the design discloses the minimum that still counts as proof.

Split decision from enforcement#

The runtime follows a standard split that makes both rules testable:

• The decision point is a pure function. Given the actor, the action, the resource, the context and the organization's active rules, it returns a verdict. No network calls, no database writes. That makes it easy to test exhaustively, including all the failure paths that should produce a deny.
• The enforcement point sits in the action path. It writes the evidence, calls the decision point and only then lets the action proceed.

Keeping the decision logic pure means I can prove "an error becomes a deny" with a unit test. Keeping enforcement in one place means there's exactly one spot where "capture before act" has to be right.

The UI has to follow the same rules#

One thing that surprised me was how much these rules shaped the interface.

No optimistic success on governed actions. Most modern apps show success immediately and reconcile later. That's great for liking a post. It's wrong for anything governed, because it shows the person a "yes" before the verdict exists. Governed actions wait for the real verdict and the receipt. It feels a beat slower. That's the correct feeling.

Hiding a button isn't a control. Navigation filtering is a nicety. The real checks happen on the server, on every action, every time. I check permissions in three independent layers: what routes are reachable, what shows up in navigation and what the server allows. Each one can fail without the others failing.

I learned the second lesson the hard way. The product's documents all said the person who submits a rule change can't approve it. The interface hid the button. The server didn't check. I wrote about finding that in Write Down Why, Not Just What. The fix was a server-side comparison, and it's now one of the tests that guards the rule.

Govern yourself first#

A governance product with an ungoverned back door is a contradiction. Platform staff can reach any customer's tenant to help them. That access is itself a risk.

So the product's own consequential actions go through the same enforcement point as customers' actions: creating API keys, changing roles, activating rule sets and staff access to a tenant. They show up in the decision log. They can require a fresh sign-in for high-impact steps, like turning off a safety setting. They produce receipts.

Building that surfaced a real bug, too. Staff actions inside a customer's tenant were being recorded under the staff member's own organization instead of the tenant they acted in. The audit trail for "everything staff did inside this customer's account" came back empty. The fix was to attribute the record to the tenant in the request path, not the acting session.

Why this matters for agents#

Everything above applies to people. It matters even more for agents.

An agent can attempt hundreds of actions in the time a person attempts one. It doesn't get tired, and it doesn't hesitate. If the check in front of it fails open, it fails open hundreds of times. If the record is written after the action, a crash in between leaves no trace of what happened.

The answer isn't to slow agents down to human speed. It's to put a fast, fail-closed decision in front of every consequential action, with a receipt written first. Low-risk actions pass in milliseconds. Hard calls go to a named person. Everything leaves evidence.

It's the same thinking I apply to my own agent loops: the agent can move quickly because the gates around it don't move at all.

The takeaway#

Fail closed and capture before you act are simple to say and easy to erode, one convenient exception at a time. Write them down as rules, not preferences. Put the decision logic in a pure function you can test. Put enforcement in one place. Make the UI wait for the real answer. And apply the same rules to yourself before asking anyone else to trust them.