How the False Floors method was built and tested

The registers themselves live on the six register pages. The hub explains what they are for.

Every register count on this page is derived by a script that runs against the registers, and the script fails the build when a page disagrees with them. The validation figures below come from the passes that produced them, each named where it is quoted.

The short version

The registers grew out of one real production build. A non-programmer directed coding agents to build a compliance product, and wrote down every failure as it happened.

That gave us rows with dates and receipts. It also gave us a bias: incident logs only record the failures loud enough to notice.

So the second half of the method walks each register's unit through its life and asks what breaks at every stage. That produces rows nobody has hit yet.

Then one rule decides what gets counted. A row is only counted in a headline once real evidence reaches it. Rows named in advance stay published, stay labelled, and count for nothing until they earn it.

Today that gives 67 evidenced rows and 61 candidates, across 128 published rows.

How this framework was built

This project is a non-developer building production software with multiple coding agents, and writing down every point at which the agent's ignorance – or mine – produced a result that looked finished and was not.

An experienced engineer would not have generated most of these rows. They would have avoided most of these failures without noticing they were avoiding them.

Stashing before a destructive command. Checking what a restore point actually covers. Distrusting a CI check you have never seen fail. Those are reflexes, learned once and then invisible.

Reflexes do not get written down. What is not written down cannot be installed for anyone else. An agent has no reflexes at all, so every one of them has to become an explicit check or it does not happen.

A great deal of what protects a codebase is tacit. Ask a working developer to list the checks that keep them safe and you get the ones with names. The register process asks it the other way around: here is a failure that happened, what would have caught it? The answers come back as things nobody would have listed.

No single developer holds the whole set. Some rows are ordinary practice. Some are specialist: the branch ruleset, the credential path that bypasses it, the constraint that races a trigger. Some cross disciplines one person is unlikely to hold at once – database grants, git internals, CI, shell, credentials, backup cadence and release governance. An agent touches all of them in an afternoon. Writing the failures down in registers forces those disciplines into one list, where the gaps between them become visible.

The register is a ratchet, not a finished list. This was never meant to be published once. A failure arrives. It is classified. The earliest point that could have caught it is identified. A check is written there. The check is then run against the known-bad input to prove it catches it. Rows retire when a mechanism closes them, and new rows arrive faster than they retire.

Two models, on opposite sides of the desk

The build ran on Claude Code as the working platform, and Codex as the audit and adversarial seat – a second model given the artefacts and a brief to refute rather than agree. The seats were swapped at points, with Claude auditing Codex's output on the same terms.

On one 104-case set, the two seats assigned findings to the Execution and Capability register 28 times and five times respectively. Every other register fell within a few cases either way.

The two seats were working from two different definitions of what the Execution and Capability register covers. The split showed the register was malformed, not merely disputed. That is why Execution and Capability is still in calibration, and why its rows are excluded from every cross-register count on this page.

A single-model build returns agreement. Agreement teaches you nothing about your own blind spots.

The process it was run under

The registers were derived while running a real control architecture over the build. That architecture is modelled directly on how a standards body amends a standard.

That was the deliberate starting point: take how an international standard is governed – proposals, evidence, approvals, records – and replace the humans in the loop with agents.

The whole of it is aligned to a series of international standards on software production and AI management that define best practice for enterprise development: ISO/IEC/IEEE 12207, ISO/IEC 5338, ISO/IEC 42001, ISO/IEC 23894, NIST SSDF and NIST AI RMF. It replaces none of them, and it certifies nothing.

The control set has an immutable core and a precedence rule. A conflict on a sensitive matter fails closed, instead of resolving to whichever clause was read last.

Changes arrive as numbered amendments. Each carries an approval pack stating what changes, on what evidence, what was considered and rejected, and what remaining risk is accepted.

A Commissioner – a named human, in this case me – approves each amendment. Each approval covers a cryptographic fingerprint over the governed sources, not a description of the change, recomputed by a validator on every run. A pack whose files have drifted since approval fails the run by design.

It is deliberately heavier than a project this size warrants. It was run at that weight to find out which parts of standards-body machinery survive contact with a delivery process where the builder and the reviewer are the same agent, and which parts are theatre.

The parts that survived are the ones with a validator behind them.

The parts that decayed were hand-maintained prose. On one occasion that included the control plane's own record of its amendments, which silently skipped two while the fingerprint check beside it passed every time.

That result is why the registers grade by mechanism rather than by intention. Once a check runs, the check becomes the operative control, and the unchecked prose beside it decays.

Making the machinery invisible so the decisions stay visible

One result was not expected and has become a large part of the work.

Most of the effort in agent-assisted building is not the building. It is the git ceremony around the code, none of which is the job: worktrees, branches, commits, pushes, pull requests, rebases, merge conflicts and stale checkouts.

So a second body of the project is skills that take that ceremony off the desk. The session runs the git operations itself. It verifies its own claims about what shipped rather than reporting a merge badge. It cleans up its own worktrees, and refuses to call work shipped without checking the destination branch.

When the ceremony is invisible, business logic sits in front of you: what the thing should do, what it must never do, and who decides when the two conflict. That is the part a non-developer can hold. The code is a consequence of those decisions rather than a substitute for them.

Hiding the ceremony does not remove the failures it can cause. Several rows on the Recovery and Provenance registers are the failures that surface when the ceremony is hidden badly rather than hidden well. They were earned that way.

Where the rows come from

A register invites one fair suspicion. Were these rows written down because they happened, or generated to fill a table?

Both, and every register states which method produced it.

Inventory – rows read off real artefacts. The Truth register's 15 rows were inventoried from a live CI check suite: every claim the suite actually makes about the work. The seventh register, Execution and Capability, is built only from incidents and holds four rows.

Incident transcription – rows read off a record of things that happened. The Authority register's 23 rows came from a production security register. Every row is an incident that happened there, or a control gap recorded against it. This is why Authority has the highest evidenced share of the six.

Enumeration in advance – rows generated by walking a unit through its life. Instruction, Context, Recovery and Provenance were built this way. Instruction was additionally grounded in a field study of 20,574 real agent sessions. Each register takes its unit – a rule, a fact, a permission, a change – through every stage and asks two questions at each one. What breaks here? And would anything tell you?

That third method is failure mode and effects analysis, the same move aerospace has made since 1949. It is used here for the same reason: incident logs select for loud failures, and the worst rows on these pages are the quiet ones.

Rows from the first two methods have dates and receipts. An automated tool that pushed a rejected schema change to a live environment, through a credential path no branch protection covered. A CI guard that was blind from the day it was written. A backup whose snapshot cadence skipped straight over the loss it existed to catch.

All three are false floors in the strict sense. A control existed, was believed to be working, and was the reason nobody was watching.

What makes a row count

A row makes four claims: the failure class exists, it is distinct from its neighbours, it matters, and a named control is the strongest available answer. A count would make a fifth claim: that the list is complete. No count published here claims the space is mapped. The corpus evidence says the opposite: the enumerated lists are too small for the space.

Every row carries an evidence label, on the row. There are two.

Evidenced means at least one receipt exists. Three kinds of receipt count, and nothing else does:

Candidate means the row was named in advance and no receipt has arrived yet. It is still a real row: published, classed, graded, and labelled.

Headline counts count evidenced rows only. Today that is 67 evidenced and 61 candidates, across 128 published rows. Of the 67, 47 are evidenced by a first-party incident and 20 by corpus coding alone.

A candidate is promoted the day its first receipt arrives. The receipt is lodged at its source, never asserted on the row, and a script re-derives the label from the receipts on every run.

A candidate that a full review window passes over – no incident, no corpus match, no field case – is retired to a holding pen that sits one step below a published row.

We keep candidates published rather than deleting them for one reason. Deleting them would erase the track record. You could no longer see how often the framework's predictions come true.

This is close to how the mature failure taxonomies grow – ATT&CK, CWE and the data-driven OWASP lists all admit entries by observation. The one departure is that we keep the unconfirmed rows visible as labelled predictions, and publish the rate at which they come true.

How the registers have been tested

Four validation events have run against the rows: the calibration pass, the inter-rater runs, the adversarial Class-A audit and the retire-and-merge pass. Each names the source in the repository that produced it.

Two of the results:

Kappa 0.654 counts as substantial agreement on the Landis and Koch scale conventionally used for this statistic. It is the second-highest of six bands, and above the 0.6 mark that published coding studies commonly treat as reliable. Two independent readers, working from the documents alone, sorted most failures into the same registers.

The adversarial audit demoted more than half of the strongest claims. Of the 26 rows claiming a failure could be refused outright, 14 could not survive an attacker-minded re-read. Those rows now grade lower. That audit is the reason only 12 rows across the whole framework still claim prevention.

The prediction record

The one test unique to the enumerated half is whether a row written down before a failure was ever recorded later gets confirmed by a dated incident.

Prospective confirmation is the statistic filler cannot fake.

The rows were enumerated on 5–8 August 2026. At that point 113 of them stood as unconfirmed predictions: named failure modes with no recorded instance anywhere.

Within 11 days, 24 of those 113 had been confirmed by a first dated incident, and a 25th followed on 25 August. That is about one in five. Each confirmation traces to a dated entry in the project's own corrections register. An incident can confirm more than one row, so the 25 rows trace to 17 entries.

Confirmation is counted narrowly. It needs the date the failure happened, never the date it was written down. Corpus matches never count – those documents predate the framework, so a corpus match is validation, not prediction. The six rows later retired stay in the denominator, because removing them would flatter the rate with survivorship. And the Authority register's founding mappings are excluded from the foresight claim by construction: that register was transcribed from the security register that recorded its incidents, so its mappings confirm fidelity, not foresight.

The cleanest case: a deny rule on destructive shell commands was specified on 7 August as a designed control. On 18 August a git reset --hard destroyed a concurrent session's unrecoverable work. The control the row had specified was built and switched on the same day.

Two things about that rate:

The rows are narrow. Each names one mechanism against one unit, so an incident cannot be waved into whichever row is nearest.

And the mapping process refuses bad fits. Of the 262 independently coded findings, 150 were coded as matching no row. The project's own log records mappings that were proposed and rejected for resembling a row without instantiating it.

The confirmation rate is bounded by where it was measured. This is one operator, one machine, heavy daily agent use, and observers who hold the register. No control condition has been run. A decoy register of plausible-but-wrong rows would be the falsification test, and until something like that runs, the rate measures how often the named classes occur where the registers were derived. It does not measure how they generalise.

Enumeration in advance is kept because it keeps coming true. It proposes rows. It never sets a published total.

What we already know is wrong with this

Five limits.

The enumerated registers all come out about the same size. Four registers were enumerated by one process onto one page layout, and even after the retirement pass they sit at 22, 22, 22 and 24 rows. That closeness is a property of the generator, not the failure space. Lists enumerated by one process onto one layout come out a similar size. The bias is quarantined by the inclusion rule – candidates never reach a headline – rather than removed.

The two registers that look nothing like the others are the two that were not enumerated: Truth, inventoried from a live CI suite at 15 rows, and Execution and Capability, built only from incidents at four.

Truncation is the larger error, not padding. Of the 262 corpus findings, 150 demanded row detail no register carries. The enumerated lists are too small for the space, not too large.

The corpus is skewed. The calibration corpus is 26 code audits of one database product. It structurally under-supplies Recovery, Instruction and Context, and over-supplies Truth and Authority. A Recovery candidate with no corpus hit is not thereby suspect.

One operator, one harness. Built states are scoped to one repository. The incident stream is one person's practice on one machine. The registers claim grounding in that environment, not generality across environments.

Row-level discrimination is untested. Kappa 0.654 is a register-level number. Whether independent coders can reliably tell one row from another has not been measured. Nor has use by coders outside this project. Both are scheduled.

There is one more, and it is a historical artefact rather than a bias. Three registers once held exactly 23 rows because their diagrams were built as siblings on one grid, for side-by-side reading. The retirement pass in August removed or merged six rows across the six registers, each recorded in its register with a reason.

The vocabulary

Every register grades its rows A, B or C. The letter means the same thing everywhere: how complete the available remedy is.

One register reads those letters aloud in different words. Treat the words as local to each register. The letter is universal.

Class C:

Counts that span registers are therefore stated in letters, never in one register's words. When you see "43 rows fall to a person", that is Class C across six registers. On Recovery those rows read as irreversible, not survives.

Register, domain, row. A register is the enumerated list of failure modes behind one question. It has rows, each carrying an ID like CL-2A. A domain groups registers. There are five domains and six populated registers, because Evidence is one domain answered by two registers that cannot substitute for each other: Truth and Provenance.

The word layer is retired. It had been used as an undefined fourth term for the same idea. Where it still appears it is either a filename not yet renamed, or the word layer in its ordinary English sense.

If you speak audit

The framework keeps its own vocabulary because it measures failure modes rather than requirements. It maps to terms assurance teams already use.

A register is a risk register taken to row depth.

Prevented and detected are audit theory's preventive and detective controls. The departure is the third term: audit's corrective assumes a remedy exists, and survives records that none does.

A gap closed by substitute is a compensating control, in exactly the sense SOC 2 and ISO assessors use the term – a different control standing in for the specified one, carrying its own evidence, its own constraint and its own maintenance owner.

If your auditors ask what this is, that is the answer.

Two things we have not reconciled

The catch ladder – the ordered points at which a failure could be caught – mixes time with pipeline position. Lifecycle stage has no classification rule.

There were three until 28 August 2026. Severity was the third, and it is now repaired: one construct read two ways, where every register grades cost multiplied by how silently the failure arrives, and they differ only in what cost means. Both remaining items are open rows in the project's own corrections register.

How the registers change

Row IDs are stable. They are never renumbered and never reused. A row leaves the counts only through its register's retired-rows section, with a dated reason, so an old citation still resolves.

Retirement has a test. No recorded incident, no distinct mechanism separating it from a stronger neighbour, and a low audit grade – with a surviving row that carries its class.

Receipts are minted at the source, never in content. Incident markers live in the corrections register. Pages may cite them and nothing more.

Enforcement is executable, not written down. Three scripts run against the registers. One verifies row data, counts, vocabulary and the evidence labels against the re-derived receipts. One fails any published page whose stated counts disagree with the registers. One refuses a published "Recorded" line with no lodged incident behind it.

Each of those gates has been run against known-bad input and watched to catch it. A gate that has never been seen failing anything is itself a row on the Authority register.

When a failure becomes a control

The registers are half the method. The operating loop runs from an issue to a wired check: an issue becomes a register row, a row earns a control, the control is validated against an input known to be bad, wired to something that actually runs it, and only then closed. Every stage short of wired is unfinished work.

That rule exists because of a specific failure. An early debrief mapped seven failures onto the registers, lodged two rows, and changed nothing – classification had produced rows, not controls.

A register row names a way the work can fail. A control is the machinery that refuses it, or catches it after the fact. Turning every row into a control would be its own failure.

Each control runs, each one can fire on work that is fine, and a check that cries wolf teaches you to skim its output. So there is a test for whether a control gets built at all, and it is applied before the work, not after.

A control is mandatory when the failure is irreversible or reaches outside the building – money, sends, merges, deletes, anything a third party sees – no matter how rarely it happens. It is also mandatory the second time a class of failure is recorded, because a repeat is evidence the written rule did not hold.

A control is refused when the check cannot be specified precisely enough to avoid firing on good work, or when there is nothing to validate it against yet. A control nobody trusts is worse than none.

A control is deferred when it is justified but the thing it would watch does not exist yet. It is recorded as dormant, with the specific condition that would wake it rather than a vague later, so nobody has to keep explaining why it is silent.

Otherwise the row is lodged, nothing is built, and recurrence is allowed to make the case.

There is deliberately no score. The obvious move is to multiply severity by frequency by detectability and build anything over a threshold. We do not, for two reasons. Multiplying judgement ratings invents a precision the inputs do not have, and lets a severe failure be averaged away by a low frequency – the standards body that popularised that formula replaced it in 2019 for exactly that reason. And this project has its own evidence: a numbered threshold written into a brief was gamed three times over, each attempt defeating the previous check.

Frequency is not an input here at all. A once-a-year irreversible failure earns a control; a daily cosmetic one does not. If you speak audit, that is ALARP – reduce risk until further reduction is grossly disproportionate to the benefit – written as a rule rather than a number.

Measuring whether the controls are worth their cost

Four instruments run against the controls. All four have run since August 2026. A catch ledger records every time a control fires on real work, marked as a true catch, as noise, or as a defect in the control itself; the ledger is append-only. A session log records the runs where nothing was found, because without them the catch rate has no denominator. Monthly metrics track how long each failure took to become a control, how many attempts building it took, where in the pipeline it catches, and when it last caught something real. A fortnightly audit asks the two questions the other three cannot: are controlled failures still happening uncaught, and are uncontrolled ones quietly reaching a second occurrence.

The ceiling is noise, not a number of controls. When the ledger's noise entries start outnumbering the true catches, the system is over the line. And a control that has not caught anything real in months, on machinery that has not changed, is a candidate for retirement. Controls are supposed to leave as well as arrive.

Maintenance is written down, because it is the real cost. Every control records what change to the codebase would invalidate the examples it was tested against. That field exists because of a recorded failure: a control's test cases were written for one repository layout, the layout changed, and nothing forced anyone to re-check them.

What happens next

Four things, in priority order. None of them blocks anything. All of them will move counts.

The registers themselves are on the six register pages. The hub explains what they are for and what to install first. If you find a row that is wrong, or a control stronger than the one we recorded, send it to us – a correction that demotes a row is more useful than a confirmation.

False Floors is a trade mark of Digital First Pty Ltd, trading as Scale100 (AU application AMCZ-2616155657). This content is CC BY 4.0; the name is not part of that licence. Citing, mapping to, or claiming conformance with the catalogue needs no permission.