What you can test

Part one left everything in one bin. Every piece of information I receive is entertainment data: acknowledged, unjudged, allowed to sit there. That is the resting state, and most data never leaves it.

But the bin has a second question written on the lid: can I test this?

The robbers

Someone tells you your house will be robbed tonight at two in the morning.

That is a different kind of sentence from “the government has known about aliens for decades”. It has a time, a place, and a procedure attached. You can check the cameras in the morning. You can sit up in the dark with a torch and see who comes. You do not have to take anyone’s word for anything; you can go and look.

I flag data like this as actionable. Not true — actionable. It is still entertainment data in the sense that I do not know its truth value. The difference is that I know how to find out, and the finding out is within my reach.

So you sit up. Two possibilities.

The robbers come. The claim passed. Whoever told you knew something, and the world did what they said it would.

The robbers do not come. Here it gets interesting, because this tells you much less than it feels like it does. Maybe there never were any robbers and the claim was simply false. Maybe there were, and they chose a different house, or got a flat tyre, or saw your lights on and drove away. You still cannot know whether the original claim was true. What you know is what happened, which is not the same thing.

That asymmetry is worth holding on to. A test that passes is strong evidence. A test that fails is weak evidence, and usually weaker than your relief wants it to be.

Tests I cannot run

Once you start sorting by testability, you notice that the biggest category is neither of the two above. It is data that is testable in principle, but not by me.

The Earth is round. I could test this myself, at some expense: sail far enough in one direction and arrive from the other. I have not. Vaccines prevent more deaths than they cause. I could not test this myself at any expense; it takes tens of thousands of people and years. Most of medicine is like this. Most of physics. Almost everything I rely on to get through a day.

Part one said the framework has no connection to other people’s opinions, and I meant it. But a reported result is not an opinion, and I need a rule for telling them apart, because I depend on other people’s tests whether I like it or not.

Here is mine. An opinion is a truth value someone else has filled in. “Vaccines are dangerous.” “The Earth is flat.” “I think they are hiding something.” It does not matter who holds it, how confident they are, or how many of them there are: an opinion is entertainment data, and no crowd is large enough to change that. A reported result is a description of a test that was run: what was done, to how many, what came out. That is actionable data with someone else’s run attached, and I treat it accordingly. The question is not whether I trust the person. It is whether the test exists, whether it could be run again, and whether anyone has. A result that others have reproduced is worth a great deal. A result nobody can reproduce is worth very little, however impressive the person reporting it.

Notice that this is not deference to authority. Authority is a kind of opinion. What I am weighing is the test, and the person only matters as far as they tell me about the test.

What keeps paying out

There is a third flag, and I used to think of it as a third bin. It is really a corner of the second.

Some actionable data, once tested, keeps passing. Not once but every time you use it. Someone teaches you to fish, and the river yields fish this year and next. A recipe works. A route through town is faster, and stays faster. The stove is hot. I call this the Promethean flame, after the gift that kept on giving: data you can carry away and use over and over, and it pays out every time.

This is where you get to stop testing. Nobody re-derives arithmetic before paying for groceries. You build on the flame instead of re-lighting it, and that is what lets a life, or a civilisation, get anywhere.

The flame is also where the framework is most easily abused, because things sneak in that have not earned the place. Something that felt true, or worked once, or that everybody around you carries as if it were fire. The test for the flame is boring on purpose: it passed, it keeps passing, and I could show you how to make it pass too. If I cannot, it is not flame yet, however warm it feels.

Three flags, one inbox

So the framework is not three bins in a row. It is one inbox and two questions. Everything arrives as entertainment data. Some of it has a test attached, and that is actionable. Some actionable data has been run so many times that I stop running it, and that is the flame.

Most of what I receive stays in the first state forever, and I have made my peace with that. But it raises a question I have been stepping around. If almost nothing I believe gets tested, and I get through the day anyway, how much does it matter what I believe at all? For most people, most of the time, the honest answer is: less than you would think. Who it does matter to is the next part.