The quality gate that rejected the post I wanted

In the first post in this series, I wrote that the rebuild wasn't really about personas — it was about making the lazy path harder than the honest path. This week I have a concrete, slightly embarrassing example of what that actually takes, because I built an automated quality check that rejected the single best post my own system produced.

This is a post about voice. Specifically, about the gap between writing down what a voice should sound like and getting a machine to produce it. I had assumed the first was the hard part. It was the easy part. Here is the four-version story of the second part.

Version 1: the rules existed and did nothing

When I rebuilt the writing pipeline around the three personas — Jay here on reasonpost, Ravi on the finance site, Daniel on the health-and-habits one — I gave each one a voice specification. Jay's said things like "open with a concrete moment, not a definition" and "anchor every claim in a number you measured or a doc you can cite." Reasonable. The kind of thing you'd put in a style guide.

Then I ran the pipeline and it produced a post titled "5 AI Tools Worth Keeping After Testing Dozens."

That title is a listicle. It leads with a number. It is the exact format I had spent a paragraph of the voice spec telling the system not to write. The body wasn't terrible — it had real specifics, no marketing clichés — but it used the word "I" exactly twice in 1,750 words. For a first-person founder log, two is functionally zero. The post read like a competent trade-magazine roundup wearing a name badge that said "Jay."

The voice spec had failed completely, and it failed for a reason I should have seen coming: it was one instruction among many, and it was losing. The same prompt also contained a "format rotation" feature I'd built months earlier — a bit of logic that deliberately varies the article structure (how-to, comparison, deep-dive, and so on) so the site doesn't feel monotonous. On this run, the rotation landed on "comparative analysis," the model read that as "make a list," and the persona voice — which was sitting politely lower in the prompt — got overruled by my own older instruction.

I had two parts of the same prompt arguing with each other, and the louder one won.

Version 2: making the rules louder

The fix for version 2 was blunt. I rewrote the persona block to explicitly outrank the format instruction:

## PERSONA VOICE — OVERRIDES THE FORMAT INSTRUCTION ABOVE
IGNORE format rotation if it conflicts with persona voice.
The persona always wins.

Then I added two hard sub-rules. A title rule: no leading numbers, no "Best," no "Top," no "Ultimate Guide" — titles must be sentences or declarative fragments, with worked examples of passing and failing titles. And a first-person rule: use "I" and "my" deliberately, target at least ten occurrences, open on a concrete moment rather than a definition.

But a prompt instruction is a request, not a guarantee. If I actually wanted these properties, I needed something downstream that checked — a gate that could reject a post the way a copy editor would, before anything got saved. So I also wrote that gate. It counted first-person pronouns, scanned the title against a list of banned listicle patterns, and refused to pass anything that failed.

I was fairly pleased with version 2. It had a rule and an enforcer for the rule. That is the correct shape.

Version 3: the gate rejected the good post

Then I ran it, and the gate rejected all three sites' output.

Not because the posts were bad. Because of this line, which I had written myself, in my own quality gate:

var firstPersonOk = firstPersonTotal >= 10 && firstPersonTotal <= 30;

I had set a ceiling. My reasoning at the time, such as it was: too much "I" reads as self-indulgent, a stream-of-consciousness diary, so cap it at 30. It sounded prudent when I typed it.

The reasonpost post that run came back titled "The AI job capability tests I ran came back mixed." That is a real sentence. It is not a listicle. It opens on a concrete action. It is, by every standard I'd written down, the post I wanted. Its first-person count was 57.

Fifty-seven is greater than thirty, so my gate stamped it firstPersonOk: false and threw it in the reject pile.

I went and counted the founder logs I'd already published by hand — the ones I was using as the examples of good voice. They ran 40 to 60 first-person uses each. My own published, human-edited writing would have failed my own automated gate. I had encoded a number that contradicted the very samples the system was supposed to imitate.

The cap went to 80. The floor of 10 stayed. The honest version of this is: I never had evidence for 30. I had a vague feeling, and I shipped the vague feeling as a hard threshold, and it took a concrete rejection to show me the feeling was wrong.

There was a second, smaller version of the same mistake in the same run. Daniel's health-and-habits persona has a rule against medical-advice language, enforced by a keyword blocklist — words like "treat," "prescription," "medical advice." His draft got rejected for containing the phrase "medical advice." It contained it because the draft's own disclaimer said "this is not medical advice." My blocklist couldn't tell the difference between giving medical advice and explicitly disclaiming it. I'd written a filter that punished the persona for following the rule.

Version 4: checking for specificity, not just against clichés

Versions 2 and 3 were all negative checks — ban this title, cap that count, blocklist those words. A post could pass every negative check and still be hollow. Polished, clichér-free, correctly first-person, and still saying nothing in particular.

So version 4 added a positive requirement. The gate now also measures specificity, and it fails a post that doesn't clear a floor:

var numericOk     = numericTokens >= 8;
var structureOk   = codeBlocks >= 1 || boldEntities >= 5 || hasMarkdownTable;
var specificityOk = numericOk && structureOk;

At least eight specific numbers with units — dollars, percentages, durations, counts — not "many" and not "several." And at least one real structural element: a code block, a comparison table, or five bold-emphasized named things. I checked these thresholds against R1, R2, and R3 first; they clear them comfortably, because writing that comes from actually doing something tends to be full of specific numbers as a side effect. Generic writing isn't. The specificity floor is really a proxy: it's hard to hit eight concrete numbers unless you had eight concrete things to say.

This post you're reading has well over eight. Not because I padded it — because a story about four versions of a bug has four version numbers, a pronoun count of 57, a ceiling of 30, a floor of 10, a new ceiling of 80. The numbers were always there. The gate just insists I don't throw them away.

What the four versions actually taught me

The lesson isn't "prompts are unreliable" or "write a quality gate." Everyone says those. The specific thing I didn't appreciate until this week:

Every rule you add is also a claim, and some of your claims are wrong. "Cap first person at 30" was a claim about good writing. It was false, and I'd have kept believing it if the gate hadn't rejected a post concrete enough for me to look at and say — no, that one's good, the gate is wrong. The gate's job turned out not to be catching bad posts. Its job was catching my bad rules, by enforcing them hard enough that their wrongness became visible.

That only works if the gate is strict. A loose gate would have passed the listicle, passed the two-pronoun post, passed the hollow-but-clean post, and I'd have learned nothing. A strict gate fails loudly, including when the rule behind it is the thing at fault. Version 3 was painful for about an hour. It was also the only version that taught me something I couldn't have reasoned my way to from my chair.

A scheduling note, because I owe one. The previous post here ended by saying Daniel would be next — the spanpost persona, his caffeine-and-focus-and-mood logging project. That post is still coming, but the only honest version of it requires real data, and the real data is roughly three weeks out from being a meaningful sample. So the order shifted. Ravi's first real distillfin post went live earlier this week instead, charting the gap between the 10-year Treasury and the 30-year mortgage; Daniel's first post will publish when the log accumulates enough entries to surface a pattern rather than a vibe.

The general lesson is the same one this post was about. It is easy to state a rule — for voice, for content, for a publication schedule — and the whole job is everything that happens after the rule meets a real example.

— Jay