Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 124 additions & 0 deletions bin/check-post-voice
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
#!/usr/bin/env ruby
# frozen_string_literal: true
#
# Mechanical check for blog-pipeline.md STEP 4b (cadence quotas) and STEP 5a
# (anti-AI tells). Both were prose checklists, and prose checklists get skipped:
# three posts shipped on 2026-08-22 having passed neither, and the slop was
# visible to a reader immediately.
#
# bin/check-post-voice content/blog/<slug>/index.md
#
# Exits non-zero if any rule fails, so it can gate a commit.
# It measures SHAPE only. It cannot tell you whether the post is any good.

require "set"

path = ARGV[0] or abort "usage: bin/check-post-voice <post.md>"
raw = File.read(path, encoding: "bom|utf-8")
parts = raw.split(/^---\s*$/m)
body = parts[2].to_s

MANNERISM = %w[discipline cycle pattern canonical workflow framework cadence
rhythm contract gate gatekeeper layer round review pass].freeze
BANNED = %w[unlock harness leverage game-changer journey seamless robust
supercharge revolutionize embark delve discover].freeze

# --- split into sections of prose paragraphs, skipping fenced code ------------
sections = Hash.new { |h, k| h[k] = [] }
current = "(lead)"
buf = []
in_code = false

flush = lambda do
para = buf.join(" ").strip
buf = []
return if para.empty?
sections[current] << para
end

body.each_line do |line|
if line.strip.start_with?("```")
flush.call unless in_code
in_code = !in_code
next
end
next if in_code

if line =~ /^##+\s+(.*)/
flush.call
current = Regexp.last_match(1).strip
elsif line.strip.empty?
flush.call
elsif !line.start_with?("#", ">", "-", "|", "!")
buf << line.strip
end
end
flush.call

failures = []
prose = sections.values.flatten
abort "no prose found in #{path}" if prose.empty?

# --- STEP 4b: cadence quotas, per H2 -----------------------------------------
sections.each do |name, paras|
sents = paras.flat_map { |p| p.split(/(?<=[.!?])\s+/) }.reject { |s| s.strip.empty? }
next if sents.empty?
lens = sents.map { |s| s.split.size }

failures << "#{name}: no sentence under 9 words (min #{lens.min})" if lens.min >= 9
failures << "#{name}: no sentence over 22 words (max #{lens.max})" if lens.max <= 22
failures << "#{name}: no single-sentence paragraph" unless paras.any? { |p| p.split(/(?<=[.!?])\s+/).reject { |x| x.strip.empty? }.size == 1 }

over = paras.count { |p| p.split(/(?<=[.!?])\s+/).reject { |x| x.strip.empty? }.size > 3 }
failures << "#{name}: #{over} paragraph(s) over the 3-sentence cap" if over.positive?
end

# --- STEP 5a: anti-AI tells --------------------------------------------------
sents = prose.flat_map { |p| p.split(/(?<=[.!?])\s+/) }.reject { |s| s.strip.empty? }

tell = lambda do |label, matches|
return if matches.empty?
failures << "#{label} (#{matches.size}): #{matches.first.strip[0, 90]}"
end

tell.("negative parallelism", sents.select { |s| s.strip =~ /\ANot [a-z]/ })
# Only flags the generic subject DOING something ("the team ships", "most
# founders assume"). "anyone on the team could remember" names a real team and
# is not the tell - matching it was a false positive on 2026-08-22.
tell.("generalized actor", sents.select { |s| s =~ /\b(the developer|the team|most teams|most founders)\s+(is|are|was|were|has|have|will|can|should|does|do|needs?|assumes?|ships?|writes?)\b/i })
tell.("definitional cadence", sents.select { |s| s =~ /\b(is|that is) the (canonical|largest|whole|real|part|version|move|rule|reason|kind)\b/i })
tell.("signposting", sents.select { |s| s =~ /\A(This post|Here'?s what|In this article|Let'?s (dive|explore))/i })
tell.("therapist voice", sents.select { |s| s =~ /that'?s the (hardest|worst|tricky)/i })
tell.("copula avoidance", sents.select { |s| s =~ /\bserves as\b/i })

banned_hits = BANNED.select { |w| body.downcase =~ /\b#{Regexp.escape(w)}\b/ }
failures << "banned words: #{banned_hits.join(', ')}" unless banned_hits.empty?
failures << "em dash present - use '-'" if body.include?("—")

mann = MANNERISM.sum { |w| body.scan(/\b#{w}s?\b/i).size }
failures << "mannerism nouns: #{mann} (cap 5)" if mann > 5

opener = prose.count { |p| p =~ /\A(The|We)\b/ }
pct = (100.0 * opener / prose.size).round
failures << "The/We paragraph openers: #{pct}% (cap 20%)" if pct > 20

# --- artifact + receipts (2026-08-22: every post owes both) ------------------
code_blocks = body.scan(/^```/).size / 2
images = body.scan(/!\[[^\]]*\]\(/).size
failures << "no artifact: 0 code blocks and 0 images" if code_blocks.zero? && images.zero?

external = body.scan(%r{\]\((https?://[^)]+)\)}).flatten.reject { |u| u.include?("jetthoughts.com") }
failures << "no receipts: zero external citations" if external.empty?

# --- report ------------------------------------------------------------------
puts "#{path}"
puts " #{prose.size} paragraphs, #{code_blocks} code blocks, #{images} images, #{external.size} external citations"

if failures.empty?
puts " PASS"
exit 0
end

puts " FAIL (#{failures.size})"
failures.each { |f| puts " - #{f}" }
exit 1
30 changes: 30 additions & 0 deletions content/blog/how-to-audit-content-you-didnt-write/domains.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 8 additions & 10 deletions content/blog/how-to-audit-content-you-didnt-write/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,17 +38,15 @@ Then a tool started drafting, and the person approving its output was not equipp

## Nobody can tell you how much of the web this is

You would think there is a number. There are several and they disagree.
You would think there is a number. There are several and they disagree - ten percent, a third, or half, depending on whose sample and whose detector.

[Graphite](https://graphite.io/five-percent/more-articles-are-now-created-by-ai-than-humans) sampled 43,000 CommonCrawl URLs published between January 2020 and May 2025, ran them through Surfer's detector, and put the crossover - more machine-written articles than human ones - in November 2024.
[Graphite](https://graphite.io/five-percent/more-articles-are-now-created-by-ai-than-humans) put the crossover, more machine-written articles than human ones, in November 2024. [Pew](https://www.pewresearch.org/data-labs/2026/08/20/how-much-of-the-internet-is-written-with-ai/) ran ~490,000 Common Crawl pages through Open Pangram this month and found 10% carrying AI-authorship signals, rising to over a third among pages published after ChatGPT shipped. Both publish their error bars, which is the habit worth stealing whatever you make of the figures.

[Pew](https://www.pewresearch.org/data-labs/2026/08/20/how-much-of-the-internet-is-written-with-ai/) ran roughly 490,000 English-language pages from the same archive through Open Pangram this month and found 10% of all sampled pages showing significant signs of AI authorship - rising to over a third when you look only at pages published after ChatGPT shipped.
Pew also split it by domain, and that is where you come in:

Ten percent, a third, or half. The numbers are not contradicting each other so much as answering different questions, on different samples, with different detectors.
![Share of pages showing AI-authorship signals by domain: .com around 10 percent, .org 4.6 percent, .edu and .gov around 1 percent.](domains.svg)

Both teams publish their error bars, and that habit is worth stealing regardless of what you make of their numbers. Graphite reports a 4.2% false-positive rate against GPT-4o content and no measurement at all of AI-assisted-then-human-edited writing. Pew says plainly that "AI detection models aren't perfect" on individual documents and only hold up in aggregate.

One more number from Pew is the one that should interest you: **around one in ten `.com` pages carry those signals - roughly double the `.org` rate and ten times what `.edu` and `.gov` show.** Commercial sites are where this concentrates, and yours is a commercial site.
Commercial sites are where this concentrates, and yours is a commercial site.

So the honest position is that the web's average is unknown and the detectors that estimate it are themselves approximate. Which is fine, because the average was never the thing you needed. **You need to know about your property, and your property is countable.**

Expand Down Expand Up @@ -95,7 +93,7 @@ Run the first one and read every hit. Real client work names the client or does

Count how many of your long posts link to nothing outside your own domain.

A post with no external citation is not necessarily wrong. It is unverifiable, which means nobody could have checked it - including whoever wrote it, at the moment they wrote it. That is the distinction I care about, and it is the one this count actually measures.
A post with no external citation is not necessarily wrong. It is unverifiable, which means nobody could have checked it - including whoever wrote it, at the moment they wrote it.

Uncheckable is where wrong survives, and the number usually comes back higher than anyone guesses.

Expand Down Expand Up @@ -131,9 +129,9 @@ Then check each against the vendor's own support table. Laravel, Rails and Node

It cannot tell you whether a claim is true.

We tried to build that and failed honestly. Two candidate patterns for detecting wrong technical claims got measured against the archive before we trusted either: one matched ten passages, mostly legitimate; the other matched a hundred and eighty-five, almost all ordinary prose. Both would have cried wolf until people stopped reading the output.
We tried to build that and failed. Two candidate patterns for detecting wrong technical claims got measured before either was trusted: one matched ten passages, mostly legitimate, the other a hundred and eighty-five, almost all ordinary prose. Both would have cried wolf until people stopped reading the output.

A wrong explanation is a well-formed sentence using correct vocabulary, pointing roughly the right direction. There is no pattern for it. What catches it is someone who knows the subject, reading with permission to disagree - which is a different investment, and the one worth making after the four checks above have narrowed where to look.
A wrong explanation is a well-formed sentence using correct vocabulary, pointing roughly the right direction. What catches it is someone who knows the subject reading with permission to disagree - a different investment, and the one worth making after these four have narrowed where to look.

## Where to start this week

Expand Down
36 changes: 36 additions & 0 deletions content/blog/what-senior-developers-catch-that-ai-misses/chain.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading