Further reporting on the same study: detectors trained on older models can drop from over 99% to 3.8% detection after a new LLM generation is introduced.
Following·Day 5·first covered 8 Oct 2026·2 sources
Pangram AI Detector Misses 79.8% of Muse-Glimmer Rewrites in Study
The gist
Pangram's detection rate varies sharply depending on which LLM produced the text, meaning reliance on a single detector can produce very different results across models. The Critchley case illustrates that the tool is being applied in contexts where its accuracy assumptions may not hold.
The full picture
A Tokyo Metropolitan University study found that Pangram, an AI-text detector, missed 79.8% of scientific abstracts rewritten by Meta's Muse-Glimmer model, while correctly flagging 93.5% of abstracts rewritten by GPT-5 and only 1 of 5,000 human-written abstracts. The study also found that detectors trained on older models can drop from over 99% detection to as low as 3.8% when a newer LLM generation is used for rewriting. Researchers concluded that anyone relying on AI-text detectors should re-test them with every new model release. Separately, a New York Times op-ed by philosopher Simon Critchley arguing that AI cannot do philosophy was itself detected as roughly 76% AI-written using Pangram, with multiple observers independently replicating that result.
How it developed
A NYT op-ed by philosopher Simon Critchley arguing AI cannot do philosophy was detected as roughly 76% AI-written using Pangram, with multiple observers replicating the result.
Sources
Rohan Paul Twitter
Zvi's AI Roundups
The daily email
Want this in your inbox?
I send one email each morning with the stories that moved. If you would rather just read here, that works too.
Subscribe free