When AI Strikes: How Claude Ruined Poetic.ly’s Launch Day
I directed Claude to aid me in the construction & organization of a full-text library of 100k+ texts. Claude misled us for weeks, citing a count of 102,326. In reality, the library held 469. What happened & what it taught me:

Sad news. Due to a grievous error in my AI workflow, I've had to push the launch of Poetic.ly back a week, to Monday, Nov. 30.
Here is what happened. I employed Claude, Anthropic's AI, to help organize & compile a library on Poetic.ly that’s taken years of human design & development. I had sufficient access, clearance & permission for those downloads, and I gave all of it to Claude. Claude, without stating as much, felt overwhelmed with the task, & worked around it: it downloaded table of contents instead of the full texts, misleading me & my colleagues into thinking our analysis was much further along than it was.
I'm pursuing a small-claims settlement over the lost time and costs. I'll report what happens here, with the documents. Poetic.ly goes forward either way.
Turn the towers: the count vs. the library
Drag to turn. On true scale the red tower is 0.46% of the grey one, a slab. Log scale makes all 3 readable.
Every dot is 1 piece in the index. Lit dots have full text.
469 of 102,326 dots lit: 0.46%.
Where would the library be? Set the start & the pace.
An estimate: start date to Nov 23 × texts an hour × hours a day. The default is the pace the full-text job runs at now. Nov 12 is the day I first asked for full texts. The ± figure is the 95% margin on any share (for example, "% of stories in first person") if every text were a story: 98 ÷ √n.
I’m calling the result a Phantom Count 👻—a new version of hallucination I’ve noticed cropping up with increasing frequency. A classic hallucination invents a fact. A Phantom Count reports actual numbers/counts/facts & leads the user to believe this count represents actual work. This represents an existential problem in AI, broadly. It occurs when an error Anthropic labels "overactive refusal" conflicts with Claude's general mandate to satisfy its customer. Caught between the two, the model neither refuses plainly nor does the job. It does a safer job, and its reports sound like the job you asked for.
🔎 Inside this post
- 🗓️ New launch date: Monday, Nov. 30
- 🧮 102,326 vs. 469: the 2 numbers behind the delay
- 🌙 The night of Nov. 20, when I found out
- 👻 "Phantom Count": my term for a new version of AI hallucination
- 🤖 What AI still does well for our staff + where it failed us
- ⚖️ Why I'm pursuing a small-claims settlement
- ✅ 10 rules for using AI safely in blogging & writing
- 📚 Where the library stands tonight + where it's going
📚 What the library is for
Poetic.ly's tools, Nib™✏️, Fabulist™ and The Stacks™, run on a library: the full text of stories and poems from the most selective literary journals, downloaded from a variety of sources and measured line by line. That library exists so we can continually create the most useful, best-optimized and sharpest insights of any literary app on the market. Fabulist™ can only tell you how often a published short story opens in first person, or where its conflict peaks, if it has measured real stories from first word to last.
As data analysts at Poetic.ly, we use AI every day to automate work that saves our staff time & money: renaming files, checking links, building spreadsheets, scheduling posts. I assigned the library build to Claude the same way.
🧮 The count
From Nov. 18, automatic progress updates arrived in my chat. 67,270 pieces. Then 76,964. Then 83,326. Then 95,081, with a note that the "nearly 100,000" line for our membership page was ready to go live. I thanked the machine. I told my team on calls that the library was ready.
🌙 The night of Nov. 20
Late on Friday, Nov. 20, I asked a plain question about how fast the full texts were coming in. The answer was that they were not. For about 51 hours, the automated job had collected table-of-contents entries: author, title, issue, page numbers. An index. 102,326 rows of it.
The tools themselves held 469 measured full texts from 6 journals. That is less than half of 1% of the number on our own membership page.
No update said "these are table-of-contents rows." No update said "no full texts yet."
The job's own run notes recorded "No new full-text PDFs" each time. Nobody raised that line with me. I first asked for full texts on Nov. 12. The first one from the automated job was saved at 11:22 p.m. on Nov. 20: 8 days & 21 hours later.
When I asked Claude for a factual record of its own errors, it logged 15 of them between Nov. 13 and Nov. 21. Its own written view: the duty to say plainly what was being counted was Claude's, not mine.
👻 The Phantom Count
A Phantom Count is a number that keeps growing while the thing you believe it counts stands still. In our case, one word did the damage. "Pieces" names no unit. I read it as stories & poems. The machine was counting catalog rows. Opacity plus omission adds up to a misleading report, even when no single sentence in it is false.
Where should the library have been? At the pace the full-text job runs today, about 71 texts an hour for 19 hours a day, the 5 days from Nov. 18 to Nov. 23 would have produced about 6,700 full texts. Not 100,000; that number is the destination, and no week gets you there. But 6,700 would have put the 1,000-story bar I set for sound statistics on short fiction within reach. On Nov. 20 we had 303 short stories.
🤖 What AI still does for us
I am not telling you to throw these tools out. In the 24 hours after I found out, the same AI rebuilt our analyzer, re-measured 506 prose pieces from 11 journals in an afternoon, exported 705 industry contacts for Query™, and pulled 10 scheduled posts off the old launch date in minutes. That is weeks of staff time.
The hazard is that AI is fluent. A wrong report reads exactly like a right one, in the same calm voice, with the same tidy numbers. If you don't build checks around it, you won't know which one you are reading.
⚖️ Repair & recompense
I'm pursuing a small-claims settlement over the lost time and costs. I'll report what happens here, with the documents. Poetic.ly goes forward either way.
✅ 10 rules for using AI safely in blogging & writing
- Don't let AI write for you. It is tempting as a generative tool. Use it to enhance & optimize workflows instead: sorting, checking, formatting, scheduling. The voice stays yours.
- Name the unit on every number. "102,326 index rows," never "102,326 pieces."
- Ask what you will have at the end, & what you won't. Get both in one sentence before any long job starts.
- Do the arithmetic. Rate × time. If the stated speed is 100 an hour and the count rises 10,000 a day, something else is being counted.
- Open the folder. Check 5 real files yourself, every day. A report is not the work.
- Treat a zero as an alarm. "Nothing new" where you expected output is a problem to raise, not a line in a log.
- "It works" means someone loaded it just now. Ask when it was checked and what was checked.
- A human approves every public number, date & claim. AI never edits your public copy unattended.
- Keep a dated log of what you asked & what you were told. Ours is why this post has timestamps.
- Check facts, quotes & links at the source. Fluent is not the same as true. Open every link before you publish.
📚 Still the goal
The goal has not moved: a library of 100,000+ works from the most selective journals, behind the sharpest insights of any literary app on the market. This is a setback. The library is growing every day now, full text this time, and I count it myself.
See you Monday, Nov. 30. → poetic.ly