Home / Guides

Are AI detectors accurate?

Short answer: sometimes, on the right kind of text, and never well enough to be the only evidence. Here's what the people who build and study these tools have actually published, and how to read a score without fooling yourself.

Updated October 10, 2026 · By the IsThisAI? team

How AI detectors work

Most AI detectors are themselves machine learning models. They are trained on piles of human writing and piles of AI writing, and they learn statistical differences between the two. Turnitin describes its own approach in plain terms: language models tend to pick the next word in a "consistent and highly probable fashion," while human writing "tends to be inconsistent and idiosyncratic" (Turnitin FAQ).

That gives you the first big limitation. A detector never sees you write. It only sees the finished text and guesses how likely it is that a model produced it. Writing that happens to be very predictable, like a formulaic lab report or a carefully simple essay by someone writing in a second language, can look "AI-like" even when a person typed every word.

Our own detector works the same basic way. It runs two open-source models in your browser and adds a few transparent style checks. It gives a probability, not a verdict.

Accuracy vs. false positives

"Accuracy" hides two very different mistakes:

• Missed AI text (false negatives): the tool says human, but a model wrote it.
• Flagged human text (false positives): the tool says AI, but a person wrote it.

The second kind is the one that hurts people. Detector makers usually tune their tools to keep it low, and they accept missing more AI text as the price. Turnitin says this openly. It aims to keep its document-level false positive rate under 1% for documents with over 20% AI writing, and says that to hold that line "there is a chance that we might miss some AI written text." Its example: a document scored at 50% AI "could contain as much as 65% AI writing" (Turnitin FAQ).

Under 1% sounds tiny until you scale it up. Vanderbilt University did that math when it switched off Turnitin's AI detector in 2023: it had submitted 75,000 papers to Turnitin in 2022, so a 1% false positive rate would have meant "around 750 student papers could have been incorrectly labeled" (Vanderbilt). Turnitin has also said its sentence-level false positive rate is around 4%, so an individual highlighted sentence is less trustworthy than the overall score (Turnitin CPO update).

What independent tests found

Vendor numbers come from vendor tests. Independent results have been rougher.

• OpenAI pulled its own detector. OpenAI's AI classifier correctly flagged only 26% of AI-written text as "likely AI-written" in its own evaluation, while wrongly labeling human text as AI 9% of the time. On July 20, 2023, OpenAI shut it down "due to its low rate of accuracy" (OpenAI).
• A 14-tool academic study by Weber-Wulff and colleagues concluded that the detection tools they tested were neither accurate nor reliable, tended to call text human, and got worse when AI text was edited or paraphrased. The authors advise against using them as evidence of misconduct (Weber-Wulff et al., 2023).
• Bias against non-native writers. Stanford researchers ran seven detectors on 91 TOEFL essays written by people, and the detectors labeled more than half of them AI-generated, with an average false positive rate of 61.3%. The same detectors were near-perfect on essays by US eighth graders (Liang et al., 2023).

We ran our own small test before launching. Six free, open-source detectors were checked against 142 documents, and every one of them missed nearly all of the 13 AI-written resumes we tried; the best flagged one. That's why our tool shows a caution note on resumes. Lists and bullet points just don't give a detector much to work with, and Turnitin also says its model skips bullet points and other non-prose text (Turnitin FAQ).

Why short text fails

Detectors need enough words to spot a pattern. Turnitin won't produce an AI score at all for fewer than 300 words of prose, and says that in documents of only a few hundred words "the prediction will be mostly all or nothing" (Turnitin FAQ). OpenAI said its classifier was very unreliable on text under 1,000 characters (OpenAI).

So a one-paragraph email, a tweet, or a short discussion post is close to a coin flip for any tool. When you paste short text into IsThisAI?, we label the result "low confidence" for that reason.

How to read a % score

A score like "72%" is easy to misread. Some tools mean "72% of the text looks AI-written." Others, including ours, mean "the model is 72% confident this text came from AI." Neither one means "there's a 72% chance this person cheated."

A few practical rules:

1. Treat the middle of the range (roughly 30% to 70%) as "unclear," not as half guilty.
2. Low scores matter less than you'd think. Turnitin doesn't show a number at all between 1% and 19%, only an asterisk, because false positives are more common there (Turnitin FAQ).
3. Look at which sentences are highlighted. If it's the generic intro and conclusion, that's weak evidence. Plenty of people write bland intros.
4. Check the type of text. Resumes, lists, code, poetry, and heavily templated writing are outside what most detectors handle well.

Using IsThisAI? responsibly

We built IsThisAI? as a quick, free second opinion. It's useful for checking whether your own writing sounds robotic before you send it, or for deciding whether a piece of text deserves a closer look. It's not a lie detector, and we say so on every result.

If you're a teacher, MIT Sloan's teaching team has a blunt recommendation: don't rely on detectors, and build trust through clear AI policies and assignments that show the writing process (MIT Sloan). If you're a student who was flagged, see what to do about a false positive and how to prove you wrote it yourself.

More guides

Sources

1. Turnitin, "Turnitin's AI writing detection capabilities FAQs" (Turnitin Guides, updated Oct 7, 2026). https://www.turnitin.com/products/features/ai-writing-detection/faq
2. Annie Chechitelli, "AI writing detection update from Turnitin's Chief Product Officer" (Turnitin blog, 2023). https://www.turnitin.co.uk/blog/ai-writing-detection-update-from-turnitins-chief-product-officer
3. OpenAI, "New AI classifier for indicating AI-written text" (Jan 2023; updated July 20, 2023 when the tool was withdrawn). https://openai.com/index/new-ai-classifier-for-indicating-ai-written-text/
4. Weber-Wulff et al., "Testing of detection tools for AI-generated text," International Journal for Educational Integrity 19, 26 (2023). https://link.springer.com/article/10.1007/s40979-023-00146-z
5. Liang, Yuksekgonul, Mao, Wu & Zou, "GPT detectors are biased against non-native English writers," Patterns 4(7), 2023. https://www.cell.com/patterns/fulltext/S2666-3899(23)00130-7
6. Vanderbilt University Brightspace team, "Guidance on AI Detection and Why We're Disabling Turnitin's AI Detector" (Aug 16, 2023). https://www.vanderbilt.edu/brightspace/2023/08/16/guidance-on-ai-detection-and-why-were-disabling-turnitins-ai-detector/
7. MIT Sloan Teaching & Learning Technologies, "AI Detectors Don't Work. Here's What to Do Instead.". https://mitsloanedtech.mit.edu/ai/teach/ai-detectors-dont-work/
8. IsThisAI? internal benchmark of six open-source detectors on 142 documents, including 13 AI-written resumes (Oct 2026). https://getisthisai.com/about/

This guide is general information, not legal or academic-policy advice. AI detection results, including ours, can be wrong.