Can you tell AI writing from human writing?

aside_dish

Member
You often hear writers lament about the increasing use of AI slop in writing circles, and how you can tell immediately that something is generated by AI and it degrades the quality of writing. And I agree it’s a problem. It completely takes the human element out of storytelling – which is something robots will never be able to truly replicate.

However, I’m not entirely convinced that people can actually tell something is written by AI – and I believe it often leads to people lofting false accusations at their fellow redditors (many of whom just paid attention in English class and learned to love use of the em dash [and, in my case, the en dash]).

Take these two passages, for example. One is written by AI, while the other is written by a human:

  1. Grayson Mudjoy was about as happy as a man could be in his situation — swell, even. And that wasn’t a figure of speech, it was literal. He’d recently rolled his ankle, which wasn’t nearly as fun as what he normally rolled, which was joints. Well, I guess the ankle is a joint.
  2. Ser Adavan was taken aback by what had just transpired in the throne room only moments ago. “Take ‘em to a’back of the room, and throw ‘em in the dungeons!” the king had shouted. Ser Adavan was absolutely boned.

Can you tell which of these two is AI? What if I told you they both were?

Or how about these two passages?

  1. Yara loved the way her kitten, Miles, laid down in his cat tree, tail tucked and paws curled. She always looked forward to it when coming out of her bedroom in the morning, and it’s what she found solace in when the would give him a kiss goodnight as she went back into the bedroom at night. Simply put: Miles was absolutely purrfect.
  2. Mark strode confidently through the hallways of Newman High. He was a football player, and football players were like the kings of the school at Newman. But kingdoms only had one king, and since Mark was the captain of the team, he was the king among kings.

Is it the first or the second?

If you guessed the first, you’re wrong! The second may read more human-like, but it’s AI-generated!

But actually, it’s the second.

Or both. Yeah, it’s both.

Another pair:

  1. Questions were like shitty music opinions in that every hipster with an eyebrow ring and a Bad Religion tattoo had about thirty of them. And the questions were usually shittier and more pointless than the music opinions and tattoos.
  2. Jane only had thirteen minutes to be at work, and the next bus wasn’t scheduled for another fifteen. She wished she could say this was the first time — or the fifth. Or the twelfth. But the truth was, Jane was habitually late — and that was exactly what would save her on this Thursday.

Did you guess that both of those were AI?

Because they’re not

— at least if you meant something else other than Artificial Intelligence. If that is what you meant, then it is.

But actually, all of this is AI. Every. Single. Passage. Not a single one was human-made. Varying quality, varying sentence structure and verbiage, and yet it’s all AI, and you likely couldn’t even tell.

Did you read the bold above and felt vindicated? Because here’s a video of me writing all of this live, using a random word generator to get started on these passages. Perhaps not the best examples — and indication of my own lack of writing ability perhaps — but I’m pretty confident none of you guessed that none of this was generated at all.


And I think that brings about the point that I started with: you can’t always tell. Everyone is so quick to say something is written by AI, when it’s not always so clear cut. So, before you go leveling accusations, ask yourself if this person used AI, or if they’re just a competent writer?

Probably the latter. But maybe not! And the fact that it's becoming increasingly harder to tell makes it all the more necessary to do your due diligence, and ensure that you're not falsely accusing people.
 
I found it implausible that AI would:

* break narrative tense and POV with Well, I guess the ankle is a joint.

* confuse lay/laid in ...the way her kitten, Miles, laid down in his cat tree...

Also, an AI trained by rulebooks would likely include the commas around Miles, but one trained by scanning masses of 20th-century fiction would omit them.

I found the Ser Adavan one and the shitty music opinions one credible as either human or AI.

I pegged Mark strode confidently and Jane only had thirteen minutes as human, but with the caveat that depending on its training, an AI might mimic the new-writer-in-first-critique-group tone of the first and the overuse of this in the second.
 
Last edited:
Yeah, it's important to not falsely accuse people. And I'm on the fence on if it's even fair to say, "You probably didn't intend this, but it reads as though it's generated."

I straight up cannot tell when it comes to poetry. Probably can't tell with most flash fiction either.

Back to prose, though, this isn't double blind or at least not written by yourself, so I'm not sure this is the best way to present your argument. For example, I could write something that seems like AI, consciously or otherwise, if it serves the point I'm trying to make:

George wasn't mad, he was furious. The kettle whistled. A dog barked in the neighbor's yard. The smell of burnt popcorn and broken dreams filled the kitchenette, but he didn't care. All that mattered was his son's report card, staring back at him like it didn't owe him anything.

Ahah, but it was a human! I even spelled neighbour the silly way to make sure it would match up. Though people could cheat and google it, passages from works published before AI gen took hold might be more fair. I can actually think of a passage from Who Goes There that seems LLM-crafted. Wow, that's ironic.


I don't think many people could tell with much certainty from one passage. The generators seems to go through phases. Not x, but y is the most hilarious overt one right now. I saw it peppered in an article someone linked to from here, made me wince. The website of that scammy short story publisher was well designed yet featured a hefty amount of AI drivel (it was like observing bad plastic surgery if that makes sense?). More broadly, it's certain uncanny choices that just don't line up: personifications, similes, metaphors that reach in strange directions and seem to sit isolated in the prose. Headless, ineffective meaning in spite of the power words: no one's driving the bus but it's full freaking throttle. Can I know for absolute certainty? No. To use the word again, uncanny is still the best definition I have for it.

On the plural site, someone PM'd me for clarification on a critique I gave. In my reply, I mentioned (hopefully not too much like an accusation), "These two sentences sound very much like something AI generated."

In his reply, he said, "Oops, that was AI generated. Didn't mean to leave that in there." For the second sentence—apparently it wasn't—he got touchy and said, "I guess my writing seems like AI." He was clearly a new writer, which leads to another thought: if a writer is working closely with AI gen prose, even his own writing will probably start to resemble the current generations of generators. It also changes what "new" writing looks like, which is a shame. Where do you start when it comes to explaining AI's subtle incoherence? Is there any point? Which in itself feels like critiquing a machine instead of the writer's work anyway. They might just copy/paste it as feedback into their LLM and move on.
 
Last edited:
Probably the latter. But maybe not! And the fact that it's becoming increasingly harder to tell makes it all the more necessary to do your due diligence, and ensure that you're not falsely accusing people.

Almost everything between "Take these two" and "was generated at all" is AI, right?
 
And on that note, this come out in the TIMES today.

‘Femgore’ horror novel pulled over claims it was written by AI​

Publisher has withdrawn Mia Ballard’s Shy Girl from sale after detection software suggested it was 78% machine-made​


 
And on that note, this come out in the TIMES today.

‘Femgore’ horror novel pulled over claims it was written by AI​

Publisher has withdrawn Mia Ballard’s Shy Girl from sale after detection software suggested it was 78% machine-made​


Not saying I disagree with that particular case, since there is more to the story and she apparently had a bit of a rep for selling AI stuff...

But... AI detection is often very wrong as well. It seems it's no better at telling the difference than us.
 
Back
Top