As I explain in my post (linked here) I asked my friends to post a bit of writing, either from them, from a friend, or created by AI. I'd then use Edward Tian's AI language detector to do some detectoring.
Results were . . . mixed.
This experiment is hardly conclusive, but it's clear Tian's AI detector has trouble with "creative" writing (most of my friends who participated are published authors, in the publishing business, or extremely well-read). The detector consistently failed to differentiate between "creative" AI writing and human-originated writing. It got it right a few times, but more often than not, it failed.
Admittedly, some of the samples included were a bit short for the AI to chew on -- but there were bits in the middle-range of length that the detector basically said "I don't know. Not enough information here," while it confidently identified shorter texts as AI- or human-generated, both right and wrong.
What does this mean? I don't know. But it's interesting. I'll give it that.
No comments:
Post a Comment