They’re not. GPTZero has a false positive rate under 1% for creative writing.
They’re not. GPTZero has a false positive rate under 1% for creative writing.
Worst performance for GPTZero is technical medical papers with a 10% false positive rate. Here’s the study from S. Korea.
Scientific scholarly writing: <4% false positive.
Creative writing is orders of magnitude easier to detect. The biggest problem for AI writing detection tools is the use of translation — so generating text, translating it into another language, and translating it back — but this ends up as a false-negative issue. False positives are very rare.
More importantly, we don’t need GPTZero to detect AI writing because professionals can identify it on sight. Add context clues (students are dumb) and suddenly it’s trivial.
EDIT: I need you to understand that college students are terrible writers. In all my decades and literally thousands of students, I’ve met maybe three undergrads whose writing was more than proficient, and that's in an environment where the average SAT scores are over 1550. The idea I’m going to mistake your writing for AI (which sounds horrible and robotic, btw) because of its “high quality” is simply unfathomable.
Props for responding constructively.
The false positive rate is pretty low, but a false negative of 35% is pretty concerning in terms of whether they're fit for purpose. Like it'll rarely misclassify human writing as AI generated, but it's somewhat closer to a coin flip in correctly identifying AI generated language. It's a signal, but I imagine it's a poor substitute for the judgement of a person who has spent time teaching teenagers / young adults to write.
Which to the original point isn't totally snake oil, but it's maybe not a great tool for the professionals who it's targeting.
Thank you for linking me to that study! Unfortunately google is a piece of shit and they’re calling me a bot and refusing to talk to me even though I’m on a standard residential internet connection. Could you tell me the title/authors so I can look for it elsewhere?
I’m particularly interested in seeing what the actual false positive rates are beyond a colloquial “1%”. Because 1% is, in the context of accusing academic cheating, absolutely unacceptable dog shit.
Take a basic example intro class with 350 students, with only two tests per semester. With “only” a 1% false positive rates you will be destroying the lives of more than a dozen innocent kids every school year. And thats only for one class, students are taking many classes per semester.
It's literally from a Chick Tract. It's an ancient meme... circulating for the past 10 years or so.
Whether the person posting it is right or wrong, it's not really an endorsement of religion. It's like "scapegoat", or "omg".
someone disagrees with you on the internet? just lie! they arent going to check anyway and everyone believes lies on the internet.
Your writing reflects a higher than normal degree of literacy. Ergo, AI use detected.
That, or my man really loves a bullet point and an emdash.
I love using emdashes in my papers, but I stopped because of AI
It’s fine, I use em dashes all the time in papers. If you do it correctly—to add more information or commentary to your overall point—instead of at random and without proper syntax you’re not going to get accused.
Don't let LLMs take your em dashes; flattening your language to the lowest common denominator to avoid doing anything too distinctive less you be accused of non-authorship.. that's a real harm to you and your reader. If you know how to use them, and enjoy using them, use them fearlessly.
I am in this picture and i dont like it
It’s actually the opposite. AI writing sounds like corpo-speak. It’s soulless, ugly, and extremely obvious.
I’ve spent about 30,000 hours reading in my life. If you think I can’t tell you’re using a token predictor for your “essay,” you’re fucking insane.
As someone who has retired from the practice of law and billed clients more than 30,000 hours for legal research over the years, I can assure you that soulless corpo-speech has been around far longer than AI and used to be created organically by depressed human drones and sociopaths. Where do you think the AIs learned to write that way?
This is why I had to quit school.
No, it's cause I was apparently one of the few in my class who actually proofread anything before submitting assignments
Personally, if I had that shit pulled on me, I would probably demand to know their process of grading papers. Because from what I've been reading, a lot of teachers and lecturers are starting to use AI tools to assist them.
And as far as I'm concerned, if I have paid an absolute fucking fortune to be there and be taught by human beings, I'm expecting them to actually do their fucking jobs and not hand it off to Elon Musk's plagiarist CP machine. Especially if they're going to be going around accusing students of using AI.
My mother is a professor and hates AI. She works at a community college and the amount of shit she has to go through is unbelievable. Her college actively pushes her to use AI but she doesn't whenever possible. She's struggling with AI more than any other person I know of.
Teachers are rarely the problem.
I'm starting a master's in a few weeks, and because most of the courses involve group work, I'm concerned about someone in my group sinking a project for everyone else by using an LLM for their portion.
And given the program, I wouldn't be surprised if there are more than a few AI boosters in the courses who will try it, above the normal number of such people you might find in postgrad studies.
I never had a problem with ratings over 50%. My writing seems to be especially prone to false positives, but I still didn't receive higher than 20-30%. Unfortunately, at my school, anything over 0% was considered suspect, so I was still penalised even if it was clearly human-written with full APA sourcing.
I was worried about this as well, but I’m in undergrad and have run into zero issues. When teachers allow us to see the percentage similarity to AI text I’ve always had under 10%, usually under 5% if you omit direct quotations. I can’t imagine it gets more likely you’ll get dinged as your subject gets more arcane. The only time I’ve seen people in trouble for it are for really obvious plagiarism or AI use.
At a art exhibit, this poor artist with over 20 years of work had to put a "Not AI" sticker. He was sick of all the dick heads.
Nobody who reads or writes for a living thinks AI slop sounds good. It has a hideous corpo-speak prosody that’s like nails on a chalkboard to professionals.
People think because they can’t tell it’s not written by a human, someone who has read literally tens of thousands of essays (not to mention millions of fucking book pages) can’t tell either.
Absolutely bonkers.
Students who use AI are like little kids standing on each others shoulders in a trench coat and trying to order alcohol.
I’ve gotten this a lot. I was also on early Reddit a lot so I guess AI probably just writes like I do, not the other way around
That's interesting. I was permabanned in the Inauguration Bloodbath after 12 years, and a million karma. I was VERY active, back when nobody had really heard of Reddit. Because my end was sudden, I never had the chance to delete anything, so they got it all.
Same boat
Oh my, that's certainly something that could have happened to me. When I deleted my account I nuked all of my posts and comments with overwriting though.
The old witchcraft trial.
If the paper is so badly written that it fails then it will be declared AI-free. (If she drowns then she wasn't a witch)
Pry those em dashes from my cold, dead hands, Louise.
There are ways of telling whether she is an AI.
What are AIs mode out of?
Silicon!
What else is made out of silicon?
Breast implants!
So logically if she weighs the same as a breast implant...
I was gonna say they're: "Spelled differently (true) and are distinct, unrelated materials (false!)". So I ended up correcting myself.
Silicone is literally made from silicon. Manufacturing starts with silica (SiO₂, found in sand/quartz), which is reduced to elemental silicon, then reacted with methyl chloride to form methylchlorosilanes, which are then polymerized into silicone.
The name "silicone" was actually coined in 1901 by English chemist Frederick Kipling, who thought the compounds R₂SiO resembled ketones (hence "-one" suffix) — the naming turns out to be a bit of a misnomer chemically, but it stuck.
Who are you, who are so wise in the ways of science?
It's like being accused of aimboting in a competitive shooter. I just take it as a compliment.
I joined a counterstrike server for newbs to practice with bots. Apparently people thought I was a bot because I sucked so badly.
Well, they're now training AI on reddit, so...
Emagine being in an academic filed, and AI being trained off your research papers, so it has the same writing style as you. Give me my em dashes back.
It's like when I used to play Black ops and battlefield 3, eventually because it wad the summer after I graduated, I basically played all day.
Eventually I was getting accused of using hacks for about 25% of my games.
The kicker: I wasn't particularly good based on the scoreboard, but after you kill the same guy 3 times through a wall because you can hear his footsteps with your headset and just spray through the wall, he gets suspicious (and probably doesn't play nearly as much)
I actually was in the top 50 worldwide for MW3 for pistol kills for awhile because for about 2 months all I did in that game was dual wield pistols, sprint after people in games that are more close quarters and go to town on my triggers.
I always liked being accused of hacking, because it made me feel like I wasn't garbage at the games.
Now I'd probably get accused of being a default bot lmao
Someone I work with who always hand writes his code used copilot to write his commit message as usual this morning.
Lo and behold it had snuck an "authored by copilot" tag onto the end of the message and github picked up on it and labeled the commit accordingly - when he saw "and copilot" next to his name on github he flipped his lid and ranted about how copilot only exists on the back of stolen code from humans and it has no right to claim any credit whatsoever to anything original like being an author ever.
He edited out the attribution and force pushed it before anyone else could commit on top of it.
(Apparently copilot writes commit messages in such a way that no matter how trivial the edit, it sounds like he'd been planning and working for hours and hours on it.)
I read that Microsoft and vscode had a bug where for a few days, all code was accidentally "authored by copilot".
I used to be extremely careful about my grammar and spelling. I never wanted to be called out for using the wrong there or forgetting punctuation.
after Ai though, i really don't care. At least im less likely to be taken for a robot.
Excuse me it's I'M or I am, not im.
What are you, some kind of low ram ai model?
😅
Well, one thing corporate AI won't do is tell you to eat a bag of dicks.
Soon, being an asshole will be the tell, that you're talking to a meatbag online!
/s
Humans can be a jerk to your face. But corporate AI will just suck it up, and quite literally suck it up for real, which would be the nearby river.
Pretty sure that's also the 3rd American red scare we've got going on right now with the whole guilt by association, put antifa in prison for 580 years.
i was shocked when my article on Adorno and Scruton comparison got rejected because "it reeked AI" even though AI sucks VERY-VERY HARD at analyzing and interpreting philosophy beyond NotebookLM literally retelling the uploaded book. it is so bad there is basically an entire subgenre of philosophy writing analyzing how various AI get it wrong
The more rules,the more AI the text becomes. So I expect a scientific text with very rigit rules in how to write is practically indistinguishable from an AI text, if written good. So tue expected value is 50% AI or human, if the text is well written. Either the AI got less rules (making it more AI) or humans make mistakes. Either way, a perfectly written text will land at 50% chance.
@lemmy.world
All posts need to have the same title: me_irl it is allowed to use an emoji instead of the underscore _
go to feed...
@lemmy.world
All posts need to have the same title: me_irl it is allowed to use an emoji instead of the underscore _
go to feed...
It's really just that AI detectors are snake oil
save