Humiliated lawyers fined $5,000 for submitting ChatGPT hallucinations in court: ‘I heard about this new site, which I falsely assumed was, like, a super search engine’

3 years ago by L4sBot to c/technology

The judge scolded the lawyers for doubling down on their fake citations.

DeriHunter 44 points 3 years ago

So, they used CHATGPT to do their work, didn't validate it and and used made up cases to support theirs and when got cought, they lied and got only $5K fine? Wtf?

path: 0 541238, hotness: undefined, score: 44, children: 5

Rooki 11 points 3 years ago

Depending on the case. They should get jailed for that.

path: 0 541238 541816, hotness: undefined, score: 11, children: 4

Gradually_Adjusting 8 points 3 years ago

It should be several thousand per false citation, and disbarment for any repeat offense.

path: 0 541238 541816 543238, hotness: undefined, score: 8, children: 3

Rooki 4 points 3 years ago

But for example in a case where the other party could get in jail for his false citation? I would not be accepting yeah you could have gone to jail for this fake citations but he gets just pat on his hands with some cash money.

path: 0 541238 541816 543238 543548, hotness: undefined, score: 4, children: 2

Gradually_Adjusting 2 points 3 years ago

Maybe you're right. If we're trying to be thorough, I'd probably go as far as offering an incentive for firms to have a dedicated paralegal as an "AI reviewer".

path: 0 541238 541816 543238 543548 544302, hotness: undefined, score: 2, children: 0

curiosityLynx 1 point 3 years ago

In this case, the worst that could have happened if NOONE checked whether the cited cases were real and relevant AND the judge somehow decided that New York law stood above federal law and international treaties, would have been for an aviation company paying someone compensation for allegedly hurting a passenger and causing a bad knee.

path: 0 541238 541816 543238 543548 554040, hotness: undefined, score: 1, children: 0

ulu_mulu 31 points 3 years ago

which I falsely assumed was, like, a super search engine

A "super search engine" is still a search engine, if you're incapable of validating the results, or if you don't know you should, you shouldn't be a lawyer at all.

path: 0 543420, hotness: undefined, score: 31, children: 1

curiosityLynx 2 points 3 years ago

I mean, dude apparently didn't even know how to read a case reference and guessed Fd3 stood for Federal District 3 (spoiler: it's Federal reporter Edition 3). Not that I would have guessed any different, but I'm not and never was in any kind of law education, not to mention being a lawyer, who supposedly deal with case references all the time.

path: 0 543420 553921, hotness: undefined, score: 2, children: 0

Candelestine 23 points 3 years ago

The LegalEagle breakdown was thoroughly entertaining.

path: 0 538214, hotness: undefined, score: 23, children: 2

thapol 17 points 3 years ago

Link for those curious. Agreed, the breakdown does far more 'justice' to this story.

path: 0 538214 538905, hotness: undefined, score: 17, children: 1

JiFish 2 points 3 years ago

The best bit about the LegalEagle breakdown is the revelation that ChatGPT itself told them it shouldn't be used this way multiple times.

path: 0 538214 538905 544036, hotness: undefined, score: 2, children: 0

itsnotlupus 22 points 3 years ago

Court documents are at https://www.courtlistener.com/...

The transcript of the hearing where the judge grilled the lawyers won't be available to the public for another 2 weeks.

I feel like the lawyers are getting off really easy, considering.

They just have to pay $5k each and notify their client and every judge they "cited" in their made-up cases that they did an oopsy.

Oh and they lost the case, but it seems like that was foreshadowed long before the lawyers decided that ChatGPT was a court docket search engine.

path: 0 540828, hotness: undefined, score: 22, children: 0

DannyMac 13 points 3 years ago

Everyone wonders why ChatGPT is highly censored, this is a good example as to why. However, maybe instead of "As an AI language model" it should say something like, "Large language models like me tend to hallucinate/make up things and confidently convey them in my response. I will leave it up to you to validate what I say." The ultimate problem is the general public is treating LLMs like they are super sci-fi AI, they are basically fantastic autocomplete.

path: 0 548186, hotness: undefined, score: 13, children: 1

redcalcium 2 points 3 years ago

Super autocomplete doesn't sound as appealing as "AI". The general public need to know that though so they can adjust their expectation. For example, it doesn't make sense to expect an autocomplete system to solve complex math problems (something people unexpectedly use ChatGPT for).

path: 0 548186 555067, hotness: undefined, score: 2, children: 0

Fubarberry 9 points 3 years ago

Even if you thought it was just a search engine, it's hard to imagine citing a case without independently validating it first.

path: 0 537780, hotness: undefined, score: 9, children: 1

wjrii 9 points 3 years ago

Here's the thing, even if you had zero intention of actually reading a case, there are STILL next steps once you get a cite. There is an entire "skill" you're taught in law school called Shepardizing (based on an older set of books that helped with this task) where you have to see if your case has been treated as binding precedent, had distinctions drawn to limit its applicability, or was maybe even overturned. Back when I was learning, the online citators would put up handy-dandy green, yellow, and red icons next to a case, and even the laziest law student would at least make sure everything was green before moving on in a Shepardizing quiz without looking deeper. And even THAT was just for a 1-credit legal research class.

These guys were lazy, cheap (they used "Fast Case" initially when they thought they had a chance in state court; it's a third-rate database that you get for free from your state bar and is indeed often limited to state law), and stupid. They didn't even commit malpractice with due diligence. I can only assume that they were "playing out the string" and extracting money from their client until the Federal case was dismissed with prejudice, but they played stupid games and won stupid prizes.

path: 0 537780 540549, hotness: undefined, score: 9, children: 0

OasisStorm 7 points 3 years ago

Surprised it was only $5,000

path: 0 537288, hotness: undefined, score: 7, children: 0

MargotRobbie 6 points 3 years ago

Here lies the problem, ChatGPT is not a search engine, instead you can think of it as a compressed JPEG of the Internet (Credits to Ted Chiang). It can get you things that LOOK right if you squint your eyes a bit, but you just can't be sure that it is not just some random compression artifacts.

The problem is that OpenAI is hyping ChatGPT up as something that it is not.

path: 0 555952, hotness: undefined, score: 6, children: 0

Web_Rand 6 points 3 years ago

Paywall.

Here's the article on archive.is: https://archive.is/aQhso

path: 0 538590, hotness: undefined, score: 6, children: 0

kredditacc 6 points 3 years ago

Hallucination is also why I don't use AI to write code for me. I either have to check for hallucinations and fix them, or accept wrong results.

path: 0 540631, hotness: undefined, score: 6, children: 2

Rooki 3 points 3 years ago

I wont let ChatGPT write 100 lines of code but with github copilot its somewhat good occasionally.

path: 0 540631 541857, hotness: undefined, score: 3, children: 1

kredditacc 2 points 3 years ago

You still have to check if it falls into the "somewhat good occasionally" category.

path: 0 540631 541857 550333, hotness: undefined, score: 2, children: 0

cloaker 5 points 3 years ago

Should be more

path: 0 539261, hotness: undefined, score: 5, children: 0

paperlama 2 points 3 years ago

I wonder if something like this might get overlooked one of these days.

path: 0 543959, hotness: undefined, score: 2, children: 3

cedarmesa 4 points 3 years ago

💀

path: 0 543959 545388, hotness: undefined, score: 4, children: 0

wmassingham 1 point 3 years ago

What makes you think it isn't already?

path: 0 543959 547039, hotness: undefined, score: 1, children: 1

ech0 2 points 3 years ago

I mean I think the above article is a pretty good indicator...

path: 0 543959 547039 547188, hotness: undefined, score: 2, children: 0

jerome 1 point 3 years ago

Again?!
They need to stop doing that.

path: 0 538758, hotness: undefined, score: 1, children: 0

technology

@lemmy.world

login for more options

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

go to feed...

technology

@lemmy.world

login for more options

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

go to feed...