Can you trust ChatGPT’s package recommendations?

3 years ago by activepeople to c/programming

ChatGPT can offer coding solutions, but its tendency for hallucination presents attackers with an opportunity. Here's what we learned.

“* People ask LLMs to write code

LLMs recommend imports that don't actually exist
Attackers work out what these imports' names are, and create & upload them with malicious payloads
People using LLM-written code then auto-add malware themselves”

load all comments

Spy 0 points 3 years ago

ChatGPT and similar LLMs don't really "know" anything. They can only predict what the answer should look like. This means that they can't be trusted for much and their answers should be reviewed before used, because anything they produce will sound correct by default.

save

path: 0 3612027, hotness: undefined, score: 0, children: 2

activepeople 1 point 3 years ago

and the devs copy+pasting code from it probably are aware of that it doesn't know anything, and that it is likely synthesizing something based on StackOverflow, which they used to happily copy+paste from a few months ago.

If the libraries ChatGPT suggests work ~80% of the time, this leaves an opportunity for someone to provide a "solution" the other 20%.

save

path: 0 3612027 3612029, hotness: undefined, score: 1, children: 1

Haus 1 point 3 years ago

This is pretty much my experience. It did a pretty good job with the grunt work of setting up a Qt UI in python, but something like 5/20 imports were wrong.

save

path: 0 3612027 3612029 3612031, hotness: undefined, score: 1, children: 0