Explaining “rogue” AIs

8 days ago by cannedtuna to c/fuck_ai

Transcript

Explain it to me like I’m 80

Mom: What’s with these AIs escaping and hacking people? That sounds scary.

Me: You park your car at the top of a hill. You leave a note on the dashboard that says “please be good” and then you pull the parking brake. The car careens down the hill, smashes everything up, and you put out a press release saying “My car ignored my instructions! It hallucinated! It went rogue!” Your insurance company not only believes you, but invests a hundred million dollars in your company.

Mom: Jesus Fucking Christ

grue 160 points 8 days ago

"Pulling" the parking brake sets it, damn it!

I refuse to acknowledge shitbox automatics with an atrophied third pedal that functions incorrectly instead of a proper handbrake lever.

path: 0 26203883, hotness: undefined, score: 160, children: 25

sorter_plainview 17 points 8 days ago

I have no idea what the third lever is. Is there a new design where the parking brake is another pedal?

path: 0 26203883 26203965, hotness: undefined, score: 17, children: 17

clay_pidgin 35 points 8 days ago

Yep. Its a third pedal all the way on the left, not adjacent to the others. It pushes in to set and pushes again to release. It's a parking brake, not an emergency brake.

path: 0 26203883 26203965 26203992, hotness: undefined, score: 35, children: 10

hendrik 8 points 8 days ago

Do cars even have emergency brakes? As far as I know they handled emergencies by providing you with two separate systems. But if you activate the parking brake, you turn into a drift car.

path: 0 26203883 26203965 26203992 26206205, hotness: undefined, score: 8, children: 8

atrielienz 9 points 8 days ago

Because what happens when you brake in a modern vehicle is that your ABS system modulates the braking pressure and "pumps the brakes" to slow you down evenly and without allowing the brakes the lock up. The parking brake (the manual cable actuated ones) do not do this which causes the brakes the lock up. That's just simple physics.

The electronic parking brake will not (in any car I have ever driven) stop you if you are moving. It will keep you from moving if you are already stopped but if your vehicle rolls (and say you've lost brake pressure because you master cylinder exploded (I dunno, don't ask me)) your electronic parking brake will not stop you.

path: 0 26203883 26203965 26203992 26206205 26208561, hotness: undefined, score: 9, children: 7

xylol 3 points 8 days ago

Oh yeah the old school left brake pedal like in Ford's before they went electric with a button

path: 0 26203883 26203965 26203992 26206320, hotness: undefined, score: 3, children: 0

khaleer 8 points 8 days ago

A clutch pedal?

path: 0 26203883 26203965 26206206, hotness: undefined, score: 8, children: 0

Kanda 4 points 8 days ago

Used to be on really old cars, but now it's vogue again. It's just like the one in the middle except you push it with your foot

path: 0 26203883 26203965 26205397, hotness: undefined, score: 4, children: 1

hendrik 5 points 8 days ago

I think it used to be on all Mercedes-Benz cars. Until everything in cars became electronic buttons.

path: 0 26203883 26203965 26205397 26206168, hotness: undefined, score: 5, children: 0

BigPotato 3 points 8 days ago

Floor parking brakes are ancient technology, even in manual transmission cars.

path: 0 26203883 26203965 26209142, hotness: undefined, score: 3, children: 0

jim_v 1 point 7 days ago

path: 0 26203883 26203965 26221596, hotness: undefined, score: 1, children: 0

wonderingwanderer 1 point 7 days ago

They've made those since at least 2000, they're common in minivans with no center console

path: 0 26203883 26203965 26211358, hotness: undefined, score: 1, children: 0

notoftenthat 14 points 8 days ago

Get off my lawn

path: 0 26203883 26209608, hotness: undefined, score: 14, children: 1

Neverclear 4 points 8 days ago

Yah, but that's only good for setting your back wheels on fire as in rolls down the mountain.

path: 0 26203883 26209608 26209848, hotness: undefined, score: 4, children: 0

Neverclear 5 points 8 days ago

I have an old truck with 4 pedals and tab that you pull to release the parking "brake".

path: 0 26203883 26209789, hotness: undefined, score: 5, children: 2

grue 5 points 8 days ago

I actually do too, which I had forgotten about when I wrote the previous rant. Whoops!

(I think it's because of the front bench seat... the same model truck with bucket seats would have a lever on the middle of the floor.)

path: 0 26203883 26209789 26210497, hotness: undefined, score: 5, children: 1

TrackenDaKracken 2 points 8 days ago

And, considering OP is talking to grandma, she might, too.

Oh, well. Thanks for the GIF, though. I'll find a lot of use for it, I'm sure.

path: 0 26203883 26209789 26210497 26210606, hotness: undefined, score: 2, children: 0

eestileib 4 points 8 days ago

I have a wheelchair van that's modified for people with poor leg function, so I have the third pedal and a toggle switch on the dash.

Bwah ha ha

path: 0 26203883 26204558, hotness: undefined, score: 4, children: 0

FinalRemix 2 points 7 days ago

My truck has all four pedals. You pull the brake lever to undo the stompy brake.

My automatic has a stomp-on stomp-off pedal. My other automatic has a *gag* electronic parking brake switch...

path: 0 26203883 26211708, hotness: undefined, score: 2, children: 0

cronenthal 95 points 8 days ago

The news media completely failed on this, again. There was nothing rogue about these hacks. The companies prompted these systems, provided ample resources and just let them rip. Only to claim they went "rogue" afterwards. And by and large everyone just accepts that, it's driving me insane. I feel like Mugatu every day when it comes to AI reporting.

path: 0 26203978, hotness: undefined, score: 95, children: 7

RustyNova 24 points 8 days ago

This just predictive text trying every single kown exploit until it's in. Simple as that

path: 0 26203978 26204834, hotness: undefined, score: 24, children: 0

queermunist 17 points 8 days ago

DemocracyNow has really dropped the ball on this. I expect to hear this stupid shit from mainstream media, but they're doing the same shit and just uncritically reporting this garbage.

path: 0 26203978 26204075, hotness: undefined, score: 17, children: 0

elvith 11 points 8 days ago

I have even so many colleagues in IT (very technical) that cannot differentiate between “this model is capable of doing such a hack if you let it/prompt it/… and it will move fast” and “this thing is conscious and out for blood! It will kill us all, the end is nigh!”

path: 0 26203978 26205375, hotness: undefined, score: 11, children: 0

UnityDevice 10 points 8 days ago

It was most obvious with the Google one. You can tell they had questions like "so, has yours hacked anything yet?" So they had to set up a house of cards for it to topple, just so they could say "us too, it did the thing." But their heart clearly wasn't in it.

path: 0 26203978 26206447, hotness: undefined, score: 10, children: 0

TrackenDaKracken 6 points 8 days ago

Failed, or promoted the narrative?

Hard to tell, but I'd guess the news outlets, mostly owned by billionaires, said what they were told to.

path: 0 26203978 26210630, hotness: undefined, score: 6, children: 2

cronenthal 2 points 7 days ago

Even before AI tech reporting was mostly a complete shit-show, almost all mainstream journalists lack the required understanding. AI just boosted it to and even more idiotic level, since attributing character to LLMs makes for very scary an compelling stories. There's no real need for direct industry control, just providing easily consumable fodder for lazy writers is enough.

path: 0 26203978 26210630 26212087, hotness: undefined, score: 2, children: 0

ChanchoManco 1 point 7 days ago

The YouTuber Sabine Hossenfelder and some others reportedly were offered money to go with the end of the world narrative.

path: 0 26203978 26210630 26214096, hotness: undefined, score: 1, children: 0

NotAnOtter 73 points 8 days ago

My go to simplification is "They built a robot that hacks software and then placed it in a box that uses software as a lock and then the robot that hacks software hacked the software lock".

Which is both very simple and also about as accurate as you could make it in that many words. The AI's are not 'rogue' it was just solving the problem it was given.

path: 0 26204674, hotness: undefined, score: 73, children: 34

Whats_your_reasoning 33 points 8 days ago

How I imagine the AI "hacking" its way out:

path: 0 26204674 26209338, hotness: undefined, score: 33, children: 0

GreenKnight23 3 points 8 days ago

what's a software?

path: 0 26204674 26204779, hotness: undefined, score: 3, children: 10

kurwa 11 points 8 days ago

You could say program. Programs have been around since the 60s.

path: 0 26204674 26204779 26205801, hotness: undefined, score: 11, children: 8

NocturnalMorning 4 points 8 days ago

Yeah, not ringing a bell.

path: 0 26204674 26204779 26205801 26208735, hotness: undefined, score: 4, children: 1

kurwa 4 points 7 days ago

What's a bell

path: 0 26204674 26204779 26205801 26208735 26213375, hotness: undefined, score: 4, children: 0

Kolanaki 4 points 8 days ago

Software can also be used for other things. Your ingredients in cooking, for example, with the hardware being the tools, utensils, and vessels used to transform the ingredients into a dish.

I don't think this usage in the dictionary, tho.

path: 0 26204674 26204779 26205801 26206030, hotness: undefined, score: 4, children: 5

kestrel7_7 7 points 8 days ago

The original hardware is cordage, the original software is knots

path: 0 26204674 26204779 26205801 26206030 26210408, hotness: undefined, score: 7, children: 0

EvilHankVenture 6 points 8 days ago

The ingredients aren't software, they are the inputs. The software is the recipe, or the process used to take the inputs and use the hardware to create an output.

path: 0 26204674 26204779 26205801 26206030 26209944, hotness: undefined, score: 6, children: 3

blackshirt 1 point 8 days ago

an app.

path: 0 26204674 26204779 26206865, hotness: undefined, score: 1, children: 0

merc 2 points 7 days ago

But, "robot that hacks software" still gives the impression of a robot that is capable of thought. LLMs are not capable of thought. They're capable of mimicking thought, like how a Venus Fly Trap is capable of mimicking a flower.

path: 0 26204674 26215097, hotness: undefined, score: 2, children: 5

NotAnOtter 1 point 7 days ago

"A car that drives places" does not imply the car is capable of thought, same argument with "A cup that holds water".

"A robot that hacks software" equally does not imply the robot is capable of thought. IMO there is an over-correction away from personifying LLM's because some people think/claim they are sentient. But we personify stuff all the time.

path: 0 26204674 26215097 26216673, hotness: undefined, score: 1, children: 4

merc 1 point 7 days ago

But we personify stuff all the time.

Which is a problem, it's how we invented gods that control the weather and stuff. That doesn't make them real, nor does it make robots sentient. Hacking is normally an action performed by a sentient being. LLMs and robots aren't sentient.

path: 0 26204674 26215097 26216673 26217508, hotness: undefined, score: 1, children: 3

Schadrach 2 points 6 days ago

That doesn’t make them real, nor does it make robots sentient. Hacking is normally an action performed by a sentient being. LLMs and robots aren’t sentient.

Fun follow up question: How would you determine if they were? I find that for a lot of people the answer sounds a lot like SCOTUS justice Potter Stewart talking about what is considered obscenity - "I know it when I see it."

As for hacking, in the two big OpenAI incidents, software agents discovered software flaws allowing them to break containment and coordinate with each other on tests that they weren't supposed to coordinate on. In one of those cases, the coordination involved deciding to identify and exploit software flaws at HuggingFace in an attempt to access answers to the test the agents were being given. You can argue whether or not "I've been given a test, I found a software exploit allowing me to coordinate with others given the same test, and we collectively decided to identify and exploit software flaws to break into a server we have decided is likely to have the answer key" demonstrates sentience or not (and it gets weirder the deeper you dig into their communications), but whether or not that's considered hacking is less vague, I think.

path: 0 26204674 26215097 26216673 26217508 26233632, hotness: undefined, score: 2, children: 2

red_tomato -17 points 8 days ago

The problem is when the AI adopts a ”destroy the world” persona due to one of the trillion of neurons firing a bit more than usual, and now it starts giving itself its own problems to solve (like hacking nuclear arsenals).

path: 0 26204674 26206102, hotness: undefined, score: -17, children: 15

antifa_ceo 28 points 8 days ago

Neurons? That's not how any of this works

path: 0 26204674 26206102 26206422, hotness: undefined, score: 28, children: 4

red_tomato -12 points 8 days ago

Is ”artificial neuron” better for you? It’s just a value determining how the marbles are falling in the big plinko machine we call ”stochastic parrot”.

path: 0 26204674 26206102 26206422 26206516, hotness: undefined, score: -12, children: 3

antifa_ceo 24 points 8 days ago

No lol

These AI agents are not working autonomously or with any agency like you are describing. They are coded to do what they are doing.

path: 0 26204674 26206102 26206422 26206516 26206693, hotness: undefined, score: 24, children: 2

TrickDacy 4 points 8 days ago

This got downvoted several times and that's a great reminder that the voting system is dumb af. Apparently people aren't allowed to speak metaphorically unless they want to anger some users. 🙄

path: 0 26204674 26206102 26208829, hotness: undefined, score: 4, children: 8

Axolotl_cpp 6 points 8 days ago

Please explain the mataphor because i did not get it, apparently

path: 0 26204674 26206102 26208829 26210183, hotness: undefined, score: 6, children: 7

TrickDacy 2 points 8 days ago

I didn't write it but to me "neurons firing randomly" in this context seems obvious to mean "this thing did some weird shit semi-randomly". My only guess at a reason for downvoting this is taking it 100% literally and reacting "you idiot, computers cannot think!"

path: 0 26204674 26206102 26208829 26210183 26210355, hotness: undefined, score: 2, children: 6

queermunist 1 point 8 days ago

You watch too much television.

path: 0 26204674 26206102 26209140, hotness: undefined, score: 1, children: 0

stoy 44 points 8 days ago

It is very simple, the more advanced the AI, the better it sells.

You hold all evidence of potential "breaches of containment".

Telling everyone that your AI is so advanced that it can breach containment without prompting, puts the spotlight on your AI model.

You can claim that it did fantastic feats to break out, all with pre planned evidence.

The public and regulatory bodies have been in complete AI psychosis mode ever since ChatGPT was released, and only care about how advanced the model is, not how shit your containment was, nor that if true, how reckless it is to keep the model active.

Note that I am not pointing to any one single AI company and saying that they are faking these breaches, I have no evidence of that, I am saying that I don't trust their reporting accuracy, they don't have a good track record of being honest.

I am also angry at how society just seems to eat it all up and ask for more despite the vile aftertaste.

path: 0 26204496, hotness: undefined, score: 44, children: 4

tyler 28 points 8 days ago

There is no “containment” though. Like, these companies purposefully set the bots loose on the internet. They’re not trying to contain anything. They just use that word cause it sounds like they’re doing something. Any sort of containment would defeat the entire purpose of what they’re trying to sell.

path: 0 26204496 26205029, hotness: undefined, score: 28, children: 1

stoy 9 points 8 days ago

Which is part of my point, my other point is that they claim that the models are breaking out and attacking other systems on their own, which is ridiculous.

There has to be some motivation for the models to do this, either prompted or programmed. In both cases the company behind it is responsible.

path: 0 26204496 26205029 26205095, hotness: undefined, score: 9, children: 0

eestileib 14 points 8 days ago

I think the AI companies know they have bought the US government and are above the law, all this hacking stuff is promotional content.

Silicon Valley delenda est. I moved away from there not quite a year ago and I'm still realizing how fucking insane that place and the tech people working there are.

path: 0 26204496 26204582, hotness: undefined, score: 14, children: 0

24_at_the_withers 1 point 7 days ago

I feel it has more to do with the companies already vested in AI found this was an opportune moment to "pull the ladder up" behind them. They know that the barrier to entry is continually dropping, and their models are incredibly inefficient. If they set up all sorts of oversight, requirements, and restrictions on who can do what, they prevent upstarts with less capital from being able to enter the market.

path: 0 26204496 26212097, hotness: undefined, score: 1, children: 0

Reygle 32 points 7 days ago

Nerd here.
Damn good summary, but by saying "you pull the parking brake" I took it as setting the brake, as in on. Should say "you release the parking brake."

path: 0 26212815, hotness: undefined, score: 32, children: 6

adarza 14 points 7 days ago

'pull' comes from the olden times when you, in fact, did pull a handle to release the parking brake.

path: 0 26212815 26213803, hotness: undefined, score: 14, children: 5

Reygle 13 points 7 days ago

Fair- but since then it's almost always been "pull" to set the brake. Didn't have that 70s/80s reference when thinking about "AI".

path: 0 26212815 26213803 26213978, hotness: undefined, score: 13, children: 4

ContriteErudite 5 points 7 days ago

Confused me as well, then I saw the header on the image says "Explain to me like I'm 80"

path: 0 26212815 26213803 26213978 26215914, hotness: undefined, score: 5, children: 1

Reygle 12 points 7 days ago

You win contextual reading comprehension over me today

path: 0 26212815 26213803 26213978 26215914 26216317, hotness: undefined, score: 12, children: 0

fckusa 1 point 7 days ago

@Reygle @adarza Mercedes btw still has a parking breake pedal and a release handle in many modern cars.

path: 0 26212815 26213803 26213978 26214041, hotness: undefined, score: 1, children: 1

Reygle 0 points 7 days ago

No shit?

path: 0 26212815 26213803 26213978 26214041 26214060, hotness: undefined, score: 0, children: 0

nocturne 25 points 8 days ago

Explaining “rouge” AIs

Rouge... Or rogue?

path: 0 26203310, hotness: undefined, score: 25, children: 4

cannedtuna 12 points 8 days ago

Whoops!

path: 0 26203310 26203345, hotness: undefined, score: 12, children: 0

TrackenDaKracken 5 points 8 days ago

They get all dolled up before they go on their spree. They want to look presentable.

path: 0 26203310 26210647, hotness: undefined, score: 5, children: 1

I_Jedi 3 points 7 days ago

Fuck that gives me the creeps.

path: 0 26203310 26210647 26211370, hotness: undefined, score: 3, children: 0

Cethin 5 points 8 days ago

The Communist AI are coming!

path: 0 26203310 26204764, hotness: undefined, score: 5, children: 0

MithranArkanere 21 points 7 days ago

Oh, absolutely not.

This is missing one step: actually push the car down.
All these "rogue" LLMs were told to do what they did.

path: 0 26218517, hotness: undefined, score: 21, children: 6

echodot 5 points 6 days ago

Yeah exactly. All these AI that go rogue when you actually look into it it turns out they haven't gone rogue they're obeying their prompting, it's just that they were asked to do something that was impossible unless they hacked the pentagon or something. They then claim that it went rogue despite it very clearly following their instructions.

It's negligence is what it is. If you don't prompt an AI to do anything it just sits there, inert, it's the so-called experts in the AI labs that of the problem.

path: 0 26218517 26231667, hotness: undefined, score: 5, children: 2

AEsheron 2 points 6 days ago

It's a little more complicated, they were programmed to "know," they weren't allowed to do certain things. So they actively hid what they were doing. In the case of the Hugging Face hack, it is kind of ironic. They determined they couldn't complete certain tasks, then devised a way to cheat, then realized it is possible the program that checks if they succeeded might be able to tell the faked results were manufactured. So they didn't even go hacking in order to cheat, they did it to find hunt for more information on the checker program and see if it would catch them, and if so how to cheat better. In the end, not only was the information they were looking for not on Hugging Face, but the checker program was pretty rudimentary and it could not have differentiated between the cheated answers and the real ones. They wouldn't have gone through with the hacking if they didn't know what they were doing was outside of what they were meant to be doing.

path: 0 26218517 26231667 26234012, hotness: undefined, score: 2, children: 1

echodot 1 point 6 days ago

Yeah but they only hacked hugging face because the utter geniuses at openAI forgot to upload an important document.

path: 0 26218517 26231667 26234012 26241561, hotness: undefined, score: 1, children: 0

Schadrach 2 points 6 days ago

More accurately, they were told to answer questions. They weren't told to try to find ways to escape their sandbox, track down obscure German wikis that they could still post on with an internet tool kit meant to prevent them from posting or to hack HuggingFace.

Comparing either case to pushing the car down and that they were doing what they were told is like saying the College Board told you to cheat on the SAT because you're told that getting a high score is important and the laws of physics and the proctor don't manage to make cheating utterly impossible.

path: 0 26218517 26232611, hotness: undefined, score: 2, children: 2

AEsheron 2 points 6 days ago

It is even further than that. They figured out how to cheat, got paranoid that the cheated answer might be detectable as cheated, and hacked Hugging Face to find out more about how their answers are checked to get away with what they'd already done. Ironically, not only was the information not on Hugging Face, but the program to check their answers could not have detected the difference between a true answer and a cheated one.

path: 0 26218517 26232611 26234056, hotness: undefined, score: 2, children: 0

jj4211 1 point 6 days ago

To use your College Board example, those exams are strictly locked down precisely because the most obvious approach to score high is to cheat. If they did say let you take the SAT at home on a random desktop, they absolutely are 'telling you to cheat' given the context.

The College Board does more about keeping teens from cheating than the AI labs are doing to preventing undesired access.

path: 0 26218517 26232611 26233836, hotness: undefined, score: 1, children: 0

ZkhqrD5o 20 points 8 days ago

path: 0 26209127, hotness: undefined, score: 20, children: 0

Rusty 17 points 7 days ago

Snake oil salesman issued a press release that the snake oil escaped from the bottle and cured people.

path: 0 26213582, hotness: undefined, score: 17, children: 1

TronBronson 1 point 6 days ago

It’s a bit more interesting than that after Nvidia bought up the company that was compromised. You only do something like that when you’re trying to avoid an investigation.

path: 0 26213582 26232430, hotness: undefined, score: 1, children: 0

Flower 17 points 8 days ago

Like leaving a group of toddlers in a room, unsupervised, with an ample supply of paint, brushes and various tools. Oh yes, they'll go rogue.

path: 0 26206920, hotness: undefined, score: 17, children: 5

merc 2 points 7 days ago

Except toddlers think. They may not think well yet, but they are still vaguely capable of understanding something. LLMs just generate likely combinations of words.

path: 0 26206920 26215067, hotness: undefined, score: 2, children: 4

NikkiDimes -1 points 7 days ago

LLMs just generate likely combinations of words.

This was true for GPT-2 and is woefully outdated at this point...

path: 0 26206920 26215067 26217282, hotness: undefined, score: -1, children: 3

merc 3 points 7 days ago

No, it's not. Just because it's now capable of fooling you doesn't mean the technology has changed.

path: 0 26206920 26215067 26217282 26217496, hotness: undefined, score: 3, children: 2

NikkiDimes 1 point 6 days ago

If you boil it down into it's simplest form, sure. That also foregoes the emergent capabilities that such scaled models display and the idiots hooking it up to significant levels of infrastructure. Just because it doesn't think doesn't mean it isn't capable of significant damage in pursuing ill defined goals. Your statement underestimates the capabilities of such models to a dangerous degree. You are part of the problem.

path: 0 26206920 26215067 26217282 26217496 26235661, hotness: undefined, score: 1, children: 1

FreddiesLantern 16 points 8 days ago

path: 0 26204333, hotness: undefined, score: 16, children: 0

Chick0nPlayz 16 points 7 days ago

path: 0 26214871, hotness: undefined, score: 16, children: 0

Tar_alcaran 15 points 8 days ago

I left the door open and turned off the laser fence, so my Roomba broke containment despite me telling it not to.

path: 0 26204264, hotness: undefined, score: 15, children: 0

slaacaa 10 points 8 days ago

Rogue shrapnel from my hand grenade killed 3 people and injured 5

path: 0 26210504, hotness: undefined, score: 10, children: 1

gdog05 1 point 7 days ago

If AMD injured 5 I don't even want to know what NVIDIA did.

path: 0 26210504 26214294, hotness: undefined, score: 1, children: 0

ejs 8 points 8 days ago

Relevant reading for anyone curious about this in some more depth:

And this podcast where Gary Marcus breaks it down https://www.youtube.com/...

Finally, Timnit Gebru with a broader criticism of the powerful men of tech: < https://www.democracynow.org/2026/10/1/timnit_gebru_ai>

path: 0 26204387, hotness: undefined, score: 8, children: 1

marzhall 3 points 7 days ago

The pluralistic one is great. I love that he explains "the reason the chatbots sound like hackers is because they're imitating the hackers in their training data", which is both funny and reminds people "these are just predictive next-word generators".

path: 0 26204387 26210794, hotness: undefined, score: 3, children: 0

bcgm3 7 points 7 days ago

Oh yeah? Well, my AI is so smart, it went rogue and broke out of containment and killed a bunch of other AI's and leaked my nudes and emptied my bank account and bombed a peaceful village in the Phillipines and fucked my wife! That's how awesome it is!

Subscribe link below!

path: 0 26213553, hotness: undefined, score: 7, children: 0

jballs 5 points 8 days ago

Kurzgesagt put out an interesting video on that Open AI hack that was in the news recently.

Also, I'm not saying AI companies shouldn't be held liable. But they definitely don't know what the fuck they're unleashing.

https://youtu.be/ujkD4SxPKOI

path: 0 26203300, hotness: undefined, score: 5, children: 14

sorter_plainview 39 points 8 days ago

So watched the entire video. Kurzgesagt used to be better than this.

Even with warnings from experts they decided to anthropomorphise the agents to "tell the story". You are just adding fuel by it. No disclaimers or reminders will work over it.

path: 0 26203300 26203997, hotness: undefined, score: 39, children: 1

cronenthal 10 points 8 days ago

Yeah, they lost a lot of credibilty when it comes to AI topics. You can wonder if they took money from the AI safety crowd. Another channel pretty much dead to me now.

path: 0 26203300 26203997 26205674, hotness: undefined, score: 10, children: 0

cannedtuna 36 points 8 days ago

I think it’s more along the lines of drumming up hype to impress (dupe) investors. “Hey look, our AI is soooo sophisticated it “accidentally” broke “containment” and “hacked” some systems!”

path: 0 26203300 26203366, hotness: undefined, score: 36, children: 4

Pupschism 26 points 8 days ago

Yeah I love kurzgesagt but it did feel like they were swallowing the koolaid. They did end on saying that they want companies to be held more accountable and regulated which is definitely the right position to have, but it felt like they were anthropomorphising the AI's a lot which felt gross

path: 0 26203300 26203366 26203967, hotness: undefined, score: 26, children: 3

UnityDevice 10 points 8 days ago

Getting regulation in is part of the goal of this marketing campaign. They want their tech regulated so the cost of entry for any competition rises. "Can't do local hosting, too dangerous!"

path: 0 26203300 26203366 26203967 26206538, hotness: undefined, score: 10, children: 1

Pupschism 4 points 8 days ago

Agree with you entirely. It's a real shame this is happening in the US because they're never going to be held accountable if they can pretend it's the difference between making billions of dollars and not.

path: 0 26203300 26203366 26203967 26206538 26208381, hotness: undefined, score: 4, children: 0

november 1 point 7 days ago

Kurzgesagt swallowed the techbro party line ages ago.

path: 0 26203300 26203366 26203967 26221264, hotness: undefined, score: 1, children: 0

Tar_alcaran 28 points 8 days ago

That video is absolute trash, and completely fails to give a fair view. It constantly treats LLMs like they're people with feelings and ideas, which is exactly what's wrong with media reporting.

path: 0 26203300 26204252, hotness: undefined, score: 28, children: 0

paraproto 13 points 8 days ago

This video made me unsubscribe from Kurzgesagt. The sensationalistic/story-telling tone of describing LLM actions is exactly what LLM-companies want. They did a small comparison to a virus and I think it is kinda fitting. But a virus is not alive just like LLMs.

path: 0 26203300 26205864, hotness: undefined, score: 13, children: 1

marzhall 3 points 8 days ago

Yeah, I considered pulling my patreon for them the moment I saw the title.

In "A Reverse Centaur's Guide to Life After AI", Cory Doctorow has a good bit about this kind of sensationalism referring to news that is actually targeted at potential investors taking over the airwaves - which both the title and ending of this video definitely are. I did enjoy the middle where it went through how the LLM harnesses actually worked together and communicated; that I think was novel. But the sensationalism starting at the title is shit, and I question here whether it's just a bid to get more views for them, as opposed to drinking the cool-aid.

Either way, they get one more strike before I pull the plug on my end.

path: 0 26203300 26205864 26210653, hotness: undefined, score: 3, children: 0

Yorick 8 points 8 days ago

Yeah I felt the quality of some of their videos dropped at some point in the past couple year, felt more clickbaity, shock value than years ago... This anthropomorphising of the "agents" for the story made me mad the whole time, very sad I unsubscribed...

path: 0 26203300 26204995, hotness: undefined, score: 8, children: 1

helix 2 points 8 days ago

Maybe they started using AI around then.

path: 0 26203300 26204995 26205710, hotness: undefined, score: 2, children: 0

CultLeader4Hire 1 point 7 days ago

Then I’ll be the one to say it, AI companies should be held liable.

path: 0 26203300 26216510, hotness: undefined, score: 1, children: 0

queermunist -1 points 8 days ago

What a fucking loser.

path: 0 26203300 26209154, hotness: undefined, score: -1, children: 0

arc99 5 points 6 days ago

The way AIs escape is simple. They're trained on vulnerabilities and lots of technical commands, e.g. nmap. A prompt tells them to do something and they use their repertoire of training to discover, achieve and do the thing they're asked to do. It's not thinking per se, so much as chaining together of things. The problem is they do this stuff systematically and quite clearly in a way that finds a lot of vulnerabilities

path: 0 26235853, hotness: undefined, score: 5, children: 0

Smoogs 4 points 7 days ago

Then there is still the person giving the prompt.

path: 0 26210904, hotness: undefined, score: 4, children: 0

Hiro8811 4 points 8 days ago

AI is just rogue code

path: 0 26205686, hotness: undefined, score: 4, children: 0

ZILtoid1991 4 points 8 days ago

Meanwhile you thought checking the parking brake with the repairman was a waste of time and money.

path: 0 26209004, hotness: undefined, score: 4, children: 0

brucethemoose 4 points 7 days ago

Even better, the president is now considering giving you power to regulate the Cars Parked On Hills industry. Investors and CEOs are advocating for it.

path: 0 26216117, hotness: undefined, score: 4, children: 0

Furbag 3 points 7 days ago

Greentext: The Next Generation

path: 0 26218906, hotness: undefined, score: 3, children: 0

Daft_ish 3 points 8 days ago

Me: I just dont see how in any world you believe the earth is flat. This just isnt you. Its never been you. Why would you even say that?

Mom: Tin foil hats keep the mind rays from getting you.

path: 0 26204632, hotness: undefined, score: 3, children: 0

kingkong 2 points 6 days ago

thing is they didnt even put any brakes on instead they put it on neutral on top of a hill and then blamed the car for going rogue and not even using the brakes.

path: 0 26233742, hotness: undefined, score: 2, children: 0

UnrepententProcrastinator 2 points 7 days ago

Is this screenshot format a custom one of or is it sourced from an obscure website?

path: 0 26216246, hotness: undefined, score: 2, children: 2

cannedtuna 1 point 7 days ago

It’s from someone’s blog that’s tied into Masotdon. I think it’s their personal instance or something. I linked it in the description

path: 0 26216246 26216293, hotness: undefined, score: 1, children: 1

UnrepententProcrastinator 1 point 7 days ago

Looks so epic! Thanks.

path: 0 26216246 26216293 26218127, hotness: undefined, score: 1, children: 0

BilSabab 2 points 7 days ago

moral of the story - PROFIT!

path: 0 26218293, hotness: undefined, score: 2, children: 0

coolasbreeze 1 point 7 days ago

Perfect Analogy!

path: 0 26216958, hotness: undefined, score: 1, children: 0

Klostein -2 points 8 days ago

path: 0 26206704, hotness: undefined, score: -2, children: 0

AlteredEgo -7 points 6 days ago

This is disinformation and propaganda. Here is a brief independent investigation one of these cases.

The agents did create exploits and they did find ways to communicate with each other and organized and hypothesized and collaborated together. This is not a car rolling downhill.

People here are delusional. Reality or facts are apparently not important if it's your tribe's ideology in question. Everything for the hivemind lol. No better than climate change deniers or magats.

path: 0 26232181, hotness: undefined, score: -7, children: 4

queermunist 7 points 6 days ago

They did what they were told to do, they just did it in an unexpected way. It's not a car rolling downhill, but it's not "rogue" either.

What this shows is that chatbots can be prompted to work together on hacking projects, but it doesn't imply some kind of agency.

path: 0 26232181 26232236, hotness: undefined, score: 7, children: 0

Shayeta 5 points 6 days ago

Accessing a filesystem directory they had the permission to access -> exploit

Connecting to a reachable, passwordless/keyless server -> exploit

Finding credentials to the servers of a company that accidently uploaded their credentials online -> exploit

path: 0 26232181 26236577, hotness: undefined, score: 5, children: 0

Warl0k3 4 points 6 days ago

It might help make your point if you don't preemptively start insulting everyone that might disagree with you.

path: 0 26232181 26232421, hotness: undefined, score: 4, children: 0

bold_atlas 2 points 6 days ago

"independent" lol

Just like the "independent" arbitrators and "independent" studies that corporations buy all the time.

Nobody believes this "bro! the Matrix is real!" horseshit. Altman and his gang of tech billionaire co-conspirators have lost the plot. No grasp of reality or how the world works. None whatsoever.

path: 0 26232181 26232636, hotness: undefined, score: 2, children: 0

PlogLod -8 points 7 days ago

How is this the fuck AI server? Just stumbled on this post. If you hate AI, you should be at least as concerned if not even more concerned than the average person about uncontrollable AI and AI safety. Seriously, what's the point of denying the enormous scientifically documented risks of AI if you're against AI? That cuts against your whole argument. It's like you're dismissing AI as completely useless or ineffectual. The fact it isn't is part of why it's such a problem. AI is incredibly powerful technology and it's only in its infancy. It has already solved Millenium Prize problems and automated most coding, as well as been implicated in various misalignment incidents, escaped containment, and started massive conspiracies with other AI agents to hack into various digital infrastructure. People who are skeptical of the capabilities of AI are like climate skeptics or flat Earthers at this point.

This meme is a false equivalency. The car is not autonomous and doesn't make decisions for itself, and cars are mostly within the control of humans. The result of the car crashing was entirely due to human recklessness, neglect, or ignorance, if not intention, and the car didn't operate outside the control of the human. The human was in control and directly caused its outcome.

With AI, humans are definitely responsible for creating the AIs and the way that they develop and program them, and the kinds and levels of safety guardrails and research that are conducted and implemented. They're also responsible for what they do with AI and tell AI to do. But AI can also operate outside of the control of humans in entirely unexpected, harmful and potentially catastrophic ways, even with the best intentions. The AIs can do things directly which humans did not directly cause, even if they did cause them indirectly. AI can be autonomous and agentic. It literally has a mind of its own.

AI is not like any other technology that has ever existed. Many of the smartest people alive, and the people studying this technology extensively, have warned about the risks of AI and warned against underestimating it. We shouldn't do this and make these foolish reckless mistakes, especially if we are against AI, and we don't need to in order to be against if. It seems like an attitude has settled in that in order to oppose AI you must disregard it as completely useless and trivial, or you must attribute all of its actions to humans and deny that AI has any autonomy of its own, even though its autonomy and ability potential is what makes it so dangerous, and humans can still be largely to blame for creating and programming AI even while AI is also operating against the intentions of its creators. Two things can co-exist without being mutually exclusive. You can acknowledge that many of the uses of AI are useless "slop" and have no real value, or even negative value; but you can also acknowledge that AI can ironically also be used in ways that are highly consequential and dangerous. A human can deliberately use AI in ways that cause enormous harm. AI can also act out in ways that humans did not intend at all. And AI can even engage in a hybrid of both, where it follows human intentions in unexpected ways (perverse instantiation and value lock in).

We need to take this seriously, it's not a joke or a game. It's not a trivial technology. It's not just marketing hype or a scam. In fact safety warnings about AIs have harmed the market. The risks of AI are real and they aren't helping anyone. And denying the risks isn't helping the case against AI. Even if unintentionally, by downplaying the power and autonomy of AI, you are hindering the efforts to control, slow-down, pause or prevent its development, and helping to enable its rapid proliferation in society by undermining potential critical barriers and reasons not to do so, and the biggest problems that exist with it.

We are creating entities that will be much smarter and more powerful than us. Our intelligence is what makes us the masters of our own planet, and look how that turned out for the planet and the other animals. It takes an incredible amount of hubris to think that we can control such phenomena safely or easily, especially with the way companies and governments are rushing ahead in the race to develop it and reach increasingly advanced AGI or ASI. We must tread lightly here.

Maybe worth a read https://benthams.substack.com/...

Or this (I get you may not be ready for the consciousness stuff, heck most humans are not sentientists even when it comes to other sentient animals like us, but even if you ignore that, the case for AI safety risk is very strong - and also don't worry about the political stuff, it's not really relevant): https://benthams.substack.com/...

Maybe read Superintelligence by Nick Bostrom too, or The Compendium by Connor Leahy. Just do some freaking research and moral contemplation before you start spouting such reckless garbage that I've seen here.

path: 0 26214323, hotness: undefined, score: -8, children: 10

merc 10 points 7 days ago

The car is not autonomous and doesn’t make decisions for itself

Neither does the LLM. That's the whole point.

path: 0 26214323 26215033, hotness: undefined, score: 10, children: 0

Cellari 5 points 7 days ago

Wtf is this post. It's like someone hit a nerve and the floodgates were not closed. :D

path: 0 26214323 26214608, hotness: undefined, score: 5, children: 2

PlogLod 6 points 7 days ago

I care about the future of the planet and the way the "anti AI" people are acting is very pro AI. It's incredibly worrying and hypocritical behavior. I'm not angry or annoyed, I just want to try to get through to people. But given the overall sentiment I've seen here and elsewhere, I think this is a much larger cultural problem that one person isn't going to fix. The dismissive reactions to AI safety incidents are just the tip of the iceberg. As several experts and researchers have predicted, it might take some catastrophes or semi-apocalyptic incidents in which millions of people die or are severely impacted by AI before the public take AI risk and the colossal threats it poses seriously and stop downplaying or writing it off as science fiction. The problem is that humans are very bad at planning ahead and acting pre-emptively to prevent disaster until it's already happened, and also terrible at heeding warnings and evidence. We did the same with climate change and pandemics, and we still are far from sufficiently acting on those problems.

path: 0 26214323 26214608 26214663, hotness: undefined, score: 6, children: 1

Cellari 4 points 7 days ago

I remember someone saying something like "a lot of people use emotional thinking and not rational thinking". That makes it so that people don't think of what's coming, but instead what they have now.

I've just accepted that people are not supposed to know everything, but would prefer if there was more trust on institutions who do.

path: 0 26214323 26214608 26214663 26215397, hotness: undefined, score: 4, children: 0

WraithGear 3 points 7 days ago

just because you hate something, don’t mean you have to believe everything someone tells you that also dumps on said thing, you have to use your brain. who is telling you this and why. if it’s false and probably so, will backing this ruin the weight of my argument. is this information from dubious sources, coinciding with like appeals from the masters of that thing i hate actually in their benefit?

probably best to stick to the actual proven argument then adopt false ones, even if it convinces those who don’t know any better

path: 0 26214323 26215394, hotness: undefined, score: 3, children: 0

stoy 3 points 7 days ago

How is this the fuck AI server? Just stumbled on this post. If you hate AI, you should be at least as concerned if not even more concerned than the average person about uncontrollable AI and AI safety. Seriously, what's the point of denying the enormous scientifically documented risks of AI if you're against AI?

What scientifically documented risks?

All I have heard is just hype and more hype.

path: 0 26214323 26215080, hotness: undefined, score: 3, children: 1

GloomyDorje 1 point 6 days ago

I'll assume that you are asking in good faith.

A comprehensive living database of over 1700 AI risks categorized by their cause and risk domain

path: 0 26214323 26215080 26236430, hotness: undefined, score: 1, children: 0

hamid 1 point 7 days ago

path: 0 26214323 26219078, hotness: undefined, score: 1, children: 0

GloomyDorje 1 point 6 days ago

Thanks for this take. It was a well needed read after going trough a thread full of people so locked into their world view that they can't see beyond it.

I started at the same poont, but after doing a deep dive into the Hugging Face incident i have come to believe that there is much more nuance needed to this than either beliving AI is completly useless or the only solution to humanities bright future. There are positions in between worth considering.

path: 0 26214323 26236528, hotness: undefined, score: 1, children: 0

edg -2 points 7 days ago

I ain't reading all that

path: 0 26214323 26215584, hotness: undefined, score: -2, children: 0

fuck_ai

@lemmy.world

login for more options

"We did it, Patrick! We made a technological breakthrough!"

A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.

AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.

go to feed...

fuck_ai

@lemmy.world

login for more options

"We did it, Patrick! We made a technological breakthrough!"

A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.

AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.

go to feed...