Show me you can solve the alignment issue with a 2.5 year old first.
Maybe the real unaligned super intelligence were the corporations we made along the way 🥺
2 years ago by BigMuffin69 to c/sneerclub
casmael 17 points 2 years ago
Cool analogy, but I think you’re overestimating OpenAI. Isn’t it more like installing airbags on a little tikes bubble car and then getting a couple of guys to smash it into a wall real fast to ‘check it out bro’
smiletolerantly 4 points 2 years ago
This is the first mention of Accelerando I've seen in the wild. (Assuming it is. I'm not sure.)
Soyweiser 3 points 2 years ago
It was more Soylent green, but it also is partially based on (friend of the club) the writings of C Stross yes. I think he also has a written lecture on corps being slow paper clipping AGI somewhere.
smiletolerantly 2 points 2 years ago
That is a beautiful comparison. Terrifying, but beautifully fitting.
I read Stross right after Banks. I think if I hadn't, I'd be an AI-hype-bro. Banks it the potential that could be, Stross is what we'll inevitably turn AI into.
gerikson 1 point 2 years ago
Banks neatly sidesteps the "AI will inevitably kill us" scenario by making the Minds keep humans around for amusement/freshness. Part of the reasons for the Culture-Idiran war in Consider Phlebas and Look to Windward was that the Idirans did not want Minds in charge of their society.
mawhrin 1 point 2 years ago
stross' artificial intelligences are very unlike corporations though, and different between the books. the eschaton ai in singularity sky is quite benevolent, if a bit harsh; the ai civilization in saturn's children is on the other hand very humanlike (and the primary reason there are no meatsacks in saturn's children et al. is that humans enslaved and abused the intelligences they created).
bleistift2 11 points 2 years ago
AI alignment research aims to steer AI systems toward a person's or group's intended goals, preferences, and ethical principles.
kamenlady 18 points 2 years ago
Misaligned AI systems can malfunction and cause harm. AI systems may find loopholes that allow them to accomplish their proxy goals efficiently but in unintended, sometimes harmful, ways (reward hacking).
They may also develop unwanted instrumental strategies, such as seeking power or survival because such strategies help them achieve their final given goals. Furthermore, they may develop undesirable emergent goals that may be hard to detect before the system is deployed and encounters new situations and data distributions.
Today, these problems affect existing commercial systems such as language models, robots, autonomous vehicles, and social media recommendation engines.
The last paragraph drives home the urgency of maybe devoting more than just 20% of their capacity for solving this.
schmorpel 10 points 2 years ago
They already had all these problems with humans. Look, I didn't need a robot to do my art, writing and research. Especially not when the only jobs available now are in making stupid robot artists, writers and researchers behave less stupidly.
BaroqueInMind 4 points 2 years ago
I cannot, please elaborate.
Zagorath 10 points 2 years ago
I genuinely think the alignment problem is a really interesting philosophical question worthy of study.
It's just not a very practically useful one when real-world AI is so very, very far from any meaningful AGI.
Soyweiser 16 points 2 years ago
One of the problems with the 'alignment problem' is that one group doesn't care about a large part of the possible alignment problems and only cares about theoretical extinction level events and not about already occurring bias, and other issues. This also causes massive amounts of critihype.
sneerclub
@awful.systems
1305
532
185
Hurling ordure at the TREACLES, especially those closely related to LessWrong.
AI-Industrial-Complex grift is fine as long as it sufficiently relates to the AI doom from the TREACLES. (Though TechTakes may be more suitable.)
This is sneer club, not debate club. Unless it's amusing debate.
[Especially don't debate the race scientists, if any sneak in - we ban and delete them as unsuitable for the server.]
See our twin at Reddit
go to feed...
sneerclub
@awful.systems
1305
532
185
Hurling ordure at the TREACLES, especially those closely related to LessWrong.
AI-Industrial-Complex grift is fine as long as it sufficiently relates to the AI doom from the TREACLES. (Though TechTakes may be more suitable.)
This is sneer club, not debate club. Unless it's amusing debate.
[Especially don't debate the race scientists, if any sneak in - we ban and delete them as unsuitable for the server.]
See our twin at Reddit
go to feed...
OpenAI: "Our AI is so powerful it's an existential threat to humanity if we don't solve the alignment issue!"
Also OpenAI: "We can devote maybe 20% of our resources to solving this, tops. We need the rest for parlor tricks and cluttering search results."
save