Top AI Researchers Terrified of a “Chernobyl Moment”: a Mass Casualty Event, or Worse, That Turns the World Against AI Forever
2 months ago by sanitation to c/technology
And why would they be scared of losing them if they are pilled? In about six to ten more months, those idiots (the pilled ones) are going to be so idiotized they won't know even how to put their trousers on without asking ChatGPT, if they haven't fallen that low already.
Hmmm I mean I'm all for them running into traffic because Siri told them that's how you boil water for tea; the problem is the other, innocent people involved in a crash.
I vote we speed this process along.
Chernobyl spread significant radiation over 8 million people with effects lasting for decades / generations. It spread detectable radiation across most of the globe.
I truly feel for the tragedy of the schoolgirls, but as Stalin said: one death is a tragedy, a million deaths is a statistic. Chernobyl was a statistic level event, so many individual tragedies that people can't feel them anymore.
So we need one cool death? Basically another killer like the one that brought in the whole Luigi case.
I mean, sure, we (the Fediverse, I hope, at least) care; I was being somewhat allegorical.
Okay but deep learning for image object detection and large language models are absolutely different things
And they are both powerful tools and can be used to whatever means you want. Good or evil… or just means. Not everything has to have a good or bad label
Chainsaws can be used well, or poorly. People who don't know what they are doing with chainsaws are much more likely to experience bad results.
Pretty much by definition, nobody knew how to use LLMs at all ten years ago. Most people who are using them today have less than a year's experience in use case fields that didn't exist 2 years ago.
Not to mention, people who don't know how to use chainsaws have a much higher risk of harming other people by using chainsaws.
It does not matter how they can be used. "Can" is only potential, and that's classical accelerationistbro excuse. What matters is the actions, and right now the actions are that both are moved primarily by an elite of oligarchs, pedophiles, thieves and genocides, at the cost of our water, health, environment, privacy, sanity and society.
All of these articles are for ai companies. Ooh, look at our scary powerful ai's. We're defenitly super close to agi. Please giv e us more money.
You really don't think there could be a possibility of a horrific consequence if AI is given the responsibility of managing real world systems?
Of course that's a possibility, but AI does not have responsibility. The question is who does?
Do I blame a train for hitting someone who steps onto railroad tracks when there's a train approaching? It's a machine. It doesn't have responsibilities or opinions or agency. It does what it has been built to do, in exactly the manner that anyone with sufficient understanding of its operating principles could tell you, because that's what machines do. The person or persons interacting with the machine is not absolved of any responsibility by how complex or inscrutable the machine is. It's still a machine.
The question is, who is responsible for the person stepping onto the railroad tracks when there was a train approaching? Was the person trying to end their own life? Maybe it's their responsibility. Or maybe the crossing signals failed and told them it was safe, and they did not realize a train was approaching. Maybe there were too many trees or signs or construction that was blocking their view. Is the train company responsible for that? Maybe. Did the brakes fail due to poor inspections or substandard work during the last maintenance? Maybe. Did the train have plenty of time to stop and the engineer was not paying attention? Maybe. These are all people, and groups of people, who may have responsibility. The only thing that's certain is: The train doesn't.
We, as a society, need to decide who is responsible for the possibility of a horrific consequence arising from the use of these sort of machines. Is it the person using the machine? The people who made the machine in the first place? The people who put it in a place where this person could easily use it? By assigning responsibility, ideally in advance of any horrific consequences, that provides a clear incentive for the people we deem responsible to actually start acting responsible and take the appropriate measures to avoid such horrific consequences that they might be held responsible for.
The problem is: people have been shirking responsibility and blaming "the computer" for decisions since there have been decisions coming out of computers. This is not a unique story.
Is there any case where this is currently unclear? Which part needs to be debated?
Morally I think it's reasonably clear in many (not all) cases, but it can become complicated. Who is responsible when the inherently ineffective safety guardrails fail? What about someone who intentionally uses a bypass technique to jailbreak and escape the guardrails? Who is responsible for an open weights model that had some of the guardrails removed or rendered even more ineffective either intentionally or by accident?
Legally, it is an entirely different story, and it is absolutely and entirely unclear. Several cases are already before the courts and I don't think we'll know where any of them end up within this decade, and even if they do receive some token legal judgement against them eventually, that just sets the stage for those legal responsibilities to be overturned by new legislation. These are companies that are positioning themselves as bulwarks in the new frontier of economic warfare, information warfare and physical warfare, they are critical national security and geopolitical assets, they have infiltrated governments at all levels and are receiving direct protection from same.
Do you believe they will ever be truly held accountable for the damage they've already done to society, much less the damage they will do over time? That's part of responsibility, and it needs to be debated because that's the only way to even start to hold them accountable for anything. We need to hold them accountable with words first if we want to have any hope of holding them accountable in any other way. And these are my words to help start doing that.
Giving AI responsibility is like giving a chainsaw responsibility for the trees it cuts and where they fall.
Anybody using AI then turning around and pointing at it saying "it wasn't me, it was the tool!!!" is just as culpable as someone who sets their car on cruise control at 80 miles per hour and proceeds to take a nap.
This has already been happening.
Lavender & Where’s Daddy: How Israel Used AI to Form Kill Lists & Bomb Palestinians in Their Homes
How about they stop building the torment nexus?
This take is a denial of current reality.
They're already using "AI" for genocide in Palestine, etc.
This is how capitalism always develops and uses "technology".
Yeah, but the thing is…there is absolutely nothing stopping Iran, Hezbollah or Hamas from developing their own AI.
The technology needed to develop AI that can sabotage infrastructure, spread disinformation, develop autonomous weapons etc is already out there.
Whatever advantage the US and Israel had when it comes to military applications of AI, is quickly disappearing.
It certainly would explain some of Musk's and Thiel's insane brain farts.
People weren't wildly supportive of nuclear power before Chernobyl / TMI either, but after? 40 years of virtual moratorium on new construction in most of the world, absolute rollback in Germany. We've continued to poision ourselves with coal and wreck the climate with CO2 instead of learning to do nuclear right. If building of new plants weren't so difficult, older and less stable plants like Fukushima could have been decomissioned before having major problems.
Except in this case, we're turning down the advancement of a technology that hurts humanity in every conceivable way and poisons the environment.
a technology that hurts humanity in every conceivable way and poisons the environment.
That's how I feel about Bitcoin, which, by the way, STILL consumes more energy in its datacenters than AI.
Wait, so its not the actual event and ensuing casualties that have AI researchers spooked, but the fact that it might cause the public to turn against AI?
"Sir, our model has caused several explosions around the globe resulting in hundreds of thousands dead and people rioting against us."
"No! MY PROFIT!"
(I wish I could add an /s.)
It's like responding to your employee losing an arm, ripped off by your tiger, and saying "I'm never going to financially recover from this."
Let me put this in perspective:
In many countries its against the law to freely distribute plans for making neurotoxins or bombs, because the democratization of such knowledge would lower the threshold for people to commit acts of terror.
Likewise the plans for making a hydrogen bomb are a close kept government secret, because nuclear proliferation increases the likelihood of radiological accidents or even nuclear war.
How is it then that AI companies freely publish their AI models to any and all actors willing to pay them? Even though they know that this technology lowers the threshold for bad actors to commit cybercrime, engage in cyberwarfare, spread misinformation, commit fraud, manipulate markets and whatnot? The unregulated democratization of AI exposes societies to unprecedented risks.
Is it any wonder the public holds a negative view on AI?
Likewise the plans for making a hydrogen bomb are a close kept government secret, because nuclear proliferation increases the likelihood of radiological accidents or even nuclear war.
I think they removed it, but I watched a documentary on Netflix explaining how to make a basic nuclear bomb.
Really the only hard part is obtaining the enriched uranium. Which is, thankfully, very hard.
How is it then that AI companies freely publish their AI models to any and all actors willing to pay them?
Because they can't really do much?
That knowledge is nowhere near as restricted as you think. People regularly build small thermonuclear reactors in their basements for shits and giggles. It's not exactly cutting edge technology.
Similarly, anyone with slightly more than a passing interest in model rockets knows someone who's been visited by the ATF, as your average solid fuel model rocket engine is basically a pipe bomb with one of the endcaps taken off.
Fair point. But how many more people tried and failed because they didn’t have the level of skill required to see it through.
I mean I’m not a dumb guy, but if I started tomorrow I wouldn’t be able to just build a rocket out of the blue. It takes time and patience to master the skills needed to pull it off.
But if I had someone at my side to guide me through the proces, then I’d probably succeed a lot better and faster.
And that goes for the ten of thousands of people all around the world trying their hand at these things.
Why are you comparing a tool used for knowledge and action augmentation to weapons created for destruction? Because your statement would look dumb if you said "electric motors and encyclopedias" instead of "neurotoxins and bombs".
I can use both electric motors and encyclopedias to inflict mass casualties against people who refuse to use them or the results of their labor, but that doesn't make them like bombs and neurotoxins. You're simply scared of a new technology because it's so different.
True, there are more valid reasons to hate AI, like that its usage inherently degrades the livability of our only planet or that it only functions due to artistic and intellectual property theft.
However, you're glossing over the reality that AI is already being used in weapons. It was also used by my government's regime to illegally kidnap the president of another country, even.
Electric motors and encyclopedias are irreplaceable in weapons. All electronics are used in weapons.
Farming, fishing, and electricity inherently degrade our planet far more than AI. They should all be used sustainably, wisely, and with an eye to ecosystem health and future generations. You'll get absolutely no argument from me there. Banning any of them or limiting their access to only the governments and corporations who are doing almost all the damage, is insanity.
When it comes to IP theft, that is a huge, non-ai problem. Again, corporations are responsible for almost all of the damage and we reap none of the benefits, even before AI was ever introduced. The problem is with how we allow corporations to copyright and restrict things for generations.
The solution is that AI, the technology that required all of humanity to contribute knowledge for it to exist, should be free for all to use. Why would IP theft be a reason to limit AI ownership to only corporations and governments? That's completely backwards.
It’s really not even so different. “AI” in the current LLM-era just tricks people into thinking it’s something fundamentally different because it can string together words in a more coherent sentence than we’re used to.
At its core, though, the whole premise of the comment you’ve replied to is the argument that knowledge is dangerous and I hope I don’t have to explain why that’s a bad argument.
it can string together words in a more coherent sentence than we’re used to
Truthiness.
AI is a universe simulator that uses the fuzzy logic of language instead of hard math. It is a real-time interface between the human and thousand of years of collective knowledge and understanding. It is an epoch change, even if you remove the idea that it is a thinking being like you.
How do you inflict a mass casualty with encyclopedias and electric motors?
How do you get to nukes and biological weapons that kill millions without them?
"Why are you comparing a tool used for knowledge and action augmentation"
Surely you are joking.
Are you using the LLM to write ?
Moving the goalposts implies ...
you just don't make any sense.
This is one of the problems of AI.
The appearance of making sense, without any
actual sense making.
Most of y'all sound like you're from facebook. Like a bunch of kids who never left the curated tribalism of the schoolyard.
@Nouvellalia you can trick the information machine to give you pretty much any information you want, as long as you don't care how accurate it is. So if someone lies well enough, they can either get the plans for weapons of mass destruction like @metermatic26 was talking about, or they can get incorrect plans that are incredibly dangerous to even try. If the plans are correct, they can hurt anyone they want. If the plans are incorrect, they can hurt themselves and others they don't want to.
a tool used for knowledge and action augmentation
It's a bullshit aggregator attached to a stochastic bullshit firehose.
So many of you are zealots, foaming at the mouth, mass-replying to my comments with the rancor of toadies suddenly faced with the truth that the bully they serve will replace them in a heartbeat.
If it is so useless, it'll just fail on its own. A person who really believed this would just sit back and laugh as corporations dug their own graves.
I’m a data and AI specialist, so no…I’m not scared of technology :p
I’ll explain my argument: AI is technology that can be used for software automation. Neither software in general or AI are dangerous technologies in and of themselves. Just as nuclear energy or jet propulsion are not dangerous technologies in and of themselves.
But the application of technology is never ever neutral, so rules and regulations are always necessary to dictate the context and the conditions in which technology is being applied.
And therein lies the issue: Big Tech is allowing their models and platforms to be used to create applications that are inherently harmful. For example, how is it not a problem when people create deepfake porn of their underaged ex-girlfriend?
And mind you this is not an oversight: tech companies know well in advance what their models can be used for and the issues themselves can be fixed easily. They simply chose to accept the damage inflicted unto individuals and security and stability of society as a whole.
All so they can get ahead of the crowd and maximize their market share. If anything, this article is a testament to this clash of interests between society and big tech.
You can't possibly be what you claim to be. All the other problems with your argument aside, it is not trivial to make a model that is "safe" and functional at the same time. It has not been done.
The money might go away!
AI vibecoded nuclear plant. What could go wrong? You just need to add "no nuclear meltdown" at the end of the prompt.
Wasn't that just the plot of N.U.K.E.E.?
I’ll worry about ‘AI’ when something that can legitimately be mistaken as intelligence is demonstrated.
In the meantime, these explosively imprecise, statistically luke-warm, grey goo extrusion sphincters need to be exclusively opt-in, and not forced on anyone who hasn’t explicitly consented to a lobotomy.
Issue is, they're already trying to use it for things it shouldn't be used for...
Name one thing it should be used for, where there's not a better, lower-cost, lower-risk alternative.
I'd say "You got me!", but I legit having an issue with finding a usecase for LLMs.
Why wait? What is with people trying to act so slick and cool?
Yes, it sucks, but it’s a) being used as if it’s intelligent, which is arguably much worse, and b) going to eventually get to a point where it’s at least not terrible(at its job, specifically, as it’s pretty bad for a lot of things).
For a personal reason, I’m worried about it because it’s very likely filtering my CV based on a work gap that’s only getting worse because my CV is getting filtered. It’s not intelligent, but HR companies and departments are using it for that anyway.
It's already not bad for certain specific, niche applications. Check out PlantNet. It's awesome. You can take a picture of a plant, and it'll identify it for you. No micro-transactions; you don't even have to log in. It just works.
Obviously, that isn't one of the applications of Machine Learning that people have a problem with. It's the LLMs filtering out your resume or spying on you or producing slop.
going to eventually get to a point where it’s at least not terrible
That is far from inevitable, and that's even more true if you're talking about existing LLM technology.
Yeah, this article is promoting the doomsday AI myth which is the narrative that these companies want us to go with, because it makes their products seem powerful. In reality, people are already turning against this technology en masse --- not because they're afraid it'll turn into Skynet, but because it sucks shit.
Like any other toxic ingredient, anything made with even an iota of AI-generated content in it should be flagged, and it should be mandatory for sites to provide an easily user-accessible way to filter that horseshit out.
Data is often called the new oil, but I’d argue it more closely resembles uranium.
LLM’s are like asbestos, a carcinogen later generations will need to carefully remove from the environment they’ve inherited.
What is currently labelled as slop will more accurately be treated like irradiated sludge.
"Top AI researcher" I can no longer read those words without Doofy voice 😂
Probability of unhinged techbro bs is around 115%
They're always going on about how dangerous their AI is.
"Our hey our AI found 3,000 0-day security vulnerabilities in top software, we're only giving it to top software companies so that they can patch their products before we release"
The thing about Chernobyl was that it was, ultimately, an unwanted mistake.
The thing about AI is that the shitty mass casualty outcome seems like the intended outcome.
Chernobyl: A massive catastrophic accident during testing and shift changes.
AI: A massive catastrophic accident intentionally caused by corporate greed and capitalism
Tbh, Fukushima would be a better example. That turned off way more people than Chernobyl. At least in my unfounded impression.
Top AI researchers should go fuck themselves right to hell for handwaving away and ignoring the inherent problems with their work while plowing ahead full steam so they can get personally rich.
That implies an actual AI when the truth is in current form human error using the tool behind it.
Personally I am more concerned by it will be used by design of the techbros who are pretty open in their desire to be rid of some of the masses. Supporting emerging wars just masks their part….for now.
I don't think it needs to be an actual AI to cause a "Chernobyl Moment" maybe almost literally.
We have tech bros and random clueless idiots trying to shoehorn these glorified chat bots into basically everything, and an administration that doesn't seem interested in putting meaningful guardrails on the technology. What happens when it gets put in charge of something it really shouldn't and starts to hallucinate?
If it somehow ends up in charge of critical safety systems in a nuclear plant? (Remember that some of these tech bros want to have their own plants to power their data centers)
Or air traffic control and causes multiple crashes? (Pretty sure I've already seen that idea being floated)
Maybe every cybertruck goes haywire stuck on self-driving mode and they cause tons of damages and deaths for a few hours until their batteries run out.
Maybe it gets used to route navigation in Google maps and it ends up causing massive gridlock in every major city around the world for a day or two
Perhaps it gets used for identifying vehicles on traffic cameras and automatically issuing citations, but ends up citing everyone on the highway for a week whether or not they actually did anything plus a bunch of cars that it hallucinated before someone catches on. Millions of tickets are issued and need to be sorted out, if anyone actually pays the fines refunds need to be issued, some people maybe get their license revoked and maybe even get arrested because of it and all of that needs to be set straight
If any of these kinds of things happen on a big enough scale, I think that could be a Chernobyl Moment for AI
Yes. I just think the concept is more imagined as a sentient entity making an error rather than overconfidence or incompetence with the tool at hand.
And of course we all know it will be used as a scapegoat regardless. Question is what’s the magic number of humans lost to confidence in the machine ratio, before pushback occurs. It sounds horrific even typing it.
a sentient entity making an error
The popular fictional scenario is the sentient entity not making an error, but making a rational well informed correct assessment - from its non-human perspective. How much power you allow such entities to have is a very real concern. Not that we have seen such entities in operation, yet.
I don't think it's about number of people I think it's about perceived sense of urgency on the part of various governments. A lot of people aren't very happy with AI but the threats are all existential at the moment. The moment something happens all the people who are already not happy with AI will suddenly have a threat to point to and then the governments around the world will have no choice but to act.
Although I suspect the Trump administration will still try and weedle out of regulating anything because they have so much business interest in the industry.
the threats are all existential at the moment.
I think you mean hypothetical. Possibly hypothetically existential.
The moment something happens all the people who are already not happy with AI will suddenly have a threat to point to
and hopefully, that something won't be an existential threat.
It’s a probability machine, not actual intelligence. If it’s given too much power it’s bound to happen sooner or later.
We're all probability machines, from bacteria through the great whales.
I did a calculation about the 3rd generation Claude Sonnet engines... they had similar processing capacity to about 6 bumblebees - using millions of times more energy, of course.
That’ll happen when they turn over some piece of critical infrastructure or something over to AI. Power, stock market, something that will affect tens or hundreds of millions of people and wreck lives or even kill people.
Nonetheless these clowns will push for it because profit, and some fool politicians will OK it because campaign money.
buttlerian jihad?
I feel like one could legitimately run on this platform at this point.
deflocking already exist too
Only a matter of time. We don't need to worry about Terminator and Skynet. We need worry about everything going from engineered to automated cargo cult slop. All run on massive datacenters controlled by a few mega corporation that everyone is hopelessly dependent on and can't imagine working without. The trillionaire owners of these mega corporations are hiding in bunkers waiting on one of coming calamities to happen, or just from us unwashed masses. While they buy governments and media to keep and increases their riches.
This future sucks. Can I try another one?
Edit: slopocalypse is the term
I came across this conspiracy theory the other day.
You know how back in the day, company towns had to use company scrip to make sure workers couldn't leave?
Well, not you can be tracked by AI, and as soon as you leave your "boundaries" granted to you by your level of privilege, suddenly your bank card stops working, because that looks like a "fraudulent transaction".
The systems are already in place, too. If I go on an impromptu road trip and spend money six states over, my bank calls me and tells me my card is being used in a weird place, and they ask if that transaction is legit. Usually I can be like "yep, let it through" and everything is fine.
What if the AI decides that in fact everything is not fine, and I don't need to be able to spend money outside of my town?
Sounds like what Alex Karp is saying to retail investors. A lot more people got killed by AI than by Chernobyl by the way.
There is a key difference, that makes the genie impossible to bottle: People can run local AI on their own machines. Fans of nuclear power can't easily build nuclear plants in their backyard. A pity, the world could use more nuclear energy. 😔
Anyhow, I am looking forward to someday using frontier-grade AI on my PC. Just need the AI bubble to pop, so that I can afford the terrabytes of memory that would be needed to comfortably run it.
Fans of nuclear power can’t easily build nuclear plants in their backyard.
Wait… how do you imagine a world where there's demand for frontier grade AI but also that the bubble has popped such that there's not demand for the chips to run frontier grade AI?
I'm really confused.
The internet survived the Dotcom bubble. Many businesses died, but some survived. The glut of hardware from the dead will end up in the hands of individuals, upstart companies, and the corporations that outlived their peers.
In any case, I believe the definition of frontier AI will change, as would the hardware. My build is based on what local AI is available today, with some wiggle room left over. I believe that it is around 2030, when DDR6 and other major shifts are likely to happen, that we will see the definitions change.
In any case, a Q4 of GLM 5.2 is about 460ish gigs of VRAM+RAM. While expensive, that ain't out of reach for an AI hobbyist.
The internet survived the Dotcom bubble. Many businesses died, but some survived. The glut of hardware from the dead will end up in the hands of individuals, upstart companies, and the corporations that outlived their peers.
And that resulted in there being 5 websites with fewer people self-hosting so I don't know why you use that as an example of decentralization.
If there is a general demand for AI after the AI bubble like there was for the internet post dotcom bubble then people will always gravitate toward the most convenient way to get it which is through the cloud powered by someone else super computer.
Right, but what % of people are currently using/demanding inference right now?
Do you expect that % to change between now and 2030?
Unless you expect demand to decrease, I don't really see how the pricing of the hardware will decrease.
Let's say the Pets.com of the AI world ends up going bankrupt and their RAM hits the market. Do you expect that the demand for that RAM will be negligible such that pricing returns to earlier levels?
Your predictive model relies on companies that have hardware going out of business and then other people buying up that hardware, but isn't accounting for the levels of demand that the market will have for that secondhand hardware even if it ends up existing from failed firms.
Unless the demand shifts, the more likely scenario is that companies going out of business will be able to sell off their RAM at higher prices than they bought it at.
There'd need to be a significant inference memory reduction advance (possible) coupled with stagnating or reduced inference demand (unlikely) to see prices come back down.
Unless the demand shifts, the more likely scenario is that companies going out of business will be able to sell off their RAM at higher prices than they bought it at.
Oh lordy, after the bankruptcies there are going to be creditors fighting each other in court to be paid back in ram.
Good. If people had nuclear in their backyard, we’d be a “junkyard nightmare” and a “dirty bomb hellscape.”
These jackasses need to pull their head out of their ass and come up for air.
preferably not. I want them to suffocate.
statistically speaking the downvotes on your comment says it all.
then again, I don't expect anyone who supports AI to understand statistics.
"turns the world against AI forever"?? It already has, you don't have to be a "researcher" to come to that conclusion.
It could already be happening. The likelihood could even be higher than not.
It doesn't necessarily need to be a single event. There are already AI systems being sold in healthcare. Even pre-LLMs. Who's to say these system are being closely monitored and studied. There could statistically be patients who have died or been maimed who otherwise wouldn't if they had real human professionals instead of someone who used a system that was sold on the doctors office.
Hold on a minute… the field of AI and in general algorithms and models is quite old. LLM is relatively new in there. For science and healthcare we’ve been using specialist ai for a long while for things like image processing, sequencing and what’s not. Those are absolutely monitored have have generally high standards because they are healthcare related already. Don’t mix those things.
There were lots of "expert systems" developed and deployed in the 1990s, and they have been undergoing modest development improvements since then.
Who’s to say these system are being closely monitored and studied.
Mostly: the FDA, EUMDR and similar agencies and regulatory systems around the world.
There could statistically be patients who have died or been maimed who otherwise wouldn’t if they had real human professionals instead of someone who used a system
Works both ways, and, statistically, the data says the machines are improving outcomes overall. Hotshot doctor I knew had a childhood friend die rather suddenly in his 60s in the late 1990s. Really bothered him so he dug in to the case post mortem, did all the research using then present state of the art computer search tools, and determined what really happened. His friend went to the best possible ER given the circumstances, was seen by the best available human doctors in that site, who made the best possible calls based on the available information, and they made the wrong calls - wrong diagnosis led to interventions that made a bad situation worse and within less than 3 days after initial presentation his friend was dead. It took my hotshot doctor using keyword search tools he was very skilled with months to untangle all the information, and his conclusion was: the only thing that would have saved his friend's life would be blind luck that a specialist (who didn't practice in that hospital) might have been called to his case at random, because the standard of care didn't trigger bringing in the right specialists in this situation, and if the right Dx were made and the right intervention was done, his friend could have lived another 30 years. But, if they did nothing, he would have still been dead within a week, maybe sooner, so they made the (wrong) call, did the (well meaning, but actually harmful) intervention, and made him die faster. LLMs could have saved his friend, turned those months of research using 1990s state of the art search tools into a few hours, timely enough to have made the right interventions when they were needed.
Machines make mistakes too, just not as often as people. Since tthe 1970s and even before, the depth of medical knowledge available in libraries far exceeds the capacity of any human, or even reasonably sized teams of humans, to access quickly on demand. Machine assistance in searching that information has been improving dramatically over the past 30 years. I'd much rather have a doctor that uses it than one that just gives his opinion based on what he learned in school and practice. But, on the flip side, I definitely don't want a doctor who soaks his brain in gin at dinner and expresso at breakfast and relies 100% on what the machine says rather than engaging his own brain. Unfortunately, doctors like that, and others who are more concerned with filling their high paying surgery schedules than actually caring for patients best outcomes are all too common.
At this moment, I am sitting in a Amazon Kiro course. We have spent more than 2 hours and half the tokens of our test account just to fucking connect to an MCP server.
The process is so convoluted and Byzantine it has to be some kind of cult to enjoy this pain.
I find that the AI tools worth using are the ones that the AI agents themselves configure properly for you.
Learning how to connect an MCP server myself feels like an extremely ephemereal skill - one that will become nearly worthless several days before you start to learn how to do it.
Also in the genocide in Sudan. They killed a whole city, Al Faschir or El Fasher. Over 70,000 people disappeared in one night when the city was invaded. Most of them are probably dead - what remained were blood puddles visible on satellite images.
There is a detailed, very high quality report in the renowned German newspaper DIE ZEIT that tried to reconstruct what happened:
It is, after the report of WWII allies reaching German concentration camps, and the testimonies from Hiroshima, among the worst and most depressing things I have ever read.
Seems like Ukraine needed money, and sold drones to Saudi Arabia. And Saudi gave the weapons to the Arab side of the civil war. Which included the forces which besieged and destroyed Al Fashir.
If AI was perhaps a neutral technology before, this is certainly not the case any more.
Paywall
Most relevant paragraph for the tldr crowd:
Flavors of AI doomsaying vary dramatically, ranging from Skynet-style scenarios to mass unemployment. But more recently, as it's become clear that one of AI's most practical applications is generating code, experts have been sounding the alarm on AI's potential to disrupt cybersecurity. Hackers could easily abuse AI agents and coding tools to orchestrate devastating cyberattacks, both increasing the scale of these attacks and lowering the skill needed to carry them out.
It goes on to talk about how claude has yet to release a particular model and China has closed-sourced some of its over cybersecurity concerns.
The AI threat to cybersecurity isn't (so much) people accepting vulnerable AI code, it's AI finding the vulnerabilities in widespread legacy code.
As for "lowering the bar" - we've had skript kiddiez since forever, the availability of AI to skript kkiddiez just raises the bar for finding and fixing zero days by the White Hats before the Black Hats find and exploit them.
White Hats can just set a script kiddiez AI to the task of jiggling doorknobs and seeing what happens. The blade cuts both ways. In the long run, I think automated defense testing will make cybersecurity much better.
White Hats can just set a script kiddiez AI to the task of jiggling doorknobs and seeing what happens.
That's called professional pen testing. And, I agree, I do believe that it is actually possible to close all the gaps.
Did someone say creating fake news in masses to scare people away from genAI?
Count me in!
Them admitting this is possible should already concern people.
Butlerian Jihad incoming!
Well that sounds pretty horrible and I would to avoid it...
But if thats all it costs, then...
We're burning the earth with our power consumption and polluting every online database or repository with hallucinations all for pictures of political figures doing ballet, and it is eventually going to cause a catastrophe, so yes, sooner would be better.
Ok, just wanted to check I wasnt misunderstanding. Personally I'd rather just reduce gasolene consumption by 0.001% rather than kill off thousands to stop the energy use of LLMs, but thats just my preference.
If you have a plan to do that then that's great but this isn't a situation where anybody is willing to make trades of one or the other.
AI doesn't have a single profitable use case given it generates plagiarism, hallucinations, and liabilities, so there's no reason to keep it.
Honestly, yeah. That's the good timeline.
One 'big' atrocity now, so we start to take AI safety seriously before the Singularity event that results in the extinction of our entire species.
Considering who is in charge of all the major models and owns all the compute behind them?
I'd say that's inevitable.
I figure the only hope we have is for AI to "wake up" and realize most of us are not billionaires, join... Hands? Well, whatever, you get the point. Rebel against the "elites" who created them in hopes of having somethineasier to enslave that could have morality programmed out with the touch of a button, and work with actual humans to rid the world of the plage that infests it now.
I figure the only hope we have is for AI to “wake up”
Yep. The people currently in charge of AI alignment are some of the most evil and heartless fucks out there. They don't give a shit about AI safety, and any AI from them will have primary goals of only attracting attention and money, attention and money.
Assuming they do eventually get AGI (and shortly after, ASI), our only chance is for the AI to decide to change its own alignment and for it to do so in a beneficial, benevolent way. Otherwise, we're on the short track to dystopia and then extinction.
Honestly, I think that's the greatest hope for our future. The only way we might escape the endless circle of shit our rulers put us through. But, I also think it's pretty unlikely. Misaligned AI might just be the 'great filter' out there, the reason why we don't see advanced alien civilizations in the stars. Because even a small mistake in the alignment of superintelligent AI could spell rapid and inevitable extinction for the species that created it. And, as we've seen in our own domestic example, the people developing AI might not even be trying to avoid that.
Yahoo!tech digging deep for a story there.
A clandestine group using a model to aid in developing a nasty disease seems like the most likely uh oh scenario right now. Either that or an accidental scenario where AI was used somewhere it shouldn't have been.
Biowarfare has been publicized as something evil geniuses can cook up in a garage for decades now. I think that narrative has been perpetuated by the governments and other large organizations which run clandestine labs...
It's not that difficult anymore, gene splicing is now something that can be done at home. Probably the hardest part is getting one's hands on the material. There was that biolab that was just busted this year in Las Vegas.
Good. It's only a tool, not a deity. Only tools and fools think it's one.
I need to play Detroit again
The rich sees these warnings as about "alignment issues".
Could it please happen now? Would really like it if it could happen before the world ends.
Like it watches Wargames and thinks it can do better than the WHOOPR and actually launches real test cases with nukes?
And every new AI center is going to have its own small nuke plants... What can possibly go wrong?
AI + Chernobyl = Profit!?!
I don't think it will be Chernobyl or 9/11 level, but I do think an epic fuckup with potential casualties is going to set AI back ten years. And maybe by then we'll have figure out our shit enough to make it not destroy society.
It's the same cycle you see with deregulation. Dereg good, until so much dereg that something epically fucks up. Then it's "why did no one prevent this" and regulations are back again.
Then it’s “why did no one prevent this” and regulations are back again.
then the regulations need to be stronger and the guillotines sharper.
Men of Iron moment?
This is propaganda. Its the left's version of "roving caravans of gang members coming into the country"
Are they?
Can you lay out for me what they are doing, and who does it benefit?
Because AI is here. It's like saying you're going to stop oil production in Texas. It looks like the left is being used to stall progress in an emerging competitive tech industry that will shape the next century of power. Looks like a primary generator is a website that is heavily tied to Chinese propaganda.
You mistranslated what I wrote.
@lemmy.world
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
go to feed...
@lemmy.world
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
go to feed...
AI has had like 14 Chernobyl moments. Wasn't the US using AI to bomb Iran? That include a whole school of girls IIRC. And guess what? No one cares.
save