Please ELI5 the arguments people use to claim it's possible to use AI/LLMs ethically, locally, openly etc.

6 days ago by A_norny_mousse to c/asklemmy

I usually don't even understand the lingo they use. "Open-weighted" is the most recent one, then it usually goes down to specific "models" that everybody is supposed to know about.

These are my thoughts (I will stick to the vague "it" for now, but of course therein lies another question: "and how does all this apply to various specialised AIs"):

Is it really feasible to run it 100% locally? I know there's plenty of people with very powerful rigs indeed, but still. Or are 99% of these people really saying "it would, in theory, be possible to run that locally, therefore your concerns are invalid"?
If yes to the previous: the software doesn't come from nowhere and ultimately still relies on gas-turbine-powered datacenters and stolen IP and stolen personal data, no?

If what I wrote above is true, what exactly are people arguing when they say it's still possible to use LLMs ethically or true to FOSS philosophy, because ... ???

edit

Thanks to all who answered.

I guess it's my fault for asking several questions in one, but this thread has attracted exactly the type of people I'm writing about; several even used the term "open-weighted models" without explaining it.

Asking to get arguments explained, I got more arguments instead.

load all comments

NoLemurs 12 points 6 days ago

It is absolutely possible to run 100% locally, but in practice, at the high end, only quantized models. The full size top end models require hundreds of GB of video ram, and while you can buy that, it's stupidly expensive. Quantized models can often perform nearly as well with a small fraction of the ram, but they do sacrifice a little in precision.

These models (well the good ones) ultimately all trace their origins to what you'd likely consider "stolen" data. Whether that's ethical is debatable. If you're in the "information should be free" camp, there may not be an issue here.

As for the power/environmental impact, for what they do LLMs are actually very low impact per-request. If you're concerned about your personal AI power use, then I hope you never fly in an airplane, and minimize your driving because those are much bigger issues.

It's the scale of use that makes AI an environmental problem, and that's a question about corporate use of AI, not personal use of AI.

save

path: 0 26189080, hotness: undefined, score: 12, children: 6

A_norny_mousse 1 point 6 days ago

Thanks. What are quantized models?

save

path: 0 26189080 26194082, hotness: undefined, score: 1, children: 2

NoLemurs 2 points 6 days ago

I'm going to simplify a little here, so don't take this completely at face value.

Models are, quite literally, long series of numbers (called weights). A model might store the weights in 16 bit numbers (that is, 16 binary digits). The size of the model (and how much memory it needs) is determined by how many weights there are, and how many bits each weight takes. You can take a 16 bit model and rework it to use 8 bit, or even 4 bit numbers. The result intuitively behaves a lot like the same model, but with less precision to the weights. That makes the model take way less space in ram, but also makes it more likely for concepts (encoded in the weights) to overlap, which impacts model quality. Often the effect is that fine distinctions get lost.

save

path: 0 26189080 26194082 26195784, hotness: undefined, score: 2, children: 0

A Fediverse community for open-ended, thought provoking questions

Rules: (interactive)

Partnered Communities:

A Fediverse community for open-ended, thought provoking questions

Rules: (interactive)

Partnered Communities: