Hello World

3 years ago by SkySyrup to c/localllama

Hi, you've found this subreddit Community, welcome!

This Community is intended to be a replacement for r/LocalLLaMA, because I think that we need to move beyond centralized Reddit in general (although obviously also the API thing).

I will moderate this Community for now, but if you want to help, you are very welcome, just contact me!

I will mirror or rewrite posts from r/LocalLLama for this Community for now, but maybe we could eventually all move to this Community (or any Community on Lemmy, seriously, I don't care about being mod or "owning" it).

scrollbars 4 points 3 years ago

Hello! This is the one community that I was a bit worried about finding an equivalent of outside of reddit. Hopefully more of us migrate over.

path: 0 40439, hotness: undefined, score: 4, children: 0
hendrik 3 points 3 years ago

thank you for using a decent platform. i doubt more than 20 people will migrate from reddit... but it make the world a better place, anyways.

path: 0 75509, hotness: undefined, score: 3, children: 0
dirac_field 2 points 3 years ago

Late to the party, but thanks for setting this up! I suspect the overlap of people both using local LLMs and hungry for reddit alternatives will be higher than average

path: 0 159119, hotness: undefined, score: 2, children: 0
mellery 2 points 3 years ago

Hello! Thanks for setting this up

path: 0 80009, hotness: undefined, score: 2, children: 0
floatingBurger 1 point 3 years ago

Thank you so much for setting uo here. Have you looked into the github project that allows cloning submittions automatically? just curious, since this must be a lot for an indicidual to do.

path: 0 437103, hotness: undefined, score: 1, children: 0
Barbarian 1 point 3 years ago path: 0 94064, hotness: undefined, score: 1, children: 0
pax 0 points 3 years ago

I could help with moderation, but I have a question, how to set up LLAma on my mac computer? any tips?

path: 0 62806, hotness: undefined, score: 0, children: 4
SkySyrup 1 point 3 years ago path: 0 62806 62807, hotness: undefined, score: 1, children: 1
gh0stcassette 2 points 3 years ago

Adding to this: text-generation-webui (https://github.com/oobabooga/text-generation-webui) works with the latest bleeding edge llama.cpp via llama-cpp-python, and it has a nice graphical front-end. You do have a manually tell pip to install llama.cpp-python with the right compiler flags to get GPU acceleration working but the llama-cpp-python github and ooba github explain how to do this.

You can even set up GPU acceleration through metal on m1 Macs I've seen some fucking INSANE performance numbers online for the higher RAM MacBook pros (20+ tokens/sec, I think with a 33b model, but it might have been 13b, either way, impressive.)

path: 0 62806 62807 199865, hotness: undefined, score: 2, children: 0
dtlnx 0 points 3 years ago path: 0 62806 75624, hotness: undefined, score: 0, children: 1
pax 1 point 3 years ago

gpt4all is dump, it even didn't tried to be smart.

path: 0 62806 75624 85707, hotness: undefined, score: 1, children: 0
localllama
localllama

@sh.itjust.works

login for more options
5316
621
618

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

Rules:

Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.

Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.

Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.

Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.

go to feed...