https://github.com/glanceapp/glance

The model is an open weight model, Google or Open AI did not create it. I use my electricity at home, So no it doesn't

Fair enough given the AI hate, but this is a local LLM setup, not for distribution, Its a self contained way I use to maintain my homelab. Some may find it useful, some may not.

Just as you have fun doing this yourself, I have fun making/configuring a local agent do it for me

An Open-Weight Model is an AI model whose core components are publicly released, allowing anyone to download it. This lets users run the model on their own computers, study how it works, and even modify it for their own specific needs.

Open-source models provide complete access to the entire model architecture, training methodology, and weights. This comprehensive access includes the model code, architecture design, training scripts, and parameter weights under licenses like MIT or Apache.

Open-weight models represent a more limited approach to model sharing. These models release only the trained parameter weights while restricting access to training methodologies and code. Also under MIT or Apache licenses

https://codeberg.org/ranjan/glance

The local LLM is run on the homelab, just like immich is run on your homelab and doesnt talk to google photos is any way, its the same for my model, self contained, inhouse with no data leaving my network

Yes, the download link for the model is in the original post

At 56, i was laid off from a Fortune 500 company, so i hear you. Today I am without a job just trying to learn and keep up everyday.

Edit:Spellings

my sudoers file restricts what the llm can actually do, also I have robust backups can can spin up any of my servers really quickly, I am not that worried and just like you deal with human errors, you can deal with agent errors.

so far this has been running for a month, no scares or unexpected behaviour other than looping on a task somethimes

It has a yaml based config structure, more details here https://gethomepage.dev/configs/

its a different workflow, takes a bit to get used to. Since you use KDE look up karousel plugin, you could test out the scrolling work flow in KDE and then install Niri if you like it

Niri is a scrolling window manager. Its workflow is diffent from a tiler. Maybe this video might help. https://youtube.com/watch?v=DeYx2exm04M

i like the simplcity of waybar+mako+rofi much better than all-in-one-shells. Besides i believe in "do one thing best" policy of the smaller leaner apps

dots added in original post

No this is https://gethomepage.dev/

Sure give me sometime and i'll post my dots

glanceapp in firefox https://github.com/glanceapp/glance

Nice write-up. Thanks for also including all the numbers. If I might ask: What is the thermal/throttling behaviour you mention? Is it still within the laptop’s thermal budget? Or does it reach throttling territory when doing inference on a long context window?

Thanks! Nowhere near throttling territory. The thermal ceiling is 95°C, and my sustained load is 72-78°C. There's a 17-23°C safety margin.

The laptop never throttles. But the fans also don't spin up properly during GPU-heavy inference unless I force it.

Why? ASUS's embedded controller monitors CPU temperature and CPU package power for fan curves. It largely ignores GPU temperature.

I use Radicale self hosted caldav