LunarWing — self-hostable AI agent framework built in Rust, focused on privacy and real secret management

5 days ago by lunarwingorg to c/localllama

Hello, Lemmy enjoyers. I've been working on LunarWing, a FOSS agentic software framework written in Rust that's designed from the ground up to be fully self-hosted. No cloud dependencies, no SaaS, no phoning home. It runs entirely on your own infrastructure. LunarWing heavily prioritizes models which route over a local routing gateway (Tensorzero)

The core ideas I want to highlight here for brevity:

  1. AI agents connecting to private, secure communication channels like DarkIRC (an encrypted p2p overlay network), self-hosted XMPP with OMEMO, and other various freedom respecting communication channels. Tools, channels, extensions, and bridges which connect to proprietary platforms, especially hosted by companies with nefarious intentions are intentionally unsupported as part of the main repository to ensure user freedom (You can read more about this on the project's README.md)

  2. Genuine value for secret preservation. The only other project that takes this seriously at all is NearAI's Ironclaw.

  3. a genuine unique take on AI agents, what they are, and what defines them

  4. REAL GENUINE SYSTEMS LEVEL ENGINEERING to make everything stable and robust!

For more information on my philosophy, you may read the MANIFESTO.md file in the repo.

More information about LunarWing:

It has built-in secret management with specialized credential handling for Postgres and LibSQL backends, and a WASM plugin system so you can extend agents with your own tools without touching the core.

Additional things you may be interested in:

  • Works (with tools) with any local model inference on consumer GPUs (I can run an entire multi-tenant instance on another machine on my LAN with 3 actual tenants with their own scheduled routines all on a PNY RTX 4090 with llama-server, routed over tensorzero without the need of a proxy or functional tool calling middleware)

  • Uses rootless podman by default for worker containers and database

  • AGPL-3.0 licensed, no CLA, lunarpunk ethos. Will always be open and free. We support freedom.

  • Actual self healing capability that nothing else has, created by a professional systems engineer (Newest blog post is about this)

Multiple Disclaimers Here: It's a hard fork of NearAI's IronClaw with significant divergence since < IronClaw 0.23.0. This explains why certain things in the repository do not necessarily fit in with the stated philosophy. I've been gradually yanking out the egregious bits over time (Slack, Discord, Google extensions), little by little, to ensure stability. I will continue proceeding with this operation across each new release. I started this project in Febuary. I was initially designing custom tools and channels in Rust for Ironclaw itself and kept a local branch that I've been making a myriad of changes ever since, long before deciding to hard fork and make this into an official project, for multiple reasons). I've been running it on my own homelab for months as my daily driver and across multiple testing environments. I reference "We" in the documents but I've been working on this all by myself, sharing it recently to a few close friends and family, and hoping that more people discover the project and begin to contribute. I am especially interested in those with more Rust experience than I have who can help provide polish, modernization, and suggestions on which libraries I can completely throw away now that I've stripped many of the proprietary channels and extensions from the core project. I believe LunarWing is unique in multiple ways that other projects simply cannot match on a technical level (or straight up refuse to). Instead of spending time developing features that every other project does, I'm focusing on what really matters, while paying attention to what some other projects do correctly or seem the most useful and interesting. I focus on aspects of agentic software that most LunarWing rivals have not spent nearly enough time on. You could say I am fairly opinionated in the decisions that I make. I am hoping to find like-minded indiviuals who can also share some of these opinions on what an "AI agent" should be. This is my third lemmy post on LunarWing.

Website: https://lunarwing.org/ Source: https://github.com/LunarWingOrg/lunarwing Community Discussion: IRC: #lunarwing on irc.libera.chat (port 6697, TLS) Happy to answer any questions about the architecture or setup. I'll periodically check back on lemmy this week. I would especially be interested in discussion of my plans further down the line to improve LunarWing, including the new features and changes I would like to make. I am open to suggestions as well. I have a small roadmap document in the docs section of the repo as to what I want to add for the foreseeable future. you can take a look at that too if you want. I've also been trying to keep up with the documentation of known bugs as well as the fixes for each.

PS: I've personally recently written a very short blog post about the importance of local models and tooling here:

https://blog.lunarwing.org/...

It only briefly touches on my philosophy for LunarWing near the end, but the point is made.

I will periodically create new blog posts at https://blog.lunarwing.org/ to discuss the direction of the project, my philosophical thoughts, and more news related to LunarWing.

I appreciate the traction I have made across the other lemmy communities I've posted this too. I have browsed the locallama subreddit previously in the past and figured some of you may be interested in this.

Thank you for reading.

load all comments
Sims 3 points 4 days ago

Good thing with open/dark channels. It have been disappointing to see all the 'claws' and shit with gateways prioritized around proprietary channels and services.

path: 0 24325112, hotness: undefined, score: 3, children: 0
localllama
localllama

@sh.itjust.works

login for more options
4838
513
447

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

Rules:

Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.

Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.

Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.

Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.

go to feed...