Curious, how do you evaluate the performance without breaking the law?

On Lemmy, everything is a bit leftist at the moment.

I see Olmo by AllenAI frequently used in research. The publish all the checkpoints of training and all training sources. https://allenai.org/olmo / https://huggingface.co/collections/allenai/olmo-3

Here's the link to the dataset used to train Olmo 3: https://huggingface.co/...

It's fairly modern. Olmo 3.1 was released January 2026.

Also note that there's OnionShare, too. You don't need a TLS certificate or domain, don't need to port forward and can run it from home safely, routes over Tor so very hard to know you are even sharing something, well known and open source etc.

I followed their instructions here: https://speech.fish.audio/

I am using the locally-run API server to do inference: https://speech.fish.audio/...

I don't know about other ways. To be clear, this is not (necessarily) an LLM, it's just for speech synthesis, so you don't run it on ollama. That said I think it does technically use Llama under the hood since there are two models, one for encoding text and the other for decoding to audio. Honestly the paper is terrible but it explains the architecture somewhat: https://arxiv.org/pdf/2411.01156

Thanks for the tips. After doing a bunch of searching, I found that what I needed was BPE, or byte-pair encoding. This allows the token set to contain sub-word sequences, which lets the tokenizer represent a unique constant like 0x0373 as ['__sow', '0x', '03', '73', '__eow'].

What I have yet to learn is how much of the intelligence and accuracy comes from the model itself and how much comes from the agentic tool system. For example, my experience with ChatGPT probably would be much worse with the free version (no thinking or container).

You are right. Their description of "SOTA Open Source TTS" caused me to assume it was open source, but it's clear that

This codebase and all models are released under CC-BY-NC-SA-4.0 License.

So, it's "source available" and not released under a permissive licence.

Thanks for the answer! I hadn't thought about asking for recipes based on the specific ingredients you have left.

The config settings are passed via /props. Use the browser developer tools network tab and you should see them passed through there.

Thanks, the quickstart guide was straightforward to follow. Do you have any suggestions on how to do word splitting with code, if any? For example, on a test run, I found that the model was not able to synthesize unique constants correctly even though this test run consisted only of obvious "a to b" relationships.

Thank you so much, that exactly answers my question with the official response (that guy works at Meta) that confirms it's the same base model!

I was concerned primarily because in the release notes it strangely didn't mention it anywhere, and I thought it would have been important enough to mention.

More importantly, what was the recipe and was it any good?

Thank you for your opinion & recommendations. Something I saw today related to "sub-agents" is in Kimi 2.6's model card it says

Elevated Agent Swarm: Scaling horizontally to 300 sub-agents executing 4,000 coordinated steps, K2.6 can dynamically decompose tasks into parallel, domain-specialized subtasks, delivering end-to-end outputs from documents to websites to spreadsheets in a single autonomous run.

So maybe Kimi 2.6 is doing the "type of thing" I am looking for, but I don't have the means to run it practically. Maybe at 1 token per second which would be brutal.

I tried out Qwen 3.6 27B but not yet in an agentic setting, so I can't really judge yet. Maybe it's just me but the small model size seems limiting. I thought gpt-oss-120b was good.

No, it's unsupervised. In pre-training, the text data isn't structured at all. It's books, documents, online sources, all put together.

Supervised learning uses data with "ground truth" labels.

Thanks for your answer. I think to be clear, what I'm looking for is a kind of masked fine-tuning. You see, I want to "steer" a particular output instead of providing complete examples, which are costly to create.

The steering would be something like this:

1. I have an LLM generate a sequence.
2. I find exactly where the LLM goes "off track" and correct it there (for only maybe 10-20 tokens instead of correcting the rest of the generation manually).
3. The LLM continues "on track" until it goes off track again.

What I would like to do is train the model based on these corrections I give it, where many corrections might be part of the same overall generation. Conceptually I think each correction must have some training value. I don't know much about masking, but what I mean here is that I don't want it to train on a few tens or hundreds of (incomplete) samples but rather thousands of (masked) "steers" that correct the course of the rest of the sample's generated text.

Yes, Opus/Fable 5. If you have the opportunity, play around with it. It writes in a format that tries so hard to be concise (while blurting out a monumental essay) that you can hardly understand it, and repeats the same phrases all the time.

This pre-training was done by Meta. It's what Llama-3.1-405B is (in contrast to Llama-3.1-405B-Instruct). https://huggingface.co/meta-llama/Llama-3.1-405B

Training Data

Overview: Llama 3.1 was pretrained on ~15 trillion tokens of data from publicly available sources. The fine-tuning data includes publicly available instruction datasets, as well as over 25M synthetically generated examples.

No, in that case there's no labelling required. That would be unsupervised learning.

https://en.wikipedia.org/...

Conceptually, unsupervised learning divides into the aspects of data, training, algorithm, and downstream applications. Typically, the dataset is harvested cheaply "in the wild", such as massive text corpus obtained by web crawling, with only minor filtering (such as Common Crawl). This compares favorably to supervised learning, where the dataset (such as the ImageNet1000) is typically constructed manually, which is much more expensive.

The article you linked to uses SFT (supervised fine tuning, a specific training technique) as its alignment strategy. There are other ways to fine-tune a model.

I guess I'm wondering if you can train on these partial responses without needing the full rest of the output, without the stop token, or if you need full examples as the article hints to.