We describe an inference-time architectural enhancement for off-the-shelf foundation models that systematically reduces perplexity and boosts accuracy across generation and reasoning tasks. Our approach incurs essentially no additional latency during generation, though it requires serial processing in the prefill phase. Motivated by the fundamental limitation that state updates in feedforward transformers are bounded by model depth, our technique, recirculation, introduces a specific form of recurrence that allows the model to act as a dynamical system and track belief states. We distinguish this technique from chain-of-thought computation---which is better reserved for complex inferences rather than basic state tracking---as well as from popular depth-recurrence techniques (looping) and the costly training of recurrent transformers. We also propose and evaluate an adaptive variant of recirculation which requires only light tuning of hyperparameters while freezing the original model weights. Relative to the off-the-shelf baseline, adaptive recirculation achieves surprising gains on the Gemma3 family, including a consistent reduction in perplexity on a suite of datasets, a 21% increase in relative accuracy on GSM8k, and reliable improvements in accuracy on other downstream tasks. Our training-free approach succeeds by leveraging the model itself to inform architectural modifications, suggesting a route to architectural evolution guided by a trained network's properties rather than forced, arbitrary design choices.

Most language models use a feedforward design which makes it hard for them to track changing states or keep facts straight over a long context. When you are running inference on a local Qwen model, you might notice how it can lose track of dynamic variables or flip flops on ambiguous words. If the model figures out that the word bank means a river edge deep in the stack, the early layers processing the next few tokens have zero access to that updated context. When the response generation starts outpacing the internal semantic convergence, you start getting contextualization errors.

Turns out that you can use an architectural tweak the paper refers to as recirculation. Instead of using a strictly linear flow, you leak a tiny fraction of the activation from a deep layer back down to a shallower layer at every step to create a continuous feedback loop that acts as a dynamic system. And the architecture can use it to track its own evolving belief states. What makes this particularly interesting for local inference pipelines is that it requires absolutely zero weight changes. You also get essentially no latency hit during the generation phase, though you do have to run serial processing during prefill.

Testing this training free approach on the Gemma3 family produced some massive performance leaps with the models seeing up to a 23% drop in perplexity across various datasets and a 21% accuracy spike on the GSM8k math benchmark. They also introduced an adaptive variant where a tiny external network learns the exact amount of information to mix back for each token, pushing the metrics even higher.

What this all means is that we can extract far better reasoning and memory out of existing foundation models just by letting their internal representations talk to each other.