AI & Agents

Stanford’s Memorilla Turns a Document Collection Into 16 Memory Tokens

Leave a comment

Stanford researchers, with co-authors at Toronto, Penn and MIT, have released Memorilla, MarkTechPost reports: an open-source memory system that reads a whole document collection and hands a frozen language model just 16 memory tokens. The code, training recipes and checkpoints are public under Apache 2.0.

The trainable piece is a 194-million-parameter memory module. A frozen Qwen3-Embedding-4B encoder embeds each document once, 16 learned slots attend to the question and the store, and those slots are prepended to a frozen Qwen3-8B decoder. On FactKG, they report a frozen Qwen3-8B scoring 84.33% with Memorilla against 80.43% with every document in context, using far fewer document tokens. Those numbers are the team’s own. The quickstart wants a single 80 GB GPU, and there aren’t quantized builds in the write-up.

Review before you install: read the release notes yourself, back up your config, and try new versions somewhere you can afford to break first.

Isolate it: try new software in a disposable VM or on a spare machine first, with a snapshot you can roll back to. Don’t point it at a machine you care about.

My take

Sixteen tokens instead of the whole pile is an appealing idea if it holds up outside the paper’s benchmarks. I haven’t run this, and I’m not going to pretend I have. If you try it, treat the scores as a starting point, and do it on a machine you can afford to load an 80 GB model onto, not on anything you care about.

God Bless, and Use the Source!

References & Further Reading

Subscribe (RSS) Follow AI & Agents via RSS

Leave a note

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.