Stanford researchers, with co-authors at Toronto, Penn and MIT, have released Memorilla, MarkTechPost reports: an open-source memory system that reads a whole document collection and hands a frozen language model just 16 memory tokens. The code, training recipes and checkpoints are public under Apache 2.0.
The trainable piece is a 194-million-parameter memory module. A frozen Qwen3-Embedding-4B encoder embeds each document once, 16 learned slots attend to the question and the store, and those slots are prepended to a frozen Qwen3-8B decoder. On FactKG, they report a frozen Qwen3-8B scoring 84.33% with Memorilla against 80.43% with every document in context, using far fewer document tokens. Those numbers are the team’s own. The quickstart wants a single 80 GB GPU, and there aren’t quantized builds in the write-up.
Review before you install: read the release notes yourself, back up your config, and try new versions somewhere you can afford to break first.
Isolate it: try new software in a disposable VM or on a spare machine first, with a snapshot you can roll back to. Don’t point it at a machine you care about.
My take
Sixteen tokens instead of the whole pile is an appealing idea if it holds up outside the paper’s benchmarks. I haven’t run this, and I’m not going to pretend I have. If you try it, treat the scores as a starting point, and do it on a machine you can afford to load an 80 GB model onto, not on anything you care about.
God Bless, and Use the Source!