AI & Agents

NVIDIA Releases Nemotron-Labs-Sigma, a Continuous Diffusion Language Model

Leave a comment

NVIDIA has released Nemotron-Labs-Sigma, MarkTechPost reports, a family of continuous diffusion language models with 14 open checkpoints at 3B and 8B. Unlike discrete diffusion models that jump between tokens, Sigma denoises text in a learned 16-dimensional embedding space. NVIDIA’s own project page describes the family as aimed at reasoning and coding, and the checkpoints are on Hugging Face under the NVIDIA Nemotron Open Model License. The collection includes base and instruct models at both sizes, few-step distilled variants, and self-speculative checkpoints. Context is 4,096 tokens, or 8,192 for the 8B instruct model.

NVIDIA’s evaluations put the Sigma-8B base ahead of several diffusion baselines on its 6-task suite, and the company reports that the 8B self-speculative model averaged a 3.46x speedup over autoregressive decoding on a GB200. The cards say the models need a CUDA GPU in bfloat16; CPU inference is not supported. Those scores and speedups are NVIDIA’s own. I have not run Sigma for this post.

Review before you install: read the release notes yourself, back up your config, and try new versions somewhere you can afford to break first.

My take

Fourteen open checkpoints is a lot of surface area for a new generation style. I will start from the official collection and the research page, not from a vendor average, if I try one. This is GPU-only, and I have not installed it.

God Bless, and Happy hacking!

References & Further Reading

Subscribe (RSS) Follow AI & Agents via RSS

Leave a note

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.