Fireworks AI VP Rob Ferguson told The Register that most companies shouldn’t run inference on-prem, because idle GPUs burn money. “Ten thousand GPUs” is roughly a billion dollars of capital.
The pitch
He argues the best “model” is really a portfolio: route each task to whichever model fits it best. Fireworks builds custom routers that grade task difficulty and model affinities, for example a model that’s a little better at JavaScript than Python. For sensitive work there’s a Sovereign offering.
My take
Routing between models makes sense, but remember who’s saying it: a managed inference vendor. If you can keep your hardware busy, or your data can’t leave, local still wins.
God Bless, and Use the Source!
