The Rise of Overfit Inference Engines

There seems to be a whole category of extremely narrow inference runtimes appearing: Strata, ninfer, DwarfStar, Splash, llamAmpere, gufo, etc. They deliberately give up the thing llama.cpp/vLLM are great at - generality - and optimize around a small number of models and
sometimes one hardware family e.g. Strix Halo

It seems that general runtimes for compatibility, disposable overfit runtimes for maximum performance is going to be the norm forward.

This is actually another good step in helping the democratization and decentralization of intelligence (models and runtimes both) and extracting more out of existing hardware where it doesn't have to be beautiful, well written, as long as it gets maximum output from one particular configuration.

Curious if people think this the future/norm.