LLM Inference Dashboard

Working on a resource dashboard, rich logs, lightweight 64mb cap, all local, scales on network API endpoints via collector, supports multiple engines (llama, strata, custom cuda engines, unsloth, LMS)

It’s better logging and metrics then the default endpoint api provides. If you’re like me, you don’t just use 1 engine

Its live: https://github.com/T-Crypt/speculum