Would you use LoRAs as persistent, switchable skills instead of relying entirely on context/RAG?

We're currently building a tool around an idea we're trying to validate: using LoRA adapters as a way to give an LLM persistent, specialized capabilities that can be switched on and off when needed.

The basic idea is that instead of continuously putting a skill or domain-specific information into the model's context, you could encode some of it into a LoRA adapter.

For example, you might have separate adapters for:

• a coding skill
• a company's internal domain
• a specific writing style
• domain-specific knowledge
• task-specific behavior

…and load or unload those capabilities depending on what you're doing.

We're interested in this because it could potentially mean less context usage, reusable specialized capabilities, keeping different capabilities separated from the base model, and potentially lower inference costs for some use cases.

But we're not sure yet whether this is actually a useful product.

That's what we're trying to figure out before going too far with the build.

If creating and managing these LoRAs were as easy as creating and managing a knowledge base, would you actually use something like this?

I'm particularly interested in hearing where you think this approach doesn't make sense.