AI Assistant

Free model on this server

Run the AI assistant on a free open model installed on the panel's own server with Ollama — no API key, nothing leaves the server, reachable only by the panel.

Last updated

The assistant can answer with a small open model that runs on the panel's own server. It needs no API key and nothing leaves the server. Answers are slower and simpler than Claude's: fine for checking status and logs, weaker at harder problems.

This needs VaultPanel 0.3.0 and agent 0.41.0 or newer on the panel's own server (This server), which must be online.

Install a model

Open the provider settings

Open System > Assistant Settings and choose Free model on this server. Under Models on this server, the page shows the server's free memory and disk space.

Pick a model and click Install

The model marked Suggested for this server is the largest one that leaves about 1.25 GB of memory for the rest of the server. Models the server cannot hold are disabled.

The first install also downloads Ollama itself (about 1.6 GB, checked against its known SHA-256), so it takes a few minutes more. The progress shows in a drawer as the job runs.

Choose it and turn the assistant on

Select the installed model as the Model, tick Turn the assistant on and save.

Models

All models are free to use commercially (Apache 2.0) and can call tools.

ModelIdDownloadMemoryNotes
Qwen3.5 2Bqwen3.5:2b-q4_K_M1.8 GB2.5 GBThe smallest and fastest. Simple answers; for servers with 4 GB of memory.
IBM Granite 4.1 3Bgranite4.1:3b2.0 GB3.5 GBGood at using the panel's tools for its size. Text only.
Qwen3.5 4Bqwen3.5:4b3.1 GB4.5 GBBetter answers, a little slower. For servers with 8 GB of memory.
Qwen3.5 9Bqwen3.5:9b6.1 GB8 GBThe best answers here, but slow without a strong CPU. For servers with 16 GB of memory. Never suggested automatically.

Memory is about what the model needs loaded with its context. Installing needs free disk space for the model plus 1 GB to spare, and about 2 GB more the first time for Ollama. Qwen3.5 4B and 9B reason before answering by default; the panel turns that off, because on a CPU it would take minutes per step.

What gets installed

WhatWhere
Ollama (the CPU build of the release the agent carries)/opt/vaultpanel-ai (bin/ollama, VERSION)
Models/var/lib/vaultpanel-ai/models, owned by the vaultpanel-ai system account
The servicevaultpanel-ai.service: ollama serve as vaultpanel-ai, on 127.0.0.1:21434 only
The firewall rule/opt/vaultpanel-ai/guard, run as root as the service starts and after it stops

The service runs one model and one answer at a time, with an 8,192-token context, at a lower CPU priority than websites, with a read-only system except its data folder and Ollama's cloud features off. Its memory is limited to the server's memory less 1 GB, so a model too big for the server stops itself rather than the panel's database, and it is the first process the kernel stops when the whole server runs out.

Isolation

Ollama has no password, so anything on the server could otherwise use it, including websites' and apps' code. The guard loads an nftables table of its own, inet vaultpanel_ai, whose rule resets every connection to port 21434 except from root (the agent) and the panel's own account.

  • Being its own table, the rule stays when UFW is turned on or off, when Docker rewrites its rules and when iptables chains are flushed.
  • The service does not start if the rule cannot be loaded, so Ollama never listens without it. nftables is installed first when nft is missing.
  • Port 21434 is outside the app ports hosting customers get (10000 to 19999), and the panel refuses it for containers on its own server.

Remove models or Ollama

  • Remove next to an installed model deletes it and frees its disk space. The model the assistant answers with cannot be removed; choose another first.
  • Remove Ollama and all models deletes Ollama, its service and every model. It is disabled while the assistant uses a model on this server.

You can install them again later.

Troubleshooting

MessageWhat to do
The panel's own server is not connected to the panelThe model can only be installed on the server the panel runs on, connected as This server.
The agent on the panel's own server cannot install the model yetUpdate the agent from the server's Overview tab.
Ollama is installed but does not answerIt may still be starting; installing a model again restarts it. Check journalctl -u vaultpanel-ai on the server.
No answer after 30 minutes (ASSISTANT_TIMEOUT)The worker may not be running, or the model is too slow for the server. A smaller model answers faster.
The model on this server did not answer within 10 minutesOne request took too long. Choose a smaller model, or free memory on the server.