# Free model on this server

URL: https://docs.vaultpanel.net/guides/ai-assistant/local-model

> Run the AI assistant on a free open model installed on the panel's own server with Ollama — no API key, nothing leaves the server, reachable only by the panel.

The assistant can answer with a small open model that runs on the panel's own server. It needs no API key and nothing leaves the server. Answers are slower and simpler than Claude's: fine for checking status and logs, weaker at harder problems.

This needs VaultPanel 0.3.0 and agent 0.41.0 or newer on the panel's own server (**This server**), which must be online.

## Install a model

### Open the provider settings

Open **System > Assistant Settings** and choose **Free model on this server**. Under **Models on this server**, the page shows the server's free memory and disk space.

### Pick a model and click Install

The model marked **Suggested for this server** is the largest one that leaves about 1.25 GB of memory for the rest of the server. Models the server cannot hold are disabled.

The first install also downloads Ollama itself (about 1.6 GB, checked against its known SHA-256), so it takes a few minutes more. The progress shows in a drawer as the job runs.

### Choose it and turn the assistant on

Select the installed model as the **Model**, tick **Turn the assistant on** and save.

## Models

All models are free to use commercially (Apache 2.0) and can call tools.

| Model | Id | Download | Memory | Notes |
| --- | --- | --- | --- | --- |
| Qwen3.5 2B | `qwen3.5:2b-q4_K_M` | 1.8 GB | 2.5 GB | The smallest and fastest. Simple answers; for servers with 4 GB of memory. |
| IBM Granite 4.1 3B | `granite4.1:3b` | 2.0 GB | 3.5 GB | Good at using the panel's tools for its size. Text only. |
| Qwen3.5 4B | `qwen3.5:4b` | 3.1 GB | 4.5 GB | Better answers, a little slower. For servers with 8 GB of memory. |
| Qwen3.5 9B | `qwen3.5:9b` | 6.1 GB | 8 GB | The best answers here, but slow without a strong CPU. For servers with 16 GB of memory. Never suggested automatically. |

Memory is about what the model needs loaded with its context. Installing needs free disk space for the model plus 1 GB to spare, and about 2 GB more the first time for Ollama. Qwen3.5 4B and 9B reason before answering by default; the panel turns that off, because on a CPU it would take minutes per step.

## What gets installed

| What | Where |
| --- | --- |
| Ollama (the CPU build of the release the agent carries) | `/opt/vaultpanel-ai` (`bin/ollama`, `VERSION`) |
| Models | `/var/lib/vaultpanel-ai/models`, owned by the `vaultpanel-ai` system account |
| The service | `vaultpanel-ai.service`: `ollama serve` as `vaultpanel-ai`, on `127.0.0.1:21434` only |
| The firewall rule | `/opt/vaultpanel-ai/guard`, run as root as the service starts and after it stops |

The service runs one model and one answer at a time, with an 8,192-token context, at a lower CPU priority than websites, with a read-only system except its data folder and Ollama's cloud features off. Its memory is limited to the server's memory less 1 GB, so a model too big for the server stops itself rather than the panel's database, and it is the first process the kernel stops when the whole server runs out.

## Isolation

Ollama has no password, so anything on the server could otherwise use it, including websites' and apps' code. The guard loads an nftables table of its own, `inet vaultpanel_ai`, whose rule resets every connection to port 21434 except from root (the agent) and the panel's own account.

- Being its own table, the rule stays when UFW is turned on or off, when Docker rewrites its rules and when iptables chains are flushed.
- The service does not start if the rule cannot be loaded, so Ollama never listens without it. `nftables` is installed first when `nft` is missing.
- Port 21434 is outside the app ports hosting customers get (10000 to 19999), and the panel refuses it for containers on its own server.

## Remove models or Ollama

- **Remove** next to an installed model deletes it and frees its disk space. The model the assistant answers with cannot be removed; choose another first.
- **Remove Ollama and all models** deletes Ollama, its service and every model. It is disabled while the assistant uses a model on this server.

You can install them again later.

## Troubleshooting

| Message | What to do |
| --- | --- |
| The panel's own server is not connected to the panel | The model can only be installed on the server the panel runs on, connected as **This server**. |
| The agent on the panel's own server cannot install the model yet | Update the agent from the server's **Overview** tab. |
| Ollama is installed but does not answer | It may still be starting; installing a model again restarts it. Check `journalctl -u vaultpanel-ai` on the server. |
| No answer after 30 minutes (`ASSISTANT_TIMEOUT`) | The worker may not be running, or the model is too slow for the server. A smaller model answers faster. |
| The model on this server did not answer within 10 minutes | One request took too long. Choose a smaller model, or free memory on the server. |
