Local AI Server Guide

LM Studio — the quick-start alternative

LM Studio(opens in a new tab) is a standalone desktop application that downloads, manages, and serves local AI models — similar to the Ollama + Open WebUI combination but packaged as a single install with a GUI. No Docker, no terminal commands. It has its own built-in model browser, a chat interface, and an OpenAI-compatible API server you can point WaxFrame at.

Use this when: you want the simplest possible path to a working local hive. Good for getting started quickly or for users who prefer a GUI.

You can use both at the same time

LM Studio and Ollama use different default ports (LM Studio: 1234, Ollama: 11434). They can coexist on the same machine and serve different models. WaxFrame's Import from Model Server has Quick Add presets for both.

Check your GPU and memory

Local AI models run on your graphics card (GPU), not your CPU. The critical resource is VRAM — the memory available to your GPU. The more you have, the larger and smarter the models you can run, and the more models you can load simultaneously for a WaxFrame hive session.

Windows (NVIDIA GPU):

1. Task Manager: Press Ctrl+Shift+Esc → Performance tab → GPU. Look for Dedicated GPU memory — that is your VRAM.
2. PowerShell:

   nvidia-smi

   This shows your GPU name, total VRAM, and current usage.

Mac (Apple Silicon):

1. About This Mac: Apple menu → About This Mac. Look for Memory — on Apple Silicon, this is your total unified memory (shared between CPU and GPU).
2. Terminal:

   sysctl hw.memsize

   This shows total system memory in bytes. Divide by 1,073,741,824 to get GB.

Either platform (Ollama installed):

ollama ps

This shows which models are currently loaded, how much memory each uses, and whether they are running on GPU or CPU.

How many models can you run simultaneously?

WaxFrame's hive needs multiple AIs running at the same time — at minimum two (one worker and one Builder), ideally three or more. Each loaded model occupies memory until it is unloaded. Here is a rough guide:

NVIDIA GPUs (Windows/Linux):

Apple Silicon Macs (unified memory):

OLLAMA_MAX_LOADED_MODELS — controlling simultaneous models

By default, Ollama loads one model at a time and unloads the previous one. For a WaxFrame hive you need multiple models loaded simultaneously. Set this environment variable to tell Ollama how many to keep in memory:

Windows (PowerShell):

[Environment]::SetEnvironmentVariable("OLLAMA_MAX_LOADED_MODELS", "3", "User")

Verify with:

[Environment]::GetEnvironmentVariable("OLLAMA_MAX_LOADED_MODELS", "User")

Mac/Linux (Terminal):

launchctl setenv OLLAMA_MAX_LOADED_MODELS 3

To make it persist across reboots on Mac, add this line to your ~/.zshrc (or ~/.bashrc on Linux):

export OLLAMA_MAX_LOADED_MODELS=3

After setting the variable on either platform, restart Ollama (quit from the system tray / menu bar and reopen it).

Set this to however many models you want loaded at once. For 16 GB of available model memory, 3 is a reasonable starting point with 3B-class models. If you set it higher than your memory can handle, Ollama will spill models to swap and performance will drop dramatically.

Confirm Ollama is working

Open PowerShell and check that Ollama is installed and has models available:

ollama --version

Then list your installed models:

ollama list

You should see the models you already installed. Test the API:

Invoke-RestMethod http://localhost:11434/api/tags

If that returns model information, Ollama is good.

Check whether Docker Desktop is installed

Run:

docker --version

If you get something like Docker version 28.x.x, skip ahead to Install and Start Open WebUI.

If PowerShell says docker isn't recognized, install Docker Desktop next.

Verify the container started

Run:

docker ps

You should see something similar to:

CONTAINER ID   IMAGE                               PORTS
xxxxxxxxxxxx   ghcr.io/open-webui/open-webui:main  0.0.0.0:3000->8080/tcp

Open Open WebUI

In your browser, open:

http://localhost:3000

You should get the Open WebUI setup screen.

Start chatting

Pick one of your Ollama models from the model dropdown and enter a normal prompt.

Ollama does the model inference; Open WebUI is the interface, conversation manager, knowledge/document system, tool layer, and provider manager sitting in front of it.

Verify the connection

After adding models, you should see them in your Worker Bees grid. Click Test All Keys — each imported model should show a green connectivity pill.

If a model shows red, check:

1. Docker Desktop is running and the open-webui container is started (docker ps).
2. Ollama is running — open a terminal and run ollama list.
3. You entered the right address — http://localhost:3000 (not https, not 8080).
4. CORS is configured — if you see a CORS error in your browser console (F12 → Console), your container may not have the CORS_ALLOW_ORIGIN flag. Recreate it using the install command above.
5. API key rejected (401) — regenerate using the console command above. Make sure you copy the key without any surrounding quotes.

Once your models are green, you're running hive sessions from waxframe.com against your local AI stack — no cloud API keys, no per-token costs, your data never leaves your machine.

Starting Open WebUI later

You normally don't have to manually start it. We used --restart always, so Docker will restart the Open WebUI container automatically.

If needed:

docker start open-webui

To stop it:

docker stop open-webui

To restart it:

docker restart open-webui

To check its status:

docker ps -a

View Open WebUI logs

If something breaks:

docker logs open-webui

Or watch them live:

docker logs -f open-webui

Press Ctrl+C to stop watching.

Update Open WebUI later

Open WebUI's documented Docker update procedure: remove the old container, pull the current image, and recreate it while retaining the persistent volume.

docker rm -f open-webui

Then:

docker pull ghcr.io/open-webui/open-webui:main

Then recreate it with the same flags as your original install:

docker run -d `
  -p 3000:8080 `
  --add-host=host.docker.internal:host-gateway `
  -v open-webui:/app/backend/data `
  -e CORS_ALLOW_ORIGIN=https://waxframe.com `
  --name open-webui `
  --restart always `
  ghcr.io/open-webui/open-webui:main

Your persistent Open WebUI data remains in the open-webui volume because that Docker volume wasn't deleted.

Connecting LM Studio to WaxFrame

1. Download and install LM Studio(opens in a new tab) and download a model from the built-in browser.
2. Load the model and click the Local Server tab (or the server icon in the sidebar).
3. Start the server — by default it runs on http://localhost:1234.
4. In WaxFrame, go to Setup → Import from Model Server. Use the LM Studio (local) Quick Add preset, which fills in:
   • Chat Endpoint: http://localhost:1234/v1/chat/completions
   • Models Endpoint: http://localhost:1234/v1/models
5. Leave the API Key blank (LM Studio's local server does not require one by default).
6. Click Fetch Models → select → Add to Hive.

Custom Modelfiles — tuning context size

Ollama's default context window (2048 tokens) is too small for most WaxFrame sessions. You can create a custom model variant with a larger context window using a Modelfile.

Create a text file (e.g. Modelfile.mymodel) with this content:

FROM qwen2.5:3b
PARAMETER num_ctx 16384

Then build it:

ollama create my-qwen25-16k -f Modelfile.mymodel

This creates a new model variant called my-qwen25-16k that uses qwen2.5:3b as its base but with a 16K context window. The variant is a thin wrapper — it does not re-download the model weights.

Context size vs. VRAM tradeoff: A larger context window means the model can see more of your document and the worker feedback at once, which is important for Builder quality. But it also uses more VRAM. 16384 (16K) is a good balance for consumer GPUs. Going to 32768 (32K) uses noticeably more VRAM — check with ollama ps after loading to see the actual impact.

Swapping models

To replace a loaded model with a different one, just run the new model — Ollama will unload the least-recently-used model if VRAM is full:

ollama run qwen2.5:7b "hi"

Then update WaxFrame: go to Setup → Import from Model Server → Fetch Models and add the new model to your hive. Remove the old one from your Worker Bees list if you no longer want it.

To check what is currently loaded at any time:

ollama ps

What we tested and why it matters

We spent three days testing every viable open-source model combination on an NVIDIA RTX 4080 SUPER (16 GB VRAM) to find out which models can actually handle WaxFrame's hive protocol — specifically, the Builder role. The Builder has strict requirements: it must wrap the updated document in %%DOCUMENT_START%% / %%DOCUMENT_END%% delimiters, it must include a %%CONFLICTS_START%% section (or report NO CONFLICTS), it must not echo back prompt metadata or instructions, and it must synthesize worker feedback into a coherent revision rather than just concatenating suggestions.

Cloud AIs (ChatGPT, Claude, Gemini) handle all of this easily — they have enough intelligence and instruction-following ability to understand the protocol and produce clean output. Local models at the 3B-7B parameter range struggle significantly. The table below shows every model and combination we tested, what went right, and what went wrong.

"Thinking" models — hidden token costs

Some models (qwen3, qwen3.5, SmolLM3) include a hidden chain-of-thought step before generating their visible response. The model "thinks" internally — you do not see this text, but it consumes tokens and time. In our testing:

• qwen3:4b burned 4,000-14,000 invisible tokens per response. It was the only model to achieve convergence and report real conflicts, but each response took 52-144 seconds — too slow for practical use.
• SmolLM3 thought more cheaply than qwen3, but the overhead scaled with prompt complexity. By Round 11, it produced 1,483 tokens for just 27 words of visible output (a 55:1 ratio).
• qwen2.5 (the generation before qwen3) has zero thinking overhead. Its responses are fast but it lacks the reasoning depth that thinking provides.

If you see a model responding slowly despite showing 100% GPU, check whether it is a thinking model. You cannot disable the thinking in most cases — it is baked into the model architecture.

The bottom line — when local AI hives make sense

Local models work for the mechanics — WaxFrame connects, rounds execute, documents update, responses come back fast. On a 16 GB GPU, you can run 2-3 models simultaneously with sub-15-second response times and zero API cost.

Local models struggle with the intelligence — at 3B-7B parameters, no model we tested could match what a cloud AI does as Builder. No convergence, no meaningful conflict detection, no critical evaluation of worker suggestions. The hive runs but the output quality does not match what you get from a ChatGPT/Claude/Gemini hive.

Where local hives shine today:

• Air-gapped environments where cloud access is not an option (defense, healthcare, classified networks). A larger GPU or a proper AI server with 48+ GB VRAM running 14B-70B models will produce substantially better results.
• Learning and experimentation — understanding how WaxFrame works, testing setup flows, building familiarity before committing API spend.
• Style and tone refinement on short documents where the content is opinion-driven rather than factually anchored — the intelligence gap matters less when there is no "right answer" to converge on.

Where cloud hives are still the right choice: anything where convergence, critical evaluation, or document quality matters. Cloud AI costs pennies per round — a full 10-round hive session with five cloud AIs typically costs $0.10-$0.50 depending on document length and provider.

As local model quality improves (and it is improving fast), this gap will narrow. The infrastructure you set up today — Ollama, your model server, your WaxFrame configuration — will run better models tomorrow without changing anything.