Unlock Advanced AI Chat with OpenWebUI and OpenRouter

This chapter will guide you through installing Docker CE and setting up OpenWebUI using Docker Compose. Docker simplifies application deployment by packaging software into isolated units called containers. Docker Compose allows you to define and run multi-container Docker applications with a single command, making complex setups like OpenWebUI easier to manage than individual docker run commands.Assumptions/Prerequisites:

• You are using a Debian-based Linux distribution (e.g., Ubuntu, Debian).
• You have sudo privileges.

Step-by-step Installation:

1. Install Docker CE:

sudo apt update
sudo apt install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

2. Add your user to the docker group (log out and back in after this):

sudo usermod -aG docker $USER

3. Create a directory for OpenWebUI and the docker-compose.yml file:

mkdir openwebui
cd openwebui

4. Create the docker-compose.yml file:

nano docker-compose.yml

Paste the following content into the file:

version: '3.8'
services:
  openwebui:
    image: ghcr.io/open-webui/open-webui:latest
    ports:
      - "8080:8080"
    volumes:
      - ./ollama_webui:/app/backend/data
    environment:
      - OLLAMA_BASE_URL=http://your_ollama_ip:11434 # Replace with your actual Ollama IP if it's on a different host. For local, remove this line or set to http://host.docker.internal:11434 if Ollama is on the host machine.

5. Start OpenWebUI:

docker compose up -d

Verification:

* Check Docker status:

sudo systemctl status docker

* Check OpenWebUI container status:

docker compose ps

You should see ‘Up’ next to the openwebui service.
* Access OpenWebUI: Open your web browser and navigate to http://localhost:8080 (or http://your_server_ip:8080).

Common Failure Modes and Fixes:

* “command not found: docker” or “permission denied”: Ensure you logged out and back in after adding your user to the docker group (step 2). If issues persist, try sudo systemctl start docker.
* OpenWebUI container not starting: Check docker compose logs openwebui for errors. Ensure port 8080 isn’t already in use by another application. Adjust the host port in docker-compose.yml (e.g., "8081:8080").
* OpenWebUI unreachable: Verify your firewall isn’t blocking port 8080. If using a cloud server, ensure security groups allow traffic on 8080.

Assumptions/Prerequisites:

• You have OpenWebUI running via Docker as described in the previous chapter.
• You have an internet connection to access OpenRouter.ai.

Step-by-step:

1. Register and Obtain OpenRouter API Key: Visit OpenRouter.ai and register for an account. After logging in, navigate to your account settings or API key section to generate a new API key. Copy this key; it typically starts with sk-or-.
2. Configure OpenWebUI with OpenRouter API Key: Open your OpenWebUI instance in a web browser. Click the settings icon (gear) in the bottom-left corner. Navigate to Connections. Click Add New Connection. Select OpenRouter. Paste your copied API key into the “API Key” field. You can leave the “Base URL” as default. Click Test to verify the connection, then Save.
3. Select and Use OpenRouter Models: Once the connection is saved, go back to the chat interface. You should now see various OpenRouter models available in the model selection dropdown menu at the top of the chat window. Select your desired model (e.g., gemma-7b-it, mistralai/mistral-7b-instruct, etc.) and start interacting.

Verification Commands:

• Verify the OpenWebUI Docker container is running: docker ps | grep openwebui
• The primary verification is observing the OpenRouter models in OpenWebUI’s dropdown and successfully receiving responses from them.

Common Failure Modes + Fixes:

• Invalid API Key: If OpenWebUI reports a connection error or models don’t load, double-check your API key for typos or missing characters. Regenerate the key on OpenRouter if necessary. Make sure there are no leading/trailing spaces when pasting.
• Service Unavailable / Network Issues: Ensure your server has active internet access. Check OpenRouter’s status page for any outages.
• OpenRouter Model Quotas: Some models on OpenRouter may have usage limits or require specific permissions. If a model consistently fails to respond, try a different one or check your OpenRouter account dashboard.

This integration unlocks a vast array of cutting-edge AI models within your self-hosted OpenWebUI, offering enhanced capabilities and flexibility without tying you to a single provider. It demonstrates the power of open-source tools and unified APIs, providing a robust, adaptable, and future-proof AI interaction platform. Experiment with different models to discover the best fit for your needs and continue exploring the endless possibilities of self-hosted AI.