Mastering Ollama on Ubuntu Server A Performance Optimization Guide

Ollama lets you run AI models, like large language models, on your own Ubuntu server. This means you can use AI tools without relying on cloud services. We assume you have a working Ubuntu server with basic command-line access.

1. First, update your package list:

   sudo apt update

2. Next, install Ollama using their convenient installation script:

   curl -fsSL https://ollama.com/install.sh | sh

   This command downloads and executes the official Ollama installation script.

3. Verify the installation by running the Ollama command:

   ollama --version

   You should see the installed version number.

4. To test if Ollama is working correctly, try downloading and running a small model:

   ollama run llama2

   The first time, this will download the llama2 model. After it downloads, you’ll be able to interact with it.

Common failure mode: “command not found.” This usually means the installation script didn’t complete successfully or the path isn’t set. Restarting your terminal or SSH session can sometimes fix path issues. If not, re-run the installation command.

To optimize Ollama, understand that GPUs accelerate AI tasks significantly over CPUs, especially for larger models. VRAM (Video RAM) on your GPU dictates the largest model size you can run efficiently. More VRAM allows larger models or more concurrent smaller models. Aim for a GPU with at least 8GB VRAM for decent performance.

Assumptions: Ollama is installed and running on Ubuntu Server.

Step-by-step Optimization:

1. Check VRAM Usage: To see current GPU memory usage:

   nvidia-smi

2. Select Appropriate Models: Choose models with fewer parameters (e.g., 7B instead of 70B) if VRAM is limited. Browse models on ollama.com/library and look for model sizes.
3. Limit Parallel Runs: If your system becomes unresponsive, reduce the number of concurrent Ollama processes or models loaded.
4. Update Drivers: Ensure your NVIDIA drivers are up-to-date for optimal GPU performance.

Common Failure Modes and Fixes:

• Slow Model Loading/High Resource Usage:
  • Fix: Your GPU lacks sufficient VRAM. Try a smaller model or upgrade your GPU. Verify with nvidia-smi.
• Model Errors (e.g., out-of-memory):
  • Fix: Similar to slow loading, this indicates insufficient VRAM. Reduce model size or free up GPU resources.
• Ollama not using GPU:
  • Fix: Ensure NVIDIA drivers and CUDA are correctly installed and detected by Ollama. Check Ollama logs for GPU detection messages.