GPU Configuration in Ollama

From the Ollama cheat sheet · GPU & Performance · verified Jul 2026

GPU Configuration

Configure GPU layers, parallel requests, and memory management.

bash

# Check which models are loaded and GPU usage
ollama ps

# Environment variables for GPU/performance
export OLLAMA_NUM_PARALLEL=4         # Concurrent requests
export OLLAMA_MAX_LOADED_MODELS=2    # Models in memory at once
export OLLAMA_FLASH_ATTENTION=1      # Enable flash attention
export OLLAMA_GPU_OVERHEAD=0         # Reserved GPU memory (bytes)

💡 Ollama automatically detects and uses available GPUs (NVIDIA, AMD, Apple Silicon)

⚡ Use OLLAMA_FLASH_ATTENTION=1 for faster inference on supported hardware

📌 Send keep_alive: 0 to immediately unload a model and free GPU memory

🟢 Preload models by sending an empty prompt — great for reducing first-response latency

gpuperformancememory

Back to the full Ollama cheat sheet