guide · #ai · 15 min

Run AI Models Locally with Ollama

Cech Tech Reviews

Run ChatGPT-level AI models on your own hardware. Complete privacy, no API costs, and works offline. Perfect for sensitive data or just saving money.

1. 1

   Install Ollama

   On Mac: brew install ollama
   On Linux: curl -fsSL https://ollama.com/install.sh | sh
   On Windows: Download from ollama.com

2. 2

   Download your first model

   Run: ollama pull llama3.1

   This downloads Meta's Llama 3.1 model (about 4GB). It runs on most modern computers with 8GB+ RAM.

3. 3

   Start chatting

   Run: ollama run llama3.1

   You are now chatting with an AI model running entirely on your machine. No internet required. No data leaves your computer.

4. 4

   Try other models

   ollama pull mistral (fast, great for coding)
   ollama pull gemma2 (Google's model, good all-rounder)
   ollama pull codellama (specialized for code)

   Each model has different strengths.

5. 5

   Connect a web interface

   For a ChatGPT-like interface, run Open WebUI:

   docker run -d -p 3000:8080 \
     -v ~/open-webui:/app/backend/data \
     --add-host=host.docker.internal:host-gateway \
     ghcr.io/open-webui/open-webui:main

   Open /:3000 for a polished chat experience.