Ollama server

Feb 5, 2025 llm linux

Overview

Setting up an old school Self-Hosted Server Stack: Mattermost, OpenWebUI, and Monitoring

This is how Claude and I installed an Ollama server from scratch.

First, we installed Ollama, which is the backend for our AI operations:

1curl https://ollama.ai/install.sh | sh
bash

After installation, we created a systemd service to manage Ollama:

1sudo vim /etc/systemd/system/ollama.service
bash

Added this configuration:

 1[Unit]
 2Description=Ollama Service
 3After=network-online.target
 4
 5[Service]
 6ExecStart=/usr/local/bin/ollama serve
 7User=root
 8Restart=always
 9
10[Install]
11WantedBy=multi-user.target
ini

Started and enabled the service:

1sudo systemctl daemon-reload
2sudo systemctl enable ollama
3sudo systemctl start ollama
bash

To verify the installation, we pulled a model:

1ollama pull phi3
bash

OpenWebUI serves as our frontend interface for Ollama. We installed it using Python's virtual environment:

 1# Install required packages
 2sudo apt install python3-full python3-pip python3-venv
 3
 4# Create and activate virtual environment
 5mkdir open-webui && cd open-webui
 6python3 -m venv openwebui-env
 7source openwebui-env/bin/activate
 8
 9# Install OpenWebUI
10pip install open-webui
bash

Created a systemd service for OpenWebUI:

1sudo vim /etc/systemd/system/openwebui.service
bash

Added the configuration (change 'username' to your username):

 1[Unit]
 2Description=OpenWebUI
 3After=network.target
 4
 5[Service]
 6Type=simple
 7User=username
 8WorkingDirectory=/home/username
 9Environment=PATH=/home/username/open-webui/openwebui-env/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
10ExecStart=/home/username/open-webui/openwebui-env/bin/open-webui serve
11Restart=always
12
13[Install]
14WantedBy=multi-user.target
ini

Then start it.

1sudo systemctl daemon-reload
2sudo systemctl enable openwebui
3sudo systemctl start openwebui
bash

We started by installing Mattermost on our Ubuntu server. Mattermost runs on port 8065 by default.

Initially, we encountered the error: "Please check connection, Mattermost unreachable. If issue persists, ask administrator to check WebSocket port." This is a common issue when setting up Mattermost, especially before configuring Nginx properly.

The solution involved:

  1. Checking if port 8065 was accessible:
1sudo netstat -tulpn | grep 8065
bash
  1. Ensuring the WebSocket configuration in config.json matched our setup:
1{
2    "ServiceSettings": {
3        "SiteURL": "http://your-server-ip:8065",
4        "WebsocketURL": "",  // Empty to use SiteURL
5        "TLSStrictTransport": false
6    }
7}
json
  1. Most importantly, properly configuring Nginx to handle WebSocket connections. The critical part in the Nginx configuration is:
1location ~ /api/v[0-9]+/(users/)?websocket$ {
2    proxy_set_header Upgrade $http_upgrade;
3    proxy_set_header Connection "upgrade";
4    proxy_set_header Host $http_host;
5    # ... other proxy settings ...
6}
nginx

To secure our Mattermost instance, we set up Nginx as a reverse proxy and added SSL certificates:

  1. Created Nginx configuration:
1sudo vim /etc/nginx/sites-available/mattermost
bash

Added:

 1upstream backend {
 2    server 127.0.0.1:8065;
 3    keepalive 32;
 4}
 5
 6proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=mattermost_cache:10m max_size=3g inactive=120m use_temp_path=off;
 7
 8server {
 9    server_name mattermost.yourdomain.com;
10
11    location ~ /api/v[0-9]+/(users/)?websocket$ {
12        proxy_set_header Upgrade $http_upgrade;
13        proxy_set_header Connection "upgrade";
14        client_max_body_size 50M;
15        proxy_set_header Host $http_host;
16        proxy_set_header X-Real-IP $remote_addr;
17        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
18        proxy_set_header X-Forwarded-Proto $scheme;
19        proxy_set_header X-Frame-Options SAMEORIGIN;
20        proxy_buffers 256 16k;
21        proxy_buffer_size 16k;
22        proxy_read_timeout 600s;
23        proxy_pass http://backend;
24    }
25
26    location / {
27        client_max_body_size 50M;
28        proxy_set_header Connection "";
29        proxy_set_header Host $http_host;
30        proxy_set_header X-Real-IP $remote_addr;
31        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
32        proxy_set_header X-Forwarded-Proto $scheme;
33        proxy_set_header X-Frame-Options SAMEORIGIN;
34        proxy_buffers 256 16k;
35        proxy_buffer_size 16k;
36        proxy_read_timeout 600s;
37        proxy_cache mattermost_cache;
38        proxy_cache_use_stale error timeout invalid_header http_500;
39        proxy_cache_revalidate on;
40        proxy_cache_min_uses 2;
41        proxy_cache_lock on;
42        proxy_cache_valid 200 7d;
43        proxy_pass http://backend;
44    }
45}
nginx

We implemented several security measures:

  1. Disabled open registration by modifying config.json:
1sudo vim /opt/mattermost/config/config.json
bash

Added:

1{
2    "ServiceSettings": {
3        "EnableOpenServer": false
4    },
5    "TeamSettings": {
6        "EnableUserCreation": false,
7        "EnableTeamCreation": false
8    }
9}
json
  1. Enabled Multi-Factor Authentication:
1{
2    "ServiceSettings": {
3        "EnforceMultifactorAuthentication": true,
4        "EnableMultifactorAuthentication": true
5    }
6}
json
  1. Set up email notifications using SendGrid:
 1{
 2    "EmailSettings": {
 3        "EnableSignUpWithEmail": true,
 4        "EnableSignInWithEmail": true,
 5        "EnableSMTPAuth": true,
 6        "SMTPUsername": "apikey",
 7        "SMTPPassword": "your-sendgrid-api-key-here",
 8        "SMTPServer": "smtp.sendgrid.net",
 9        "SMTPPort": 587,
10        "ConnectionSecurity": "STARTTLS",
11        "SendEmailNotifications": true,
12        "FeedbackName": "Mattermost",
13        "FeedbackEmail": "your-verified-sender@yourdomain.com",
14        "ReplyToAddress": "your-verified-sender@yourdomain.com"
15    }
16}
json
  1. Created necessary users and directories:
1sudo useradd --no-create-home --shell /bin/false prometheus
2sudo mkdir /etc/prometheus
3sudo mkdir /var/lib/prometheus
bash
  1. Downloaded and installed Prometheus:
1wget https://github.com/prometheus/prometheus/releases/download/v2.49.1/prometheus-2.49.1.linux-amd64.tar.gz
2tar xvf prometheus-*.tar.gz
3cd prometheus-*/
4sudo cp prometheus /usr/local/bin/
5sudo cp promtool /usr/local/bin/
6sudo cp -r consoles/ /etc/prometheus
7sudo cp -r console_libraries/ /etc/prometheus
bash
  1. Created Prometheus configuration:
1sudo vim /etc/prometheus/prometheus.yml
bash

Added basic configuration:

 1global:
 2  scrape_interval: 15s
 3
 4scrape_configs:
 5  - job_name: 'prometheus'
 6    static_configs:
 7      - targets: ['localhost:9090']
 8
 9  - job_name: 'node'
10    static_configs:
11      - targets: ['localhost:9100']
yaml
  1. Set up Node Exporter for system metrics:
1wget https://github.com/prometheus/node_exporter/releases/download/v1.7.0/node_exporter-1.7.0.linux-amd64.tar.gz
2tar xvf node_exporter-*.tar.gz
3sudo cp node_exporter-*/node_exporter /usr/local/bin/
4sudo useradd --no-create-home --shell /bin/false node_exporter
5sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter
bash
  1. Created systemd service for Node Exporter:
1sudo vim /etc/systemd/system/node_exporter.service
bash

Added:

 1[Unit]
 2Description=Node Exporter
 3Wants=network-online.target
 4After=network-online.target
 5
 6[Service]
 7User=node_exporter
 8Group=node_exporter
 9Type=simple
10ExecStart=/usr/local/bin/node_exporter
11
12[Install]
13WantedBy=multi-user.target
ini
  1. Added Grafana repository and installed:
1sudo apt-get install -y apt-transport-https software-properties-common
2wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
3echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/sources.list.d/grafana.list
4sudo apt-get update
5sudo apt-get install grafana
bash
  1. Set up Nginx as reverse proxy for Grafana:
1sudo vim /etc/nginx/sites-available/grafana
bash

Added:

 1server {
 2    server_name grafana.yourdomain.com;
 3
 4    location / {
 5        proxy_pass http://localhost:3000;
 6        proxy_http_version 1.1;
 7        proxy_set_header Upgrade $http_upgrade;
 8        proxy_set_header Connection 'upgrade';
 9        proxy_set_header Host $host;
10        proxy_cache_bypass $http_upgrade;
11    }
12}
nginx

If you're using bash/zsh, you might want to add these helpful aliases to your .bashrc or .zshrc:

 1# Service management
 2alias mm-restart='sudo systemctl restart mattermost'
 3alias mm-status='sudo systemctl status mattermost'
 4alias mm-logs='sudo journalctl -u mattermost -f'
 5
 6# Quick edits
 7alias mm-config='sudo vim /opt/mattermost/config/config.json'
 8alias ng-edit='cd /etc/nginx/sites-available'
 9
10# Monitoring
11alias prom-status='sudo systemctl status prometheus'
12alias graf-status='sudo systemctl status grafana-server'
bash

We now have a fully functional, secure server stack with:

  • Mattermost for team communication
  • OpenWebUI for AI interactions
  • Comprehensive system monitoring with Prometheus and Grafana
  • Everything secured with SSL certificates
  • Basic monitoring dashboards set up

Next steps could include:

  • Setting up specific Mattermost monitoring
  • Adding Ollama monitoring
  • Creating custom Grafana dashboards
  • Setting up alerting rules
  • Implementing backup solutions
  • Monitor token per second

Remember to regularly update all components and monitor system resources to ensure everything runs smoothly. Setting up a Self-Hosted Server Stack: Mattermost, OpenWebUI, and Monitoring