---
name: mac-mini-llm-lab
description: Configure a Mac mini as a reliable local LLM server with remote access,
  observability, and power-safe operation. Use when building an always-on private
  AI inference server on Apple Silicon.
category: devops
risk: critical
source: https://github.com/BagelHole/DevOps-Security-Agent-Skills
source_repo: BagelHole/DevOps-Security-Agent-Skills
source_type: community
date_added: '2026-09-20'
license: MIT
license_source: https://github.com/BagelHole/DevOps-Security-Agent-Skills/blob/main/LICENSE
compatibility: Requires the relevant OS/platform tooling and privileged access where
  noted. Docs-only; helper scripts and templates not bundled.
metadata:
  author: devops-skills
  version: '1.0'
---

# Mac mini LLM Lab

Turn a Mac mini into a low-noise, always-on local AI appliance.

## When to Use This Skill

Use this skill when:
- Setting up a dedicated local LLM inference server
- Building a private AI development environment
- Need always-on model serving without cloud costs
- Running models that fit the actual machine’s unified memory
- Creating a home lab AI server for a small team

## Prerequisites

- An actual Apple Silicon Mac mini configuration with sufficient available memory for the selected model
- macOS Sonoma 14+ or Sequoia 15+
- Ethernet connection (recommended over Wi-Fi)
- UPS for power protection (optional but recommended)

## Host and exposure boundaries

Check actual model hardware/memory and installed macOS/tool support first. The throughput/RAM table is illustrative, not measured evidence or a fit guarantee; include KV cache/context/concurrency overhead. Keep Ollama loopback-only by default; TLS alone in the Caddy example does not add authentication. Configure an authenticated private proxy/tunnel and access policy before remote exposure. Loopback binding must be deliberately adapted for Docker access, without exposing an unauthenticated LAN listener. Approve persistent services, downloads, updates, reboots and FileVault/power changes separately as applicable; preserve existing settings and recovery access. Create script/log directories before writing them and do not run privileged diagnostics unattended from cron.

## Initial System Setup

```bash
# Update macOS
softwareupdate --install --all

# Install Xcode command-line tools
xcode-select --install

# Install Homebrew
# Obtain the official Homebrew installer at a reviewed version, inspect it,
# and run the verified local file only after installation authorization.

# Core packages
brew install tmux htop btop wget jq git neovim

# Python environment (for MLX and custom scripts)
brew install python@3.12 uv

# Monitoring
brew install prometheus node_exporter
```

## Ollama Setup

```bash
# Install Ollama
brew install ollama

# Pull models based on your RAM
# 16GB Mac mini:
ollama pull llama3.1:8b
ollama pull nomic-embed-text
ollama pull codellama:7b

# 32GB Mac mini:
ollama pull llama3.1:8b
ollama pull qwen2.5:14b
ollama pull deepseek-coder-v2:16b
ollama pull nomic-embed-text

# 64GB+ Mac mini:
ollama pull llama3.1:70b
ollama pull qwen2.5:32b
ollama pull codellama:34b

# Verify Metal acceleration
ollama run llama3.1:8b --verbose
# Look for: "metal" in output
```

## MLX Framework (Apple Silicon Native)

MLX runs models natively on Apple Silicon with excellent performance:

```bash
# Install MLX
uv pip install mlx mlx-lm

# Run a model
python3 -c "
from mlx_lm import load, generate
model, tokenizer = load('mlx-community/Llama-3.1-8B-Instruct-4bit')
response = generate(model, tokenizer, prompt='Explain Docker in 3 sentences', max_tokens=200)
print(response)
"

# MLX server (OpenAI-compatible API)
uv pip install mlx-lm[server]
mlx_lm.server --model mlx-community/Llama-3.1-8B-Instruct-4bit --port 8080
```

## Auto-Start with launchd

```xml
<!-- ~/Library/LaunchAgents/com.ollama.serve.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.ollama.serve</string>
    <key>ProgramArguments</key>
    <array>
        <string>/opt/homebrew/bin/ollama</string>
        <string>serve</string>
    </array>
    <key>EnvironmentVariables</key>
    <dict>
        <key>OLLAMA_HOST</key>
        <string>127.0.0.1:11434</string>
        <key>OLLAMA_NUM_PARALLEL</key>
        <string>4</string>
        <key>OLLAMA_MAX_LOADED_MODELS</key>
        <string>2</string>
        <key>OLLAMA_FLASH_ATTENTION</key>
        <string>1</string>
    </dict>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
    <key>StandardOutPath</key>
    <string>/tmp/ollama.log</string>
    <key>StandardErrorPath</key>
    <string>/tmp/ollama.err</string>
</dict>
</plist>
```

```bash
# Load the service
launchctl load ~/Library/LaunchAgents/com.ollama.serve.plist

# Check status
launchctl list | grep ollama

# Unload if needed
launchctl unload ~/Library/LaunchAgents/com.ollama.serve.plist
```

## Power & Reliability

```bash
# Prevent idle sleep on this desktop (record existing values for rollback)
sudo pmset -a disablesleep 1
sudo pmset -a sleep 0

# Auto-restart after power failure
sudo pmset -a autorestart 1

# Schedule weekly reboot (Sunday 4 AM)
sudo pmset repeat shutdown U 03:55:00 poweron U 04:00:00

# Check power settings
pmset -g
```

## Remote Access

### Tailscale (Recommended)

```bash
# Install Tailscale for easy secure remote access
brew install --cask tailscale

# Enable from menu bar, authenticate
# Access your Mac mini from anywhere: http://mac-mini:11434
```

### SSH Hardening

```bash
# Enable remote login
sudo systemsetup -setremotelogin on

# Edit SSH config
sudo nano /etc/ssh/sshd_config
# Add:
# PasswordAuthentication no
# PubkeyAuthentication yes
# PermitRootLogin no
# AllowUsers yourusername

# Restart SSH
sudo launchctl unload /System/Library/LaunchDaemons/ssh.plist
sudo launchctl load /System/Library/LaunchDaemons/ssh.plist
```

### Reverse Proxy with Caddy

```bash
brew install caddy

# Caddyfile
cat > /opt/homebrew/etc/Caddyfile << 'EOF'
llm.local:443 {
    tls internal
    reverse_proxy localhost:11434

    @api path /v1/*
    handle @api {
        reverse_proxy localhost:11434
    }
}

webui.local:443 {
    tls internal
    reverse_proxy localhost:3000
}
EOF

brew services start caddy
```

## Open WebUI Setup

```bash
# Run Open WebUI via Docker
docker run -d \
  --name open-webui \
  -p 3000:8080 \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  -e WEBUI_AUTH=true \
  -v open-webui:/app/backend/data \
  --restart unless-stopped \
  ghcr.io/open-webui/open-webui:main

# Or install Docker first if not available
brew install --cask docker
```

## Monitoring

```bash
# Health check script
cat > ~/scripts/llm-health.sh << 'SCRIPT'
#!/bin/bash
# Check Ollama
if curl -sf http://localhost:11434/api/tags > /dev/null; then
    echo "$(date): Ollama OK"
    curl -s http://localhost:11434/api/ps | python3 -m json.tool
else
    echo "$(date): Ollama DOWN"
    # Restart
    launchctl kickstart -k gui/$(id -u)/com.ollama.serve
fi

# System stats
echo "CPU: $(top -l 1 -n 0 | grep 'CPU usage')"
echo "Memory: $(vm_stat | head -5)"
echo "Disk: $(df -h / | tail -1)"
echo "Thermal: $(sudo powermetrics --samplers smc -n 1 2>/dev/null | grep 'CPU die' || echo 'N/A')"
SCRIPT
chmod +x ~/scripts/llm-health.sh

# Schedule health check every 5 minutes
# Add to crontab: crontab -e
# */5 * * * * ~/scripts/llm-health.sh >> ~/logs/llm-health.log 2>&1
```

### Memory Usage by Model

| Model | RAM Required | Tokens/sec (M2) | Tokens/sec (M4) |
|-------|-------------|-----------------|-----------------|
| llama3.1:8b (Q4) | ~5 GB | ~25 t/s | ~45 t/s |
| qwen2.5:14b (Q4) | ~9 GB | ~15 t/s | ~30 t/s |
| llama3.1:70b (Q4) | ~40 GB | ~5 t/s | ~10 t/s |
| nomic-embed-text | ~300 MB | N/A | N/A |
| codellama:13b | ~8 GB | ~18 t/s | ~35 t/s |

## Security Checklist

```bash
# Enable FileVault disk encryption
sudo fdesetup enable

# Enable firewall
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --setglobalstate on
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --setstealthmode on

# Disable unnecessary sharing services
sudo launchctl disable system/com.apple.screensharing
sudo launchctl disable system/com.apple.AirPlayXPCHelper

# Set strong admin password
# System Settings > Users & Groups

# Restrict Ollama to local network only (if not using Tailscale)
# Set OLLAMA_HOST=127.0.0.1 in launchd plist
```

## Performance Tuning

```bash
# Increase file descriptor limits for concurrent requests
sudo launchctl limit maxfiles 65536 200000

# Check unified memory pressure
memory_pressure

# Monitor GPU usage (Metal)
sudo powermetrics --samplers gpu_power -n 1

# Optimize for inference (disable Spotlight indexing on model dirs)
mdutil -i off ~/.ollama
```

## Troubleshooting

| Issue | Solution |
|-------|---------|
| Model loading slow | First load caches to memory; subsequent loads are fast |
| Out of memory | Use smaller quantization (Q4_K_M), reduce `OLLAMA_MAX_LOADED_MODELS` |
| Mac sleeping | Run `sudo pmset -a disablesleep 1` |
| Ollama not starting | Check `launchctl list | grep ollama`, view `/tmp/ollama.err` |
| Slow over Wi-Fi | Use Ethernet; Wi-Fi adds latency to streaming responses |
| Thermal throttling | Ensure adequate ventilation, check `powermetrics` |

## Related Skills

- ollama-stack (`ollama-stack`) — Software stack with Docker Compose and LiteLLM
- ssh-configuration (`ssh-configuration`) — Secure remote access
- vpn-setup (`vpn-setup`) — Remote access via WireGuard/Tailscale

## Limitations

- Infrastructure commands can disrupt services: confirm target host/scope and have backups/snapshots before mutating state.
- Docs-only import: upstream scripts and templates not bundled.



---
Source attribution: BagelHole/DevOps-Security-Agent-Skills contributors, via Agentic Awesome Skills
Original project: https://github.com/BagelHole/DevOps-Security-Agent-Skills
License: MIT + CC-BY-4.0
License information: https://creativecommons.org/licenses/by/4.0/
Original upstream instructions; downloaded from Mega AI Hub.

## Upstream license notice

MIT License

Copyright (c) 2026 Toby Miller

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.



# Mega AI Hub resource metadata

Creator / publisher: BagelHole/DevOps-Security-Agent-Skills contributors, via Agentic Awesome Skills
Original project: https://github.com/BagelHole/DevOps-Security-Agent-Skills
Attribution basis: Recorded upstream repository

Fields of work: Development & Engineering
Roles: ML / AI Engineer
Specialties: Specialty not specified
Tools: Tool not specified
Country applicability: Country not specified
Classification basis: Dave AI task classification from package title, description, and declared metadata
Instruction SHA-256: cb0145b8c90e690c6a0e46422802d09e3f3ac0bd5ce27cfa1848f76f8a32ee44
Metadata reviewed: 2026-10-27
This metadata supersedes earlier Mega AI Hub catalog attribution. Original instruction and supporting files are preserved.