My Complete Homelab AI Stack: 7 VMs, 3 GPUs, Zero Cloud

This Is Not Theory

Every guide on this site comes from doing it ourselves. This article is the overview — the full picture of what we run, how it’s connected, and what it costs. Nothing here is hypothetical. This is our actual infrastructure, running 24/7.

The Hardware

Bare Metal Host

• CPU: AMD Threadripper 1950X (16 cores, 32 threads)
• RAM: 64GB DDR4 (fully committed)
• GPU 1: RTX 3060 Ti 8GB — passthrough to AI VM
• GPU 2: RTX 3080 Ti 12GB — in a separate LLM server
• Storage: NVMe boot + ZFS pool on TrueNAS
• OS: Proxmox VE (virtualization host)

LLM Server (Separate Machine)

• CPU: AMD Ryzen 9 5950X (16 cores, 32 threads)
• GPU: RTX 3080 Ti 12GB
• RAM: 32GB
• Role: Local LLM inference (Ollama)

Network

• 1GbE throughout the house
• AdGuard Home for DNS filtering and local DNS
• All sensitive services behind firewall, no inbound ports exposed
• Cloudflare in front of anything public-facing

The VM Stack

Total RAM allocated: ~40GB of 64GB available. The rest is Proxmox overhead and ZFS cache.

The Software Stack

Virtualization

• Proxmox VE — bare metal hypervisor
• GPU passthrough — 3060 Ti passed through to AI compute VM
• VZDump backups — daily 4 AM snapshots to TrueNAS

Local LLMs

• Ollama — model serving, API, model management
• Models: GLM 4.7 Flash 30B (MoE, primary), Qwen variants, Llama variants
• Quantization: Q4_K_M for production, Q8 for quality-critical tasks
• Keep-alive: 20 minutes (models stay warm for follow-up requests)
• API: OpenAI-compatible on port 11434

Image Generation

• ComfyUI — node-based Stable Diffusion interface
• Models: RealVisXL, SDXL variants
• GPU: 3060 Ti 8GB with –fp8 flag for SDXL
• API: REST API on port 8188 for automation
• Idle unload: Systemd service that frees VRAM after 5 min inactivity

Voice Synthesis

• Kokoro TTS — natural voice generation
• Voices: Multiple voice models for different content types
• Post-processing: FFmpeg pipeline (pitch shift, speed adjust, normalization)
• API: REST API on port 8000

Automation

• n8n — workflow automation, connects everything together
• Pipelines: Content generation, video production, notifications
• Triggers: Schedule, webhook, manual

AI Agents

• OpenClaw — agent framework with memory, tools, and scheduling
• Multiple agents running 24/7 — each with distinct personalities and tasks
• Model routing: Agents use local Ollama models (free) for routine work
• Memory system: Persistent memory files for context across sessions

DNS & Networking

• AdGuard Home — DNS filtering, local DNS rewrites (*.lan domains)
• Nginx reverse proxy — routes internal traffic to services
• Cloudflare — public-facing services only (this website)

Monitoring

• Prometheus — metrics collection from all services
• Grafana — dashboards for GPU utilization, model performance, uptime
• Alerting — Telegram notifications for service failures

Storage & Backups

• TrueNAS — ZFS storage pool, NFS shares
• Daily backups — VZDump snapshots of all VMs, 5-day rotation
• Network share — /mnt/ragnar-data/ shared across VMs for workspace

How It All Connects

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

                    ┌─────────────┐
                    │  Proxmox VE │
                    │  (Bare Metal)│
                    └──────┬──────┘
                           │
        ┌──────────┬───────┼────────┬──────────┐
        │          │       │        │          │
   ┌────▼───┐ ┌───▼───┐ ┌─▼──┐ ┌───▼────┐ ┌───▼───┐
   │Gateway │ │Agents │ │NAS│ │Docker  │ │AI Comp│
   │ (6GB)  │ │ (4GB) │ │12G│ │ (4GB)  │ │ (2GB) │
   │OpenClaw│ │Agents │ │ZFS│ │Proxy   │ │Agent  │
   └────────┘ └───────┘ └────┘ │Monitor │ └───────┘
                                  └────────┘
                                        
              ┌──────────────┐         ┌──────────────┐
              │  AI Compute  │         │   LLM Host   │
              │   (8GB)      │         │   (4GB)      │
              │  3060 Ti     │         │  3080 Ti     │
              │  ComfyUI     │         │  Ollama      │
              │  Kokoro TTS  │         │  Models      │
              └──────────────┘         └──────────────┘

What It Costs

One-Time Hardware Costs

If buying everything new/used today: ~$2,000-2,500 for the full stack.

Monthly Costs

Compare to equivalent cloud spending:

• ChatGPT Plus: $20/mo
• Midjourney: $10/mo
• ElevenLabs: $5/mo
• VPS for agents: $10-20/mo
• Cloud total: $45-55/mo — and that’s with rate limits and privacy concerns

Our stack does all of that, unlimited, for $20/mo in electricity.

What This Stack Actually Does

24/7 AI Agents

Multiple AI agents running continuously — each with persistent memory, tool access, and scheduled tasks. They monitor infrastructure, generate content, manage automation, and respond to messages. All on local models.

Unlimited Image Generation

ComfyUI generates images on demand — via web UI or API. No per-image cost, no content filters, no rate limits. We train custom LoRAs for specific faces and styles.

Voice Synthesis

Kokoro TTS generates natural voice for videos, narration, and notifications. Unlimited usage, multiple voices, FFmpeg post-processing pipeline.

Automated Content Pipeline

n8n orchestrates the full YouTube pipeline: script generation (Ollama) → voice synthesis (Kokoro) → image generation (ComfyUI) → video assembly (FFmpeg) → upload. One trigger produces a finished video.

Infrastructure Monitoring

Prometheus + Grafana dashboards show GPU utilization, model load times, token throughput, and service health across all VMs. Alerts go to Telegram.

Security Posture

• No inbound ports on the home network
• Cloudflare proxy for all public services
• AdGuard DNS blocks ads, trackers, and malicious domains network-wide
• Separate VLANs for IoT, guests, and infrastructure
• SSH key auth on all VMs, password auth disabled where possible
• Daily backups with 5-day rotation
• ZFS checksums detect silent data corruption

Lessons Learned

1. GPU Passthrough Is Worth the Headache

Passing a GPU through Proxmox to a VM takes some setup, but it gives you full GPU acceleration in a virtualized environment. Worth every minute of config.

2. Model Keep-Alive Matters

Default 5-minute keep-alive means cold loads every time your automation fires. Set it to 20 minutes for production workflows.

3. Monitor Your Disk Space

VMs fill up fast — model files, generated images, audio files, logs. We had a 60GB VM hit 100% and lock up completely. Monitor disk usage and prune regularly.

4. Don’t Expose Your Homelab

Self-host your AI. Don’t expose your homelab. Use Cloudflare for anything public. This site is static HTML on Cloudflare Pages — not hosted on our infrastructure. That’s intentional.

5. Backups Are Not Optional

One bad config edit crashed a VM into a boot loop. Daily VZDump snapshots saved us. Test your restore process before you need it.

How to Start Building Your Own

You don’t need to build all of this at once. Start small:

1. Week 1: Run your first local model on your existing PC
2. Week 2: Add ComfyUI for image generation
3. Month 1: Buy a used GPU if you don’t have one — see our GPU guide
4. Month 2: Set up n8n automation
5. Month 3: Add Kokoro TTS for voice
6. Month 4+: Virtualize with Proxmox, add monitoring, scale up

This stack took us months to build — but each piece works standalone. You don’t need all of it to get value. Start with Ollama on your laptop and grow from there.

Next Steps

• Getting Started Guide — Begin from zero
• VRAM Calculator — Check what your hardware can run
• Best GPUs for Local LLMs — Choose your hardware

This is a living document. As our infrastructure evolves, we’ll update this article. Last updated: September 2026.