The Hardware That Started It All
Last week I wrote about why I built a homelab instead of paying for cloud services. The tl;dr: a broken Synology, astronomical NAS prices, and a refurbished HP G3 that cost a quarter of a 2-bay Synology enclosure — before drives.
What I didn’t cover: how that single node became two nodes, why the RAM split is lopsided, and why the architecture divides into "infra" (Proxmox + ZFS) and "smart" (bare-metal Debian + GPU) instead of just cloning the same setup twice.
The Two Nodes (Current State)
| Infra Node | Smart Node | |
|---|---|---|
| Hardware | HP EliteDesk 800 G3 (refurbished) | Custom build on ASUS ROG STRIX B450-F |
| CPU | Intel i5-6500 (4C/4T, 3.2 GHz) | AMD Ryzen 5 5600 (6C/12T, 3.5/4.4 GHz) |
| RAM | 16 GB DDR4 | 32 GB DDR4 |
| GPU | None (iGPU only) | GTX 1060 6 GB OC |
| Storage | 1 TB SSD (OS) + 3×2 TB HDD in RAID-Z (/tank) |
1 TB NVMe (OS only) |
| Network | 1 GbE → upgraded to 2.5 GbE | 1 GbE → upgraded to 2.5 GbE |
| OS | Proxmox VE 8.2 (host) | Debian 12 (bare metal) |
| Role | ZFS, NAS, infra services, Home Assistant, Nextcloud | Local AI, media stack, GPU transcoding |
Why the RAM split? When I built the Smart Node, I didn’t have enough DDR4 sticks for 32 GB on both. The i5-6500 runs Proxmox + ZFS + a few VMs comfortably in 16 GB. The Ryzen 5600 needed 32 GB for local LLMs (7B–13B models) + Docker overhead. Pragmatism over symmetry.
Why Not Cluster Them?
Everyone asks: "Why not Proxmox cluster the two nodes?"
Short answer: Different OSes, different roles, no HA requirement.
- Infra Node = Proxmox host (ZFS, VMs)
- Smart Node = Bare-metal Debian (Docker, Ollama, media stack)
Clustering would mean installing Proxmox on the Smart Node — adding a hypervisor layer I don’t need for a single-purpose compute box. No shared storage (Smart Node has only NVMe for OS). No HA need (if Smart Node dies, media transcoding pauses; infra keeps running).
Lesson: Cluster when you need HA and shared state. Don’t cluster just because you have two boxes.
The Split Logic: Infra vs. Smart
Infra Node — Proxmox Host (ZFS + VMs)
Runs on Proxmox host:
- ZFS pool
/tank(3×2 TB RAID-Z = 4 TB usable) — single source of truth for all data - Samba VM (Debian 12 + Webmin) — NAS shares for the house
- Home Assistant VM (Debian 12) — Zigbee/Z-Wave USB passthrough
- Docker VM (Debian 12) — Nextcloud + Collabora, AdGuard, Docmost, Immich, n8n, Twingate connector
- Cloudflare Tunnel (on host or Docker VM) — public ingress for selected services
- Twingate connector — zero-trust access to entire infra network
Why Proxmox here?
- ZFS on the host — direct disk access, no virtualization overhead
- VM isolation for Home Assistant (radio passthrough), Samba (Webmin UI), Docker (separate failure domain)
- Built-in VM-level backup, snapshots, replication
Smart Node — Bare-Metal Debian (Docker + GPU + AI)
Runs on bare metal Debian 12:
- Docker + Portainer — media stack (*Arr, qBittorrent, Plex), Ollama, local LLMs
- GPU transcoding — GTX 1060 NVENC/NVDEC for Plex (Plex Pass required)
- Local AI inference — 32 GB RAM fits 7B–13B quantized models
- Mounts
/tank/mediavia NFS from Infra Node — all media library lives on ZFS
Why bare metal Debian?
- Single-purpose compute node — no hypervisor tax, no 1–2 GB RAM overhead
- GPU passthrough to Docker is simpler (
nvidia-container-toolkit+ runtime) than Proxmox PCI passthrough → VM → Docker - Portainer on Debian is lighter than Proxmox for this workload
The Media Stack Migration (And Why It Mattered)
Originally: Everything on Infra Node (Docker VM).
Infra Node (Proxmox) → Docker VM → Plex + *Arr + qBittorrent
Problem: i5-6500 + no GPU = CPU transcoding = slow, poor quality, fans screaming. The i5 did handle it for about two weeks before I moved the stack.
Solution: Move media stack to Smart Node.
Infra Node: ZFS (/tank/media) → NFS export → Smart Node: Plex + *Arr + GPU transcoding
Result:
- Plex Direct Play on LAN → zero transcoding
- Remote stream → GTX 1060 NVENC → smooth 1080p/4K transcoding
- i5-6500 sits at 5% CPU, fans silent
Why Plex, not Jellyfin? Compared both. Plex won on: TV app availability (pre-installed on most TVs, some don’t allow sideloading), family sharing UX, Plex Pass GPU transcoding maturity. Jellyfin is excellent — just not the right fit for my household.
The Network Upgrade That Wasn’t Planned
Right after I created the ZFS pool and started migrating TB of documents, photos, and media to /tank, the network choked. Sustained 1 GbE transfers would freeze the router — internet died, had to unplug/replug the router to recover.
Root cause: Cheap consumer switches/routers have low forwarding rate (packets per second) and switching capacity. Large sequential transfers saturate the switch fabric, not the port speed.
Fix: Upgraded the home network to 2.5 GbE — 2.5 GbE switch, 2.5 GbE NICs on both nodes and my main PC, Cat6a cabling. The router already supported 2GbE, but the switch didn’t so it got replaced too.
Lesson: 1 GbE port speed ≠ 1 GbE sustained throughput. Forwarding rate matters more than marketing numbers.
The GPU Reality Check
GTX 1060 6 GB in 2026:
| Model | Fits? | Tokens/sec (est.) | Notes |
|---|---|---|---|
| Qwen 2.5 3B (Q4) | ✅ | ~45 | Quick tasks |
| Qwen 3.5 4B (Q4) | ✅ | ~35 | Good balance |
| Gemma 4:latest (8B) | ✅ | ~20 | Solid reasoning |
| Qwen 2.5 7B (Q4) | ✅ | ~25 | Daily driver coding |
| Qwen 2.5 14B (Q4) | ❌ | — | Needs 8 GB+ VRAM |
My daily drivers: qwen2.5:7b (coding), gemma4:latest (reasoning), qwen2.5:3b / qwen3.5:4b (quick tasks).
For truly large contexts: nemotron-3-ultra via build.nvidia.com — not local, but saves the day when 32K context isn’t enough.
The 6 GB wall is real. Smart Node is for always-available local AI — not the biggest models.
The TrueNAS SCALE Dilemma (Still Weighing)
Looking back, I wonder: Why not TrueNAS SCALE on the i5-6500 + Proxmox on the Ryzen?
| Option | Storage OS | Compute OS |
|---|---|---|
| What I did | Proxmox (ZFS on host) | Debian bare metal |
| Alternative | TrueNAS SCALE (bare metal) | Proxmox VE |
TrueNAS SCALE on i5-6500 would give:
- ZFS management UI that doesn’t hate you (scrub alerts, expansion, SMART, SLOG/L2ARC config)
- Native SMB/NFS with ACL support — proper user management, per-share permissions
- Built-in Docker (Apps) + VMs
- Single OS to patch for storage
But TrueNAS SCALE’s VM management is not Proxmox:
- No LXC (only KVM VMs)
- GPU passthrough works but feels bolted on
- No integrated backup, no HA, no clustering
Proxmox on Ryzen would give:
- Better virtualization for the compute node
- But then ZFS lives on TrueNAS → need NFS/SMB to Proxmox → extra hop
The core tension: Storage-first OS (TrueNAS) vs Virtualization-first OS (Proxmox). I chose virtualization-first because my workload mix (HA + Samba + Docker + VMs) needs Proxmox’s VM flexibility more than TrueNAS’s storage UI.
Where I land today:
| Area | Verdict | Reason |
|---|---|---|
| Docker/Apps | Stay | Portainer features replaceable via CLI/monitoring containers |
| SMB/NFS/ACLs | Fix in place | Willing to fix Debian config properly instead of migrating |
| ZFS UI | Minor gain | "Almost no time lately" — TrueNAS helps initial config only |
| HA | Move to Smart Node | No downside |
| GPU/Smart Node | Stay bare metal | Backup not the argument; NFS/SMB control is |
| Migration effort | Unknown | Weekend or month? Need to estimate |
The core question: Is the Samba/NFS/ACL pain on Debian worth fixing in place, or does it justify a full storage OS migration?
I’m leaning toward fixing in place — investing a weekend in proper Samba config (users, shares, ACLs, systemd automounts for NFS), maybe Ansible-izing it. But the TrueNAS SCALE dilemma is real enough that I keep revisiting it.
For now: Architecture 1 stays. The dilemma is documented. If the Samba config fight gets old enough, I’ll revisit.
What I Would Change (If Starting Over)
| Decision | Current | Would Change To | Why |
|---|---|---|---|
| Infra Node OS | Proxmox + ZFS on host | TrueNAS SCALE (bare metal) | ZFS-first UI, native SMB/NFS, still runs Docker/VMs |
| Smart Node OS | Debian + Docker | Keep Debian bare metal | No hypervisor tax, simpler GPU passthrough |
| Media transport | NFS (LAN) | Keep NFS | Simple, fast, no VPN overhead |
| GPU | GTX 1060 6 GB | Same GTX 1060 | It’s what I had — would still have it |
| Network | 1 GbE → 2.5 GbE | Start at 2.5 GbE | Avoid the router reboot dance |
| Clustering | None | Keep none | Different roles, no HA need |
The Honest Summary
Two nodes. One location. Zero clustering. One NFS mount. One Twingate connector. It works.
| What Works | What’s Fragile |
|---|---|
| Media playback (Direct Play + GPU transcode) | NFS mount dependency — if Infra Node reboots, media pauses |
| Local AI inference (7B–9B models) | 6 GB VRAM ceiling — no 14B+ models locally |
| Remote access (Twingate → Plex) | Internet outage = no remote access |
| ZFS snapshots + send/receive for backups | Manual — no automated off-site yet |
| Home Assistant local control | Zigbee radio only on Infra Node |
Next week: "Why Proxmox Won (Over TrueNAS, Unraid, and Bare Metal Docker)" — the evaluation matrix, the ZFS-on-ZFS trap I almost fell into, why mp0 bind mounts don’t work for QEMU VMs, and why VirtIO-FS is the answer for VM storage.
Currently running: Infra Node (Proxmox 8.2, ZFS 2.2, 6 TB RAID-Z, 16 GB RAM) + Smart Node (Debian 12, Docker, GTX 1060, 32 GB RAM, Ollama, Hermes Agent). Zero cloud subscriptions.