From f1c197422d2cf3b061228200bc42f1e44cef5297 Mon Sep 17 00:00:00 2001 From: Snarf Date: Thu, 5 Nov 2026 11:20:57 -0500 Subject: [PATCH] docs: add DNS setup plan documentation --- docs/DNS_Setup_Plan.md | 185 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 185 insertions(+) create mode 100644 docs/DNS_Setup_Plan.md diff --git a/docs/DNS_Setup_Plan.md b/docs/DNS_Setup_Plan.md new file mode 100644 index 0000000..2e2600e --- /dev/null +++ b/docs/DNS_Setup_Plan.md @@ -0,0 +1,185 @@ +# Project Blueprint: High-Availability DNS Stack, Network Honeypot & VLAN Security Architecture + +## 1. Project Objectives & Current State Assessment + +### 1.1 Objectives +1. **Enterprise-Grade DNS Stack:** + - Deploy a resilient, scalable, dual-cluster DNS architecture across **Aerodrome** (`local`) and **Spaceport** (`c-m-4fgkl75b`). + - Implement **Pi-hole** as the front-end ad/malware filtering and caching engine. + - Implement **Technitium DNS Server** as the authoritative local DNS resolver (`snarfnet.net` split-horizon and PTR zones) and recursive DNS-over-HTTPS (DoH) upstream. + - Ensure Active-Active / High Availability across both clusters with zero rate-limiting bottlenecks. +2. **Network Security & Honeypot:** + - Deploy an automated, containerized honeypot (**OpenCanary** or **Cowrie**) with centralized logging and instant alerts to Gotify / Zabbix. +3. **VLAN Segmentation & Firewall Policy Hardening:** + - Audit and enforce inter-VLAN firewall rules on the **UniFi Dream Machine Pro (UDM Pro)** (`140.44.4.1`). + - Isolate Smart Home / IoT, Storage, and Guest traffic from the Management/Infrastructure plane. + - Apply Kubernetes `NetworkPolicy` resources to restrict east-west namespace traffic. + +--- + +### 1.2 Inspection of Existing Cluster "Stubs" + +We probed both clusters and discovered why the existing DNS implementations stalled: + +| Cluster | Component | Current State | Root Cause Failure | Recommendation | +|---|---|:---:|---|---| +| **Aerodrome** (`140.44.4.11`) | `pihole` (deploy) | `0/1 Pending` | PVC `pihole` does not exist (`FailedScheduling`) | **Delete and rebuild cleanly** | +| **Aerodrome** (`140.44.4.11`) | `technitium-dns` (deploy) | `0/1 Pending` | PVC `technitium-longhorn-pvc` does not exist | **Delete and rebuild cleanly** | +| **Aerodrome** (`140.44.4.11`) | `pihole-dns-tcp/udp` | NodePort | Mapped to ephemeral high ports (`32655`/`31195`); standard DNS clients cannot query NodePorts | **Replace with Port 53 LoadBalancer VIP** | +| **Spaceport** (`140.44.4.41`) | `pihole` (deploy) | `1/1 Running` | Uses unmanaged local storage, exposed via NodePort only | **Delete and replace with unified manifest** | +| **Spaceport** (`140.44.4.41`) | `technitium-dns` (deploy) | `0/1 Pending` | PVC `technitium-longhorn-pvc` missing | **Delete and rebuild cleanly** | + +> [!IMPORTANT] +> **Decision: Clean Teardown & Declarative Deployment.** Rather than patching fractured, incomplete stubs, we will prune the orphaned resources in `infra` and `infrastructure` and deploy a unified, declarative GitOps stack with dedicated Longhorn storage volumes and MetalLB VIPs on standard port 53. + +--- + +## 2. High-Availability DNS Architecture (Pi-hole + Technitium) + +```mermaid +flowchart TD + subgraph Clients["LAN Clients across all VLANs (VLAN 20, 40, 50, Guest)"] + ClientDevices["Laptops, Phones, Servers, IoT"] + end + + subgraph UDM["UniFi Dream Machine Pro (140.44.4.1)"] + DHCP["DHCP Options 6:
Primary DNS: 140.44.4.35 (Aerodrome VIP)
Secondary DNS: 140.44.4.65 (Spaceport VIP)"] + end + + subgraph AeroCluster["Aerodrome K3s Cluster"] + AeroVIP["MetalLB VIP: 140.44.4.35:53 (UDP/TCP)"] + AeroPihole["Pi-hole Instance 1
(Ad-blocking, Caching, FTL rate-limit: 0)"] + AeroTech["Technitium Instance 1 (Primary Authoritative)
(snarfnet.net Zone, PTR Records, DNSSEC)"] + AeroStorage["Longhorn Persistent Volume"] + end + + subgraph SpaceCluster["Spaceport K3s Cluster"] + SpaceVIP["MetalLB VIP: 140.44.4.65:53 (UDP/TCP)"] + SpacePihole["Pi-hole Instance 2
(Ad-blocking, Caching, FTL rate-limit: 0)"] + SpaceTech["Technitium Instance 2 (Secondary Authoritative)
(Zone Transfer AXFR/IXFR from Primary)"] + SpaceStorage["Longhorn Persistent Volume"] + end + + subgraph Upstream["Encrypted Upstream Resolvers"] + DoH["DNS-over-HTTPS (Quad9 / Cloudflare / Mullvad)"] + end + + ClientDevices -->|Queries 140.44.4.35| AeroVIP + ClientDevices -.->|Fallback 140.44.4.65| SpaceVIP + AeroVIP --> AeroPihole + SpaceVIP --> SpacePihole + AeroPihole -->|Recursive / Local Lookup| AeroTech + SpacePihole -->|Recursive / Local Lookup| SpaceTech + AeroTech <==>|Automatic Zone Replication (AXFR)| SpaceTech + AeroTech -->|Encrypted Upstream| DoH + SpaceTech -->|Encrypted Upstream| DoH + AeroPihole --- AeroStorage + AeroTech --- AeroStorage + SpacePihole --- SpaceStorage + SpaceTech --- SpaceStorage +``` + +### 2.1 Solving the Historical Rate-Limiting & Performance Bottleneck +Your earlier Raspberry Pi Unbound/Pi-hole suffered when local devices (especially smart home polling, K8s CoreDNS forwards, and mobile apps) sent bursts of queries, tripping Pi-hole FTL's default rate-limit (`1000 queries in 60s per client`) and saturating single-threaded Unbound. + +**How our cluster architecture resolves this:** +1. **Configurable Rate Limiting:** + - Set `FTLCONF_RATE_LIMIT=0` (or `10000/60` for WAN only) in the Pi-hole configuration so internal bursts are never blocked. +2. **Technitium Engine vs. Unbound:** + - Technitium is built on .NET 8/9 with an asynchronous, high-throughput DNS engine capable of handling tens of thousands of QPS without degrading. + - Built-in caching with negative caching and aggressive prefetching ensures queries are answered from RAM in `<0.5ms`. +3. **Clustered Failover:** + - MetalLB VIP on Aerodrome (`140.44.4.35`) and Spaceport (`140.44.4.65`) guarantees high availability: if an entire Kubernetes node or cluster is undergoing maintenance, clients seamlessly failover to the secondary VIP without DNS interruption. +4. **Zone Synchronization:** + - Aerodrome acts as the primary Technitium DNS server. + - Spaceport acts as the secondary Technitium DNS server, replicating zones via standard DNS NOTIFY / AXFR transfers automatically. + +--- + +## 3. Network Security & Honeypot Architecture + +```mermaid +flowchart LR + subgraph VLANs["Network VLAN Segments (UDM Pro)"] + VLAN20["VLAN 20: Infra / Management
(140.44.4.0/24)"] + VLAN40["VLAN 40: Storage
(14.10.10.0/24)"] + VLAN50["VLAN 50: Smart Home / IoT
(14.4.4.0/24)"] + VLAN90["VLAN 90: Isolated DMZ / Honeypot
(14.99.99.0/24)"] + VLAN80["VLAN 80: Guest Network
(14.80.80.0/24)"] + end + + subgraph UDM_FW["UDM Pro Firewall Rules"] + DropInterVLAN["Block Inter-VLAN Routing"] + AllowEstablished["Allow Established & Related"] + AllowDNS["Allow Port 53 to DNS VIPs"] + end + + subgraph SecurityCluster["Security Monitoring Services"] + Honeypot["OpenCanary / Cowrie Honeypot
(Simulating SSH, HTTP, SMB, RDP)"] + GotifyAlert["Gotify Alerting Bridge
(Instant Mobile Notification)"] + ZabbixLog["Zabbix / Syslog Telemetry"] + end + + VLAN50 -.->|Blocked from accessing| VLAN20 + VLAN80 -.->|Blocked from accessing| VLAN20 + VLAN80 -.->|Blocked from accessing| VLAN40 + VLAN50 -->|Allowed only port 53| DropInterVLAN + VLAN90 --> Honeypot + Honeypot -->|Triggered Event| GotifyAlert + Honeypot -->|Audit Event| ZabbixLog +``` + +### 3.1 Honeypot Implementation: OpenCanary / Cowrie +* **OpenCanary:** + * Low-interaction, highly modular honeypot that exposes simulated services: + * Port 22 (SSH fake login banner) + * Port 80 / 8080 (Fake router admin login) + * Port 445 (Fake SMB share) + * Port 3389 (Fake RDP prompt) + * Port 21 (Fake FTP) + * Any packet or connection attempt to these ports triggers an immediate high-priority alert via webhook to Gotify and logs to syslog/Zabbix. +* **Placement Options:** + * Option A: Deployed on a dedicated honeypot IP in the IoT/Guest subnet to detect lateral movement. + * Option B: Containerized pod in Spaceport with hostPort binding to a decoy IP. + +### 3.2 UDM Pro Firewall Rules Matrix + +| Rule Name | Direction | Source | Destination | Action | Purpose | +|---|---|---|---|:---:|---| +| `Allow-Established-Related` | `LAN IN` | Any | Any | **ACCEPT** | Allows responses to existing connections | +| `Allow-DNS-Traffic` | `LAN IN` | All VLANs | `140.44.4.35`, `140.44.4.65` (port 53) | **ACCEPT** | Allows all devices to query the DNS stack | +| `Allow-NTP-Traffic` | `LAN IN` | All VLANs | `140.44.4.11`, `14.4.4.44` (port 123) | **ACCEPT** | Allows Chrony time synchronization | +| `Isolate-IoT-to-Infra` | `LAN IN` | `VLAN 50 (SmartHome)` | `VLAN 20 (Infra)`, `VLAN 40 (Storage)` | **DROP** | Prevents compromised IoT devices from pivoting into servers | +| `Isolate-Guest-Network` | `GUEST IN` | `VLAN 80 (Guest)` | RFC1918 / Private Subnets | **DROP** | Isolates guest devices to Internet-only | +| `Isolate-Storage-VLAN` | `LAN IN` | Non-Storage Subnets | `VLAN 40 (14.10.10.0/24)` | **DROP** | Restricts Synology NFS/iSCSI to hypervisors and K8s storage nodes | + +--- + +## 4. Phased Execution Roadmap + +### Phase 1: Clean Up Legacy Stubs +- Delete broken deployments, services, and lingering pods in namespace `infra` (Aerodrome) and `infrastructure` (Spaceport). +- Verify MetalLB IP pool allocations: + - Allocate `140.44.4.35` on Aerodrome for Primary DNS. + - Allocate `140.44.4.65` on Spaceport for Secondary DNS. + +### Phase 2: Deploy Pi-hole + Technitium on Aerodrome (Primary Stack) +- Create Longhorn PVCs for Pi-hole config (`etc-pihole`, `etc-dnsmasq`) and Technitium config (`technitium-data`). +- Deploy Technitium DNS container (port 5353 internally). +- Deploy Pi-hole container (port 53 externally via MetalLB `140.44.4.35`, upstream pointing to Technitium `127.0.0.1:5353`). +- Configure `FTLCONF_RATE_LIMIT=0` and subscribe to standard curated blocklists. +- Configure `snarfnet.net` forward and reverse zones in Technitium. + +### Phase 3: Deploy Pi-hole + Technitium on Spaceport (Secondary Stack) +- Deploy identical stack on Spaceport bound to MetalLB VIP `140.44.4.65`. +- Establish Technitium zone replication between Primary (Aerodrome) and Secondary (Spaceport). +- Test dual-stack failover by querying both VIPs independently. + +### Phase 4: Network Honeypot & Alerting Setup +- Deploy OpenCanary / Cowrie with active listeners on decoy ports. +- Configure Gotify webhook integration for instant mobile push notifications upon probe detection. + +### Phase 5: UDM Pro Network Firewall & VLAN Policy Enforcement +- Configure DHCP on UDM Pro to distribute `140.44.4.35` and `140.44.4.65` across all VLANs. +- Apply and verify LAN IN / GUEST IN firewall rules on UDM Pro. +- Apply Kubernetes `NetworkPolicy` objects for cluster namespace segmentation.