diff --git a/docs/infrastructure_improvement_plan.md b/docs/infrastructure_improvement_plan.md index b862a62..8791d25 100644 --- a/docs/infrastructure_improvement_plan.md +++ b/docs/infrastructure_improvement_plan.md @@ -57,8 +57,8 @@ Our automated live probe across the homelab APIs identified the following baseli 1. **Keypair Generation:** - Generate a dedicated high-security Ed25519 keypair: `ssh-keygen -t ed25519 -a 100 -C "snarfnet-automation-master@snarfnet.net" -f id_ed25519_snarfnet`. 2. **Standardized Automation Account (`snarf-admin` / `root`):** - - Bootstrap all 30+ hosts across VLANs 1, 2, 20, 40, and 50 with a single enrollment script that: - - Installs the public key into `/root/.ssh/authorized_keys` and `/home/snarf/.ssh/authorized_keys` (plus `rancher` on Harvester nodes and `admin` on Synology DSM). + - Bootstrap all 28 target hosts across VLANs 1, 20, 40, and 50 (skipping `spacedock` Harvester nodes `140.44.4.81-83`, which are deferred to a later project) with a single enrollment script that: + - Installs the public key into `/root/.ssh/authorized_keys` and `/home/snarf/.ssh/authorized_keys` (plus `admin` on Synology DSM). - Configures `/etc/sudoers.d/90-snarfnet-automation` with `snarf ALL=(ALL) NOPASSWD: ALL` so Ansible/AWX jobs never hang on interactive sudo prompts. - Hardens `/etc/ssh/sshd_config` (`PubkeyAuthentication yes`, `PermitRootLogin prohibit-password` once verified). 3. **Secure Key Storage (Zero Plaintext in Git):** @@ -68,8 +68,8 @@ Our automated live probe across the homelab APIs identified the following baseli - **GitOps Secret Encryption:** Use **Ansible Vault** (for playbooks in Gitea) and **SOPS + Age** (for Kubernetes secrets in ArgoCD) so the Gitea repository contains zero plaintext credentials. #### Predicted Roadblocks & Mitigations -- **Roadblock 1.1 — Harvester HCI Immutable Root (`/home/rancher`):** Harvester v1.8.0 uses an elemental immutable filesystem; writing to `/etc/ssh` directly can be wiped on upgrade. - - *Mitigation:* Persist the SSH public key via Harvester's `/oem/99_custom.yaml` cloud-init configuration and the `Harvester Setting -> ssh-keys` CRD in Kubernetes (`spacedock`). +- **Roadblock 1.1 — Harvester HCI Immutable Root (`/home/rancher`):** + - *Status:* Skipped / Deferred. Harvester nodes (`140.44.4.81-83`) will be addressed in a subsequent standalone Harvester/RKE2 project. - **Roadblock 1.2 — Synology DSM Home Directory Permissions:** Synology DSM (`14.10.10.71` & `.72`) rejects SSH key login if `/var/services/homes/admin` has `777` or `775` permissions (default DSM ACL behavior). - *Mitigation:* Run `chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys && chmod 755 ~` on both NAS appliances during enrollment. - **Roadblock 1.3 — Currently Offline Nodes (`140.44.4.21`, `.25`/`.27`, `.73`, `.75`):** @@ -95,10 +95,9 @@ flowchart TD NTP_SEC["Secondary Redundant NTP Server\naerodrome-ctl01 (140.44.4.11) / nutserver (14.4.4.44)\nChrony Engine (Port UDP 123)"] end - subgraph InternalClients ["Homelab Infrastructure Clients (31+ Hosts)"] + subgraph InternalClients ["Homelab Infrastructure Clients (28 Target Hosts)"] K8S_A["aerodrome K3s (10 Nodes)\n140.44.4.11-13, 21-27"] K8S_S["spaceport K3s (10 Nodes)\n140.44.4.41-44, 51-56"] - HVS["spacedock Harvester (3 Nodes)\n140.44.4.81-83"] SWM["shipyard Swarm (5 Nodes)\n140.44.4.71-75"] NAS["Synology NAS (2 Appliances)\n14.10.10.71, 72"] SMART["Smart Home & AI Pis (4 Hosts)\n14.4.4.20, 14.4.4.44, 140.44.4.7-8"] @@ -118,7 +117,6 @@ flowchart TD NTP_PRI -->|"Sub-millisecond NTP Broadcast / Unicast"| K8S_A NTP_PRI -->|"Sub-millisecond NTP Broadcast / Unicast"| K8S_S - NTP_PRI -->|"Sub-millisecond NTP Broadcast / Unicast"| HVS NTP_PRI -->|"Sub-millisecond NTP Broadcast / Unicast"| SWM NTP_PRI -->|"Sub-millisecond NTP Broadcast / Unicast"| NAS NTP_PRI -->|"Sub-millisecond NTP Broadcast / Unicast"| SMART @@ -167,7 +165,8 @@ flowchart TD - **DietPi Nodes (`spaceport-ctl01-04`, `spaceport-wrk01-04, 06`):** Configure `/etc/systemd/timesyncd.conf` or `/etc/chrony/chrony.conf` to prioritize the internal server, disabling DietPi's delayed hourly cron sync in favor of persistent daemon-based synchronization. - **Harvester HCI Nodes (`spacedock` `140.44.4.81-83`):** - Inject the internal NTP server configuration via Harvester's cloud-init OEM layer (`/oem/99_custom.yaml` or Harvester NTP settings UI), ensuring hypervisor node clocks match K8s downstreams. + > [!NOTE] + > **Project Scope Exclusion:** The baremetal Harvester HCI nodes (`spacedock` cluster `140.44.4.81-83`) are explicitly skipped and deferred to a separate subsequent project. All steps target the 28 active hosts across Aerodrome, Spaceport, Shipyard, Synology NAS, AI Pis, and NUT Master. - **Synology DSM (`14.10.10.71` & `14.10.10.72`):** Set Regional Options $\rightarrow$ NTP Server to `140.44.4.1` with daily synchronization schedule.