Compare commits
36 Commits
main
...
a07cecb6f3
| Author | SHA1 | Date | |
|---|---|---|---|
| a07cecb6f3 | |||
| ffc9ab8ab0 | |||
| d4b367794c | |||
| 3871d04d48 | |||
| 0ed293b6ea | |||
| d5c2f6eae5 | |||
| 8e1c5f1e66 | |||
| 1965d37687 | |||
| aea8608f3d | |||
| 06c203485f | |||
| b12c143791 | |||
| d9ca093155 | |||
| 1b8c862b10 | |||
| 00f22f33af | |||
| 47acedb455 | |||
| fffb446206 | |||
| fa969fc1d3 | |||
| f1c197422d | |||
| 17ca433d0c | |||
| 88191155d2 | |||
| cbdb37a453 | |||
| 1fce3f2488 | |||
| 14fac7151f | |||
| b974980d08 | |||
| 5b955fcbc3 | |||
| 80738fd8c1 | |||
| 11749df634 | |||
| 1e7935e789 | |||
| 706fccacb1 | |||
| 114b619c1f | |||
| 45f9b557b9 | |||
| 07dae0c73e | |||
| 93e13c7482 | |||
| 8b21087c89 | |||
| ba47f6a384 | |||
| 1b378b29f3 |
182
docs/DNS_Setup_Plan.md
Normal file
182
docs/DNS_Setup_Plan.md
Normal file
@@ -0,0 +1,182 @@
|
|||||||
|
# Project Blueprint: High-Availability DNS Stack, Network Honeypot & UDM Pro Network Security
|
||||||
|
|
||||||
|
## 1. Executive Summary & Live UDM Pro Topology Assessment
|
||||||
|
|
||||||
|
Based on the live UDM Pro controller diagnostics, we have mapped the exact network topography, identified critical security exposures, and tailored the implementation plan.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 1.1 Live Network & VLAN Topography
|
||||||
|
|
||||||
|
| VLAN ID | Network Name | Subnet / CIDR | Gateway IP | Active Leases | Security Classification |
|
||||||
|
|:---:|---|---|---|:---:|---|
|
||||||
|
| **1** | **LAN** | `14.44.4.0/25` | `14.44.4.1` | 14 / 117 | Trusted Internal Clients |
|
||||||
|
| **2** | **Management** | `140.44.44.0/24` | `140.44.44.1` | 9 / 51 | Out-of-band (iDRAC, Switches, APs) |
|
||||||
|
| **3** | **Spacedock** | `140.44.1.0/24` | `140.44.1.1` | 0 / 249 | Harvester VM / Hypervisor Tenant |
|
||||||
|
| **20** | **Compute** | `140.44.4.0/25` | `140.44.4.1` | 1 / 15 (Static Nodes) | Mission-Critical K3s Clusters, Swarm |
|
||||||
|
| **30** | **Media** | `192.168.1.0/25` | `192.168.1.1` | 1 / 91 | Streaming & Entertainment Devices |
|
||||||
|
| **40** | **Storage** | `14.10.10.0/25` | `14.10.10.1` | 2 / 125 | Synology NAS (`.71`, `.72`) NFS / iSCSI |
|
||||||
|
| **50** | **Smart Home** | `14.4.4.0/25` | `14.4.4.1` | 53 / 85 | IoT Devices, Gaming Host (`14.4.4.15`) |
|
||||||
|
| **60** | **HA** | `140.4.4.0/25` | `140.4.4.1` | 0 / 7 | High-Availability Interconnects |
|
||||||
|
| **VPN** | **WireGuard / Teleport** | `192.168.3.0/24` / `.2.0/24` | UDM Pro | Remote | Ingress Remote Access |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 1.2 Critical Security Findings from UDM Pro Diagnostics
|
||||||
|
|
||||||
|
1. **Default Security Posture is `Allow All`:**
|
||||||
|
* Currently, the UDM Pro routes traffic freely between all subnets without restriction.
|
||||||
|
* **Risk:** The 53 IoT devices in Smart Home (`14.4.4.0/25`) and Media devices can directly initiate connections to Kubernetes nodes, etcd ports, SSH, and Synology NAS shares in `Compute` (`140.44.4.0/25`) and `Storage` (`14.10.10.0/25`).
|
||||||
|
2. **WAN Port Forwarding to Smart Home Host (`14.4.4.15`):**
|
||||||
|
* Port forwards for game servers (`FS25` ports 8443/10823, `Stationeers` ports 6443/27015) point directly from WAN (`174.106.252.96`) to host `14.4.4.15` on the Smart Home VLAN.
|
||||||
|
* If an exploit compromises this host, an attacker currently has unhindered lateral access to the entire homelab infrastructure.
|
||||||
|
3. **DNS Leaks & Lack of Local Filtering:**
|
||||||
|
* UDM Pro WAN is set to `Auto DNS Server` (Spectrum ISP DNS `0.0.0.0`).
|
||||||
|
* No DNS ad/malware filtering or split-horizon DNS caching is currently active across the subnets.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. High-Availability DNS Stack (Pi-hole + Technitium)
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
subgraph Clients["LAN Clients across all VLANs (VLAN 1, 20, 30, 50, VPN)"]
|
||||||
|
Devices["IoT Devices (53), Laptops, Servers, Hypervisors"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph UDM["UniFi Dream Machine Pro (140.44.4.1)"]
|
||||||
|
DHCP["DHCP Option 6 per VLAN:<br>Primary DNS: 140.44.4.35 (Aerodrome VIP)<br>Secondary DNS: 140.44.4.65 (Spaceport VIP)"]
|
||||||
|
NAT["DNS Redirection Rule:<br>Capture port 53 to hardcoded DNS (8.8.8.8) -> Redirect to VIPs"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph AeroCluster["Aerodrome K3s Cluster (VLAN 20)"]
|
||||||
|
AeroVIP["MetalLB VIP: 140.44.4.35:53 (UDP/TCP)"]
|
||||||
|
AeroPihole["Pi-hole Instance 1<br>(Ad-blocking, Caching, FTL rate-limit: 0)"]
|
||||||
|
AeroTech["Technitium Instance 1 (Primary Authoritative)<br>(snarfnet.net Zone, Reverse PTR, DNSSEC)"]
|
||||||
|
AeroStorage["Longhorn Storage Volume"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph SpaceCluster["Spaceport K3s Cluster (VLAN 20)"]
|
||||||
|
SpaceVIP["MetalLB VIP: 140.44.4.65:53 (UDP/TCP)"]
|
||||||
|
SpacePihole["Pi-hole Instance 2<br>(Ad-blocking, Caching, FTL rate-limit: 0)"]
|
||||||
|
SpaceTech["Technitium Instance 2 (Secondary Authoritative)<br>(Zone Transfer AXFR/IXFR from Primary)"]
|
||||||
|
SpaceStorage["Longhorn Storage Volume"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph Upstream["Encrypted Upstream Resolvers"]
|
||||||
|
DoH["DNS-over-HTTPS (Quad9 / Cloudflare / Mullvad)"]
|
||||||
|
end
|
||||||
|
|
||||||
|
Devices -->|DNS Queries (UDP/TCP 53)| DHCP
|
||||||
|
DHCP -->|Primary| AeroVIP
|
||||||
|
DHCP -.->|Secondary / Failover| SpaceVIP
|
||||||
|
AeroVIP --> AeroPihole
|
||||||
|
SpaceVIP --> SpacePihole
|
||||||
|
AeroPihole -->|Recursive / Local Lookups| AeroTech
|
||||||
|
SpacePihole -->|Recursive / Local Lookups| SpaceTech
|
||||||
|
AeroTech <==>|Automatic Zone Sync (AXFR)| SpaceTech
|
||||||
|
AeroTech -->|Encrypted DoH| DoH
|
||||||
|
SpaceTech -->|Encrypted DoH| DoH
|
||||||
|
AeroPihole --- AeroStorage
|
||||||
|
AeroTech --- AeroStorage
|
||||||
|
SpacePihole --- SpaceStorage
|
||||||
|
SpaceTech --- SpaceStorage
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.1 Addressing the Rate-Limiting & Sluggishness Issue
|
||||||
|
1. **FTL Rate Limit Bypass:** Set `FTLCONF_RATE_LIMIT=0` so local bursts (such as 53 Smart Home devices polling simultaneously) are never throttled.
|
||||||
|
2. **Technitium Engine:** Replaces single-threaded Unbound with a high-performance .NET 9 asynchronous resolver capable of servicing tens of thousands of queries per second with sub-millisecond RAM caching.
|
||||||
|
3. **High Availability:** MetalLB VIPs on Aerodrome (`140.44.4.35`) and Spaceport (`140.44.4.65`) allow continuous service even during node reboots or maintenance.
|
||||||
|
4. **Authoritative Split-Horizon:** Technitium will authoritatively host the internal `snarfnet.net` forward zone and reverse PTR zones (`4.44.140.in-addr.arpa`, `4.4.14.in-addr.arpa`, `10.10.14.in-addr.arpa`), eliminating slow mDNS or external lookup delays.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Network Honeypot & UDM Pro Firewall Hardening
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
subgraph WAN_Threats["External & Lateral Threats"]
|
||||||
|
Internet["Public Internet"]
|
||||||
|
CompromisedIoT["Compromised IoT / Host (e.g. 14.4.4.15)"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph UDM_FW["UDM Pro Firewall Rules (LAN IN)"]
|
||||||
|
Rule1["Rule 1: Allow Established & Related"]
|
||||||
|
Rule2["Rule 2: Allow DNS (Port 53 to .35 & .65)"]
|
||||||
|
Rule3["Rule 3: Allow NTP (Port 123 to .11 & .44)"]
|
||||||
|
Rule4["Rule 4: Drop IoT -> Compute (VLAN 50 -> 20)"]
|
||||||
|
Rule5["Rule 5: Drop IoT -> Storage (VLAN 50 -> 40)"]
|
||||||
|
Rule6["Rule 6: Drop Media -> Compute (VLAN 30 -> 20)"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph HoneypotNet["Honeypot Decoy (VLAN 50 / Decoy IP 14.4.4.99)"]
|
||||||
|
OpenCanary["OpenCanary Honeypot<br>(Simulating SSH:22, HTTP:80, SMB:445, RDP:3389)"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph AlertSystem["Alerting & Audit Pipeline"]
|
||||||
|
Gotify["Gotify Push Notification<br>(Instant alert to your phone)"]
|
||||||
|
Zabbix["Zabbix Security Audit Log"]
|
||||||
|
end
|
||||||
|
|
||||||
|
CompromisedIoT -.->|Probes decoy IP| OpenCanary
|
||||||
|
OpenCanary -->|High Priority Webhook| Gotify
|
||||||
|
OpenCanary -->|Syslog Event| Zabbix
|
||||||
|
CompromisedIoT -->|Blocked by Rule 4| UDM_FW
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.1 Network Honeypot Deployment
|
||||||
|
* **Tool:** **OpenCanary** (lightweight, zero false-positives).
|
||||||
|
* **Placement:**
|
||||||
|
* Assigned a decoy IP on the Smart Home subnet (e.g., `14.4.4.99`) where the gaming port forwards exist.
|
||||||
|
* Emulates vulnerable services: SSH (`22`), Telnet (`23`), Web admin (`80`/`8080`), SMB (`445`), and RDP (`3389`).
|
||||||
|
* **Alerting:**
|
||||||
|
* When any internal device attempts a port scan or lateral connection to `14.4.4.99`, OpenCanary immediately sends a critical push notification to **Gotify** (`gotify.snarfnet.net`) and generates an alert in **Zabbix**.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 3.2 UDM Pro Firewall Rules Matrix (LAN IN)
|
||||||
|
|
||||||
|
We will define firewall groups on the UDM Pro:
|
||||||
|
* **RFC1918_Subnets:** All internal private ranges (`140.44.0.0/16`, `14.0.0.0/8`, `192.168.0.0/16`).
|
||||||
|
* **K8s_DNS_VIPs:** `140.44.4.35`, `140.44.4.65`.
|
||||||
|
* **NTP_Servers:** `140.44.4.11`, `14.4.4.44`.
|
||||||
|
|
||||||
|
| Order | Rule Name | Type | Source | Destination | Action | Purpose |
|
||||||
|
|:---:|---|:---:|---|---|:---:|---|
|
||||||
|
| **2001** | `Allow Established/Related` | LAN IN | Any | Any (State: Established, Related) | **ACCEPT** | Essential return traffic |
|
||||||
|
| **2002** | `Allow Cluster DNS` | LAN IN | All VLANs | `K8s_DNS_VIPs` (Port: 53 UDP/TCP) | **ACCEPT** | Clients resolve DNS |
|
||||||
|
| **2003** | `Allow Cluster NTP` | LAN IN | All VLANs | `NTP_Servers` (Port: 123 UDP) | **ACCEPT** | Chrony time sync |
|
||||||
|
| **2004** | `Allow Trusted to All` | LAN IN | `VLAN 1 (LAN)`, `VLAN 20 (Compute)` | RFC1918_Subnets | **ACCEPT** | Admins & servers access all VLANs |
|
||||||
|
| **2005** | `Isolate SmartHome to Compute` | LAN IN | `VLAN 50 (Smart Home)` | `VLAN 20 (Compute)` | **DROP** | Block IoT pivoting into K8s |
|
||||||
|
| **2006** | `Isolate SmartHome to Storage` | LAN IN | `VLAN 50 (Smart Home)` | `VLAN 40 (Storage)` | **DROP** | Protect Synology NAS from IoT |
|
||||||
|
| **2007** | `Isolate Media to Compute` | LAN IN | `VLAN 30 (Media)` | `VLAN 20 (Compute)` | **DROP** | Block media devices from cluster |
|
||||||
|
| **2008** | `Isolate Guest Network` | GUEST IN | Guest Network | RFC1918_Subnets | **DROP** | Restrict guest traffic to Internet only |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Phased Implementation Roadmap
|
||||||
|
|
||||||
|
### Phase 1: Teardown Lingering Stubs
|
||||||
|
* Cleanly delete orphaned, broken deployments and services in `infra` (Aerodrome) and `infrastructure` (Spaceport):
|
||||||
|
* Remove `deployment/pihole`, `deployment/technitium-dns`, and non-functional NodePort services.
|
||||||
|
|
||||||
|
### Phase 2: Deploy Dual-Cluster Pi-hole + Technitium
|
||||||
|
* Create Longhorn storage volumes on both clusters for persistent configs and blocklists.
|
||||||
|
* Deploy **Aerodrome Primary Stack**:
|
||||||
|
* Technitium on internal port `5353`.
|
||||||
|
* Pi-hole with MetalLB LoadBalancer VIP **`140.44.4.35:53`** (UDP & TCP).
|
||||||
|
* Deploy **Spaceport Secondary Stack**:
|
||||||
|
* Technitium on internal port `5353` configured for AXFR zone transfers from Aerodrome.
|
||||||
|
* Pi-hole with MetalLB LoadBalancer VIP **`140.44.4.65:53`** (UDP & TCP).
|
||||||
|
* Populate `snarfnet.net` local records and reverse PTR zones.
|
||||||
|
|
||||||
|
### Phase 3: Network Honeypot Deployment
|
||||||
|
* Deploy containerized OpenCanary with virtual listener on `14.4.4.99`.
|
||||||
|
* Configure webhook bridge to Gotify server (`140.44.4.70`) for immediate push notification upon any probe.
|
||||||
|
|
||||||
|
### Phase 4: UDM Pro Network Configuration & Policy Hardening
|
||||||
|
* Configure DHCP Scope DNS servers on UDM Pro for all VLANs:
|
||||||
|
* Primary: `140.44.4.35`
|
||||||
|
* Secondary: `140.44.4.65`
|
||||||
|
* Set UDM Pro WAN DNS to `140.44.4.35` & `140.44.4.65` (or Quad9 fallback).
|
||||||
|
* Apply LAN IN firewall rule matrix on UDM Pro to enforce strict VLAN isolation.
|
||||||
59
docs/opencanary_honeypot.md
Normal file
59
docs/opencanary_honeypot.md
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
# OpenCanary Decoy Honeypot Architecture & Configuration
|
||||||
|
|
||||||
|
## 1. Overview
|
||||||
|
As part of the **Snarfnet Network Security Hardening Blueprint (Phase 3)**, an **OpenCanary Honeypot** has been deployed to detect lateral movement, unauthorized reconnaissance, and compromised devices inside the **Smart Home VLAN (VLAN 50, `14.4.4.0/25`)**.
|
||||||
|
|
||||||
|
* **Decoy IP:** `14.4.4.99` (Subnet: `14.4.4.0/25`, Gateway: `14.4.4.1`)
|
||||||
|
* **Host Machine:** `snshpd4lnxswm01` (`140.44.4.71`)
|
||||||
|
* **Physical Uplink:** Direct 802.1Q trunk port with tagged VLAN 50 support.
|
||||||
|
* **Network Mode:** Native Docker `macvlan` attached to subinterface `eth0.50`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Emulated Services & Ports
|
||||||
|
|
||||||
|
The honeypot presents realistic service banners designed to trap automated scanners and unauthorized intruders without exposing real systems:
|
||||||
|
|
||||||
|
| Port | Protocol | Emulated Service | Decoy Profile |
|
||||||
|
|:---:|:---:|:---|:---|
|
||||||
|
| **21** | TCP | **FTP** | Linux FTP server (`FTP server ready`) |
|
||||||
|
| **22** | TCP | **SSH** | Ubuntu OpenSSH banner (`SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.10`) |
|
||||||
|
| **23** | TCP | **Telnet** | Ubuntu login prompt (`Ubuntu 22.04 LTS login: `) |
|
||||||
|
| **80** | TCP | **HTTP Web** | Synology/NAS Web Admin Login skin |
|
||||||
|
| **1433** | TCP | **MSSQL** | Microsoft SQL Server 2012 |
|
||||||
|
| **3306** | TCP | **MySQL** | MySQL 5.7 database server |
|
||||||
|
| **3389** | TCP | **RDP** | Windows Remote Desktop Protocol listener |
|
||||||
|
| **5000** | TCP | **VNC** | Remote framebuffer / VNC listener |
|
||||||
|
| **8080** | TCP | **HTTP Proxy** | Squid HTTP proxy decoy |
|
||||||
|
| **9418** | TCP | **Git** | Git daemon repository service |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Real-Time Alerting Pipeline
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
Attacker["Suspicious Host / IoT Device<br>(e.g. 14.4.4.15)"] -->|TCP / Port Probe| Decoy["OpenCanary (14.4.4.99:22/80/445/3389)"]
|
||||||
|
Decoy -->|Structured JSON Log| LogFile["/opt/opencanary/logs/opencanary.log"]
|
||||||
|
LogFile -->|Real-Time Tail| Watcher["opencanary-watcher.service<br>(Python Daemon)"]
|
||||||
|
Watcher -->|HTTP POST (Priority 8)| Gotify["Gotify Server (App 6)<br>(gotify.snarfnet.net)"]
|
||||||
|
Gotify -->|Instant Push Notification| Phone["Admin Mobile App"]
|
||||||
|
Gotify -->|Discord Webhook Bridge| Discord["Discord #infra-alerts"]
|
||||||
|
```
|
||||||
|
|
||||||
|
1. Any TCP connection, banner grab, authentication attempt, or probe arriving on `14.4.4.99` generates a structured event in `/opt/opencanary/logs/opencanary.log`.
|
||||||
|
2. The `opencanary-watcher.service` daemon ingests the event in real-time, extracts the attacker's source IP, source port, targeted port, service type, and credentials (if entered).
|
||||||
|
3. The event is posted to **Gotify Application #6** (`A9oPeNcAnArYhOnEyPoT`) with high priority (`8`).
|
||||||
|
4. `gotify-discord-bridge` formats a rich embed and posts the alert directly into the Discord `#infra-alerts` channel.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Systemd Services & Maintenance
|
||||||
|
|
||||||
|
* **VLAN Subinterface Service:**
|
||||||
|
`/etc/systemd/system/opencanary-vlan.service` (Ensures `eth0.50` is brought up on boot).
|
||||||
|
* **Alert Watcher Service:**
|
||||||
|
`/etc/systemd/system/opencanary-watcher.service` (Tails log file and delivers push alerts).
|
||||||
|
* **Docker Container:**
|
||||||
|
`opencanary` (Managed via Docker with `restart: unless-stopped` on network `opencanary_net`).
|
||||||
|
|
||||||
Submodule homelab-gitops-repo updated: cb8e60b086...2bd66045c3
65
scripts/apply_36.py
Normal file
65
scripts/apply_36.py
Normal file
@@ -0,0 +1,65 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.11", username="root", key_filename=key_path, timeout=5)
|
||||||
|
|
||||||
|
cmd = """
|
||||||
|
k3s kubectl -n dns delete svc pihole-dns-vip pihole-web-vip --wait=false
|
||||||
|
sleep 2
|
||||||
|
cat << 'EOF' | k3s kubectl apply -f -
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: pihole-dns-vip
|
||||||
|
namespace: dns
|
||||||
|
annotations:
|
||||||
|
metallb.universe.tf/loadBalancerIPs: 140.44.4.36
|
||||||
|
metallb.universe.tf/allow-shared-ip: "aerodrome-dns-key"
|
||||||
|
labels:
|
||||||
|
app: pihole
|
||||||
|
spec:
|
||||||
|
type: LoadBalancer
|
||||||
|
selector:
|
||||||
|
app: pihole
|
||||||
|
ports:
|
||||||
|
- name: dns-udp
|
||||||
|
port: 53
|
||||||
|
targetPort: 53
|
||||||
|
protocol: UDP
|
||||||
|
- name: dns-tcp
|
||||||
|
port: 53
|
||||||
|
targetPort: 53
|
||||||
|
protocol: TCP
|
||||||
|
---
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: pihole-web-vip
|
||||||
|
namespace: dns
|
||||||
|
annotations:
|
||||||
|
metallb.universe.tf/loadBalancerIPs: 140.44.4.36
|
||||||
|
metallb.universe.tf/allow-shared-ip: "aerodrome-dns-key"
|
||||||
|
labels:
|
||||||
|
app: pihole
|
||||||
|
spec:
|
||||||
|
type: LoadBalancer
|
||||||
|
selector:
|
||||||
|
app: pihole
|
||||||
|
ports:
|
||||||
|
- name: web-http
|
||||||
|
port: 80
|
||||||
|
targetPort: 80
|
||||||
|
protocol: TCP
|
||||||
|
EOF
|
||||||
|
sleep 3
|
||||||
|
k3s kubectl -n dns get svc
|
||||||
|
"""
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd)
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
c.close()
|
||||||
|
|
||||||
52
scripts/apply_and_set_pihole_pw.py
Normal file
52
scripts/apply_and_set_pihole_pw.py
Normal file
@@ -0,0 +1,52 @@
|
|||||||
|
import paramiko
|
||||||
|
import time
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
|
||||||
|
def apply_and_update(ip, name, manifest_path):
|
||||||
|
print(f"\n=======================================================")
|
||||||
|
print(f"Applying to {name} ({ip})...")
|
||||||
|
with open(manifest_path, 'r', encoding='utf-8') as f:
|
||||||
|
content = f.read()
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect(ip, username='root', key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("k3s kubectl apply -f -")
|
||||||
|
stdin.write(content)
|
||||||
|
stdin.channel.shutdown_write()
|
||||||
|
print("Apply output:", stdout.read().decode())
|
||||||
|
err = stderr.read().decode()
|
||||||
|
if err:
|
||||||
|
print("Apply stderr:", err)
|
||||||
|
|
||||||
|
print(f"Waiting for Pi-hole deployment rollout on {name}...")
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("k3s kubectl rollout status deployment/pihole -n dns --timeout=60s")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
# Get new pod name
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("k3s kubectl get pods -n dns -l app=pihole -o jsonpath='{.items[0].metadata.name}'")
|
||||||
|
pod_name = stdout.read().decode().strip()
|
||||||
|
print(f"New Pi-hole pod: {pod_name}")
|
||||||
|
|
||||||
|
# Set password to snarfnetadmin
|
||||||
|
print("Setting Pi-hole password to 'snarfnetadmin'...")
|
||||||
|
stdin, stdout, stderr = ssh.exec_command(f"k3s kubectl exec -n dns {pod_name} -- pihole setpassword snarfnetadmin")
|
||||||
|
out = stdout.read().decode()
|
||||||
|
err = stderr.read().decode()
|
||||||
|
print("Set password output:", out)
|
||||||
|
if err:
|
||||||
|
print("Set password stderr:", err)
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
|
aero_manifest = r"C:\Users\Snarf\OneDrive\Desktop\code\SnarfCode\homelab-gitops-repo\manifests\aerodrome\dns\dns-stack.yaml"
|
||||||
|
space_manifest = r"C:\Users\Snarf\OneDrive\Desktop\code\SnarfCode\homelab-gitops-repo\manifests\spaceport\dns\dns-stack.yaml"
|
||||||
|
|
||||||
|
apply_and_update("140.44.4.11", "Aerodrome", aero_manifest)
|
||||||
|
apply_and_update("140.44.4.41", "Spaceport", space_manifest)
|
||||||
|
print("\nDone!")
|
||||||
|
|
||||||
48
scripts/apply_coredns_spaceport.py
Normal file
48
scripts/apply_coredns_spaceport.py
Normal file
@@ -0,0 +1,48 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
cm_yaml = """cat << 'EOF' | k3s kubectl apply -f -
|
||||||
|
apiVersion: v1
|
||||||
|
kind: ConfigMap
|
||||||
|
metadata:
|
||||||
|
name: coredns-custom
|
||||||
|
namespace: kube-system
|
||||||
|
data:
|
||||||
|
snarfnet.override: |
|
||||||
|
hosts {
|
||||||
|
140.44.4.34 aerodrome.snarfnet.net
|
||||||
|
140.44.4.90 harvester.spacedock.snarfnet.net
|
||||||
|
fallthrough
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
"""
|
||||||
|
|
||||||
|
print("Applying coredns-custom on Spaceport...")
|
||||||
|
stdin, stdout, stderr = client.exec_command(cm_yaml)
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
print(stderr.read().decode('utf-8'))
|
||||||
|
|
||||||
|
print("\nRestarting coredns deployment on Spaceport...")
|
||||||
|
client.exec_command("k3s kubectl -n kube-system rollout restart deployment/coredns")
|
||||||
|
client.exec_command("k3s kubectl -n kube-system rollout status deployment/coredns --timeout=60s")
|
||||||
|
|
||||||
|
print("\nRestarting cattle-cluster-agent on Spaceport...")
|
||||||
|
client.exec_command("k3s kubectl -n cattle-system rollout restart deployment/cattle-cluster-agent")
|
||||||
|
client.exec_command("k3s kubectl -n cattle-system rollout status deployment/cattle-cluster-agent --timeout=60s")
|
||||||
|
|
||||||
|
time.sleep(5)
|
||||||
|
print("\nChecking cattle-cluster-agent logs:")
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl -n cattle-system logs -l app=cattle-cluster-agent --tail=25")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
client.close()
|
||||||
|
|
||||||
35
scripts/apply_dns_updates.py
Normal file
35
scripts/apply_dns_updates.py
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
SSH_KEY = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
|
||||||
|
def apply_manifest(host, local_file):
|
||||||
|
with open(local_file, "r") as f:
|
||||||
|
manifest_content = f.read()
|
||||||
|
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
client.connect(host, username="root", key_filename=SSH_KEY, timeout=10)
|
||||||
|
|
||||||
|
# Write to remote /tmp/dns-stack.yaml and kubectl apply
|
||||||
|
sftp = client.open_sftp()
|
||||||
|
with sftp.open("/tmp/dns-stack.yaml", "w") as f:
|
||||||
|
f.write(manifest_content)
|
||||||
|
sftp.close()
|
||||||
|
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl apply -f /tmp/dns-stack.yaml")
|
||||||
|
out = stdout.read().decode()
|
||||||
|
err = stderr.read().decode()
|
||||||
|
client.close()
|
||||||
|
return out, err
|
||||||
|
|
||||||
|
print("=== Applying to Aerodrome (140.44.4.11) ===")
|
||||||
|
out, err = apply_manifest("140.44.4.11", r"homelab-gitops-repo\manifests\aerodrome\dns\dns-stack.yaml")
|
||||||
|
print(out)
|
||||||
|
if err:
|
||||||
|
print("Stderr:", err)
|
||||||
|
|
||||||
|
print("=== Applying to Spaceport (140.44.4.41) ===")
|
||||||
|
out, err = apply_manifest("140.44.4.41", r"homelab-gitops-repo\manifests\spaceport\dns\dns-stack.yaml")
|
||||||
|
print(out)
|
||||||
|
if err:
|
||||||
|
print("Stderr:", err)
|
||||||
140
scripts/apply_npm_fixes.py
Normal file
140
scripts/apply_npm_fixes.py
Normal file
@@ -0,0 +1,140 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.71", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
print("1. Creating backup of database and configs...")
|
||||||
|
backup_cmd = """
|
||||||
|
cp /var/lib/docker/volumes/nginx-proxy-manager_nginx-data/_data/database.sqlite /var/lib/docker/volumes/nginx-proxy-manager_nginx-data/_data/database.sqlite.bak.$(date +%s)
|
||||||
|
"""
|
||||||
|
stdin, stdout, stderr = c.exec_command(backup_cmd)
|
||||||
|
stdout.channel.recv_exit_status()
|
||||||
|
print("Backup complete.")
|
||||||
|
|
||||||
|
print("2. Updating database.sqlite for hosts 45, 41, and 37...")
|
||||||
|
py_db = """
|
||||||
|
import sqlite3, datetime
|
||||||
|
conn = sqlite3.connect("/var/lib/docker/volumes/nginx-proxy-manager_nginx-data/_data/database.sqlite")
|
||||||
|
cur = conn.cursor()
|
||||||
|
now = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||||
|
|
||||||
|
updates = [
|
||||||
|
(45, "140.44.4.34", 443, "https"), # Longhorn -> Aerodrome Traefik
|
||||||
|
(41, "140.44.4.62", 443, "https"), # Semaphore -> Spaceport Traefik
|
||||||
|
(37, "140.44.4.34", 443, "https"), # Uptime Kuma -> Aerodrome Traefik
|
||||||
|
]
|
||||||
|
|
||||||
|
for pid, host, port, scheme in updates:
|
||||||
|
cur.execute('''UPDATE proxy_host
|
||||||
|
SET forward_host = ?, forward_port = ?, forward_scheme = ?,
|
||||||
|
allow_websocket_upgrade = 1, http2_support = 1, modified_on = ?
|
||||||
|
WHERE id = ?''', (host, port, scheme, now, pid))
|
||||||
|
print(f"DB updated: Host {pid} -> {scheme}://{host}:{port}")
|
||||||
|
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
"""
|
||||||
|
stdin, stdout, stderr = c.exec_command(f"python3 -c '{py_db}'")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
err = stderr.read().decode()
|
||||||
|
if err: print("DB err:", err)
|
||||||
|
|
||||||
|
print("3. Generating updated nginx configuration files...")
|
||||||
|
|
||||||
|
def generate_conf(pid, domains, cert_id, host, port, scheme):
|
||||||
|
return f"""# ------------------------------------------------------------
|
||||||
|
# {domains}
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
|
map $scheme $hsts_header {{
|
||||||
|
https "max-age=63072000; preload";
|
||||||
|
}}
|
||||||
|
|
||||||
|
server {{
|
||||||
|
set $forward_scheme {scheme};
|
||||||
|
set $server "{host}";
|
||||||
|
set $port {port};
|
||||||
|
|
||||||
|
listen 80;
|
||||||
|
listen [::]:80;
|
||||||
|
|
||||||
|
listen 443 ssl;
|
||||||
|
listen [::]:443 ssl;
|
||||||
|
|
||||||
|
server_name {domains};
|
||||||
|
http2 on;
|
||||||
|
|
||||||
|
# Let's Encrypt SSL
|
||||||
|
include conf.d/include/letsencrypt-acme-challenge.conf;
|
||||||
|
include conf.d/include/ssl-cache.conf;
|
||||||
|
include conf.d/include/ssl-ciphers.conf;
|
||||||
|
ssl_certificate /etc/letsencrypt/live/npm-{cert_id}/fullchain.pem;
|
||||||
|
ssl_certificate_key /etc/letsencrypt/live/npm-{cert_id}/privkey.pem;
|
||||||
|
|
||||||
|
# Asset Caching
|
||||||
|
include conf.d/include/assets.conf;
|
||||||
|
|
||||||
|
# Block Exploits
|
||||||
|
include conf.d/include/block-exploits.conf;
|
||||||
|
|
||||||
|
# Force SSL
|
||||||
|
set $trust_forwarded_proto "F";
|
||||||
|
include conf.d/include/force-ssl.conf;
|
||||||
|
|
||||||
|
proxy_set_header Upgrade $http_upgrade;
|
||||||
|
proxy_set_header Connection $http_connection;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
|
||||||
|
access_log /data/logs/proxy-host-{pid}_access.log proxy;
|
||||||
|
error_log /data/logs/proxy-host-{pid}_error.log warn;
|
||||||
|
|
||||||
|
location / {{
|
||||||
|
proxy_set_header Upgrade $http_upgrade;
|
||||||
|
proxy_set_header Connection $http_connection;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
|
||||||
|
# Proxy!
|
||||||
|
include conf.d/include/proxy.conf;
|
||||||
|
}}
|
||||||
|
|
||||||
|
# Custom
|
||||||
|
include /data/nginx/custom/server_proxy[.]conf;
|
||||||
|
}}
|
||||||
|
"""
|
||||||
|
|
||||||
|
configs = [
|
||||||
|
(45, "longhorn.aerodrome.snarfnet.net", 37, "140.44.4.34", 443, "https"),
|
||||||
|
(41, "semaphore.snarfnet.net", 33, "140.44.4.62", 443, "https"),
|
||||||
|
(37, "uptime.snarfnet.net", 27, "140.44.4.34", 443, "https"),
|
||||||
|
]
|
||||||
|
|
||||||
|
for pid, dom, cert_id, host, port, scheme in configs:
|
||||||
|
cfg = generate_conf(pid, dom, cert_id, host, port, scheme)
|
||||||
|
path = f"/var/lib/docker/volumes/nginx-proxy-manager_nginx-data/_data/nginx/proxy_host/{pid}.conf"
|
||||||
|
cmd = f"cat << 'EOF' > {path}\n{cfg}\nEOF\n"
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd)
|
||||||
|
stdout.channel.recv_exit_status()
|
||||||
|
print(f"Wrote config: {path}")
|
||||||
|
|
||||||
|
print("4. Testing Nginx config inside container...")
|
||||||
|
stdin, stdout, stderr = c.exec_command("docker exec $(docker ps -q -f name=nginx-proxy-manager_app | head -n1) nginx -t")
|
||||||
|
out = stdout.read().decode()
|
||||||
|
err = stderr.read().decode()
|
||||||
|
print("nginx -t stdout:", out)
|
||||||
|
print("nginx -t stderr:", err)
|
||||||
|
|
||||||
|
if "successful" in out or "successful" in err:
|
||||||
|
print("5. Reloading Nginx via Swarm service update...")
|
||||||
|
stdin, stdout, stderr = c.exec_command("docker service update --force nginx-proxy-manager_app")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
print("Reload completed.")
|
||||||
|
else:
|
||||||
|
print("ERROR: Nginx config test failed!")
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
26
scripts/apply_pihole_redirect.py
Normal file
26
scripts/apply_pihole_redirect.py
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
def apply_manifest(host, cluster_name, file_path):
|
||||||
|
print(f"Applying {file_path} to {cluster_name} ({host})...")
|
||||||
|
with open(file_path, 'r') as f:
|
||||||
|
manifest_content = f.read()
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect(host, username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command('k3s kubectl apply -f -')
|
||||||
|
stdin.write(manifest_content)
|
||||||
|
stdin.channel.shutdown_write()
|
||||||
|
|
||||||
|
out = stdout.read().decode('utf-8')
|
||||||
|
err = stderr.read().decode('utf-8')
|
||||||
|
print("Output:")
|
||||||
|
print(out)
|
||||||
|
if err:
|
||||||
|
print("Stderr:", err)
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
|
apply_manifest('140.44.4.11', 'Aerodrome', r'homelab-gitops-repo/manifests/aerodrome/dns/dns-stack.yaml')
|
||||||
|
apply_manifest('140.44.4.41', 'Spaceport', r'homelab-gitops-repo/manifests/spaceport/dns/dns-stack.yaml')
|
||||||
|
|
||||||
72
scripts/apply_semaphore_fix.py
Normal file
72
scripts/apply_semaphore_fix.py
Normal file
@@ -0,0 +1,72 @@
|
|||||||
|
import paramiko
|
||||||
|
import urllib.request
|
||||||
|
import ssl
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
|
||||||
|
print("1. Updating Authentik Provider 10 redirect_uris...")
|
||||||
|
ctx = ssl.create_default_context()
|
||||||
|
ctx.check_hostname = False
|
||||||
|
ctx.verify_mode = ssl.CERT_NONE
|
||||||
|
|
||||||
|
token = "SnarfNetPermanentAutomationToken2026"
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Bearer {token}",
|
||||||
|
"Content-Type": "application/json"
|
||||||
|
}
|
||||||
|
|
||||||
|
provider_update = {
|
||||||
|
"redirect_uris": [
|
||||||
|
{"matching_mode": "strict", "url": "https://semaphore.snarfnet.net/auth/oidc/authentik/redirect"},
|
||||||
|
{"matching_mode": "strict", "url": "https://semaphore.snarfnet.net/api/auth/oidc/authentik/redirect/"},
|
||||||
|
{"matching_mode": "regex", "url": "https://semaphore\\.snarfnet\\.net/.*"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
req = urllib.request.Request(
|
||||||
|
"https://authentik.snarfnet.net/api/v3/providers/oauth2/10/",
|
||||||
|
headers=headers,
|
||||||
|
data=json.dumps(provider_update).encode('utf-8'),
|
||||||
|
method="PATCH"
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req, context=ctx, timeout=8) as resp:
|
||||||
|
res = json.loads(resp.read().decode())
|
||||||
|
print(" ✓ Authentik Provider 10 updated successfully:", res.get('redirect_uris'))
|
||||||
|
except Exception as e:
|
||||||
|
print(" ✗ Failed to update Authentik Provider 10:", e)
|
||||||
|
|
||||||
|
print("\n2. Applying updated Semaphore manifest to Spaceport (140.44.4.41)...")
|
||||||
|
manifest_path = r"C:\Users\Snarf\OneDrive\Desktop\code\SnarfCode\homelab-gitops-repo\manifests\spaceport\semaphore\semaphore.yaml"
|
||||||
|
with open(manifest_path, 'r', encoding='utf-8') as f:
|
||||||
|
content = f.read()
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("k3s kubectl apply -f -")
|
||||||
|
stdin.write(content)
|
||||||
|
stdin.channel.shutdown_write()
|
||||||
|
print("Apply output:", stdout.read().decode())
|
||||||
|
err = stderr.read().decode()
|
||||||
|
if err: print("Apply stderr:", err)
|
||||||
|
|
||||||
|
print("3. Rolling out Semaphore deployment...")
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("k3s kubectl rollout restart deployment/semaphore -n infrastructure; k3s kubectl rollout status deployment/semaphore -n infrastructure --timeout=90s")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
# Check new pod env
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("k3s kubectl get pods -n infrastructure -l app.kubernetes.io/name=semaphore -o jsonpath='{.items[0].metadata.name}'")
|
||||||
|
new_pod = stdout.read().decode().strip()
|
||||||
|
print(f"New pod: {new_pod}")
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command(f"k3s kubectl exec -n infrastructure {new_pod} -- env | grep SEMAPHORE_OIDC_PROVIDERS")
|
||||||
|
print("New env:", stdout.read().decode())
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
print("Done!")
|
||||||
|
|
||||||
161
scripts/apply_synology_fixes.py
Normal file
161
scripts/apply_synology_fixes.py
Normal file
@@ -0,0 +1,161 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.71", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
print("1. Updating SQLite database for hosts 11 and 28...")
|
||||||
|
dsm_adv = """proxy_buffer_size 128k;
|
||||||
|
proxy_buffers 8 256k;
|
||||||
|
proxy_busy_buffers_size 256k;
|
||||||
|
proxy_read_timeout 3600s;
|
||||||
|
proxy_send_timeout 3600s;
|
||||||
|
proxy_ssl_verify off;"""
|
||||||
|
|
||||||
|
py_db = f"""
|
||||||
|
import sqlite3, datetime
|
||||||
|
conn = sqlite3.connect('/var/lib/docker/volumes/nginx-proxy-manager_nginx-data/_data/database.sqlite')
|
||||||
|
cur = conn.cursor()
|
||||||
|
now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
|
||||||
|
|
||||||
|
adv = '''{dsm_adv}'''
|
||||||
|
|
||||||
|
# Host 11: sn-nas.snarfnet.net -> 14.10.10.71:5001
|
||||||
|
cur.execute('''UPDATE proxy_host
|
||||||
|
SET forward_host = "14.10.10.71", forward_port = 5001, forward_scheme = "https",
|
||||||
|
allow_websocket_upgrade = 1, http2_support = 1, advanced_config = ?, modified_on = ?
|
||||||
|
WHERE id = 11''', (adv, now))
|
||||||
|
|
||||||
|
# Host 28: snarfnas.snarfnet.net -> 14.10.10.72:5001 (Clean DSM, remove dead outpost)
|
||||||
|
cur.execute('''UPDATE proxy_host
|
||||||
|
SET forward_host = "14.10.10.72", forward_port = 5001, forward_scheme = "https",
|
||||||
|
allow_websocket_upgrade = 1, http2_support = 1, advanced_config = ?, modified_on = ?
|
||||||
|
WHERE id = 28''', (adv, now))
|
||||||
|
|
||||||
|
conn.commit()
|
||||||
|
print("Database updated for hosts 11 and 28.")
|
||||||
|
conn.close()
|
||||||
|
"""
|
||||||
|
|
||||||
|
sftp = c.open_sftp()
|
||||||
|
with sftp.open('/tmp/update_nas_db.py', 'w') as f:
|
||||||
|
f.write(py_db)
|
||||||
|
sftp.close()
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command("python3 /tmp/update_nas_db.py")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
err = stderr.read().decode()
|
||||||
|
if err: print("DB err:", err)
|
||||||
|
|
||||||
|
print("2. Generating Nginx configuration files for hosts 11 and 28...")
|
||||||
|
|
||||||
|
def make_dsm_conf(pid, domain, cert_id, host, port):
|
||||||
|
return f"""# ------------------------------------------------------------
|
||||||
|
# {domain}
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
|
map $scheme $hsts_header {{
|
||||||
|
https "max-age=63072000;includeSubDomains; preload";
|
||||||
|
}}
|
||||||
|
|
||||||
|
server {{
|
||||||
|
set $forward_scheme https;
|
||||||
|
set $server "{host}";
|
||||||
|
set $port {port};
|
||||||
|
|
||||||
|
listen 80;
|
||||||
|
listen [::]:80;
|
||||||
|
|
||||||
|
listen 443 ssl;
|
||||||
|
listen [::]:443 ssl;
|
||||||
|
|
||||||
|
server_name {domain};
|
||||||
|
http2 on;
|
||||||
|
|
||||||
|
# Let's Encrypt SSL
|
||||||
|
include conf.d/include/letsencrypt-acme-challenge.conf;
|
||||||
|
include conf.d/include/ssl-cache.conf;
|
||||||
|
include conf.d/include/ssl-ciphers.conf;
|
||||||
|
ssl_certificate /etc/letsencrypt/live/npm-{cert_id}/fullchain.pem;
|
||||||
|
ssl_certificate_key /etc/letsencrypt/live/npm-{cert_id}/privkey.pem;
|
||||||
|
|
||||||
|
# Asset Caching
|
||||||
|
include conf.d/include/assets.conf;
|
||||||
|
|
||||||
|
# Block Exploits
|
||||||
|
include conf.d/include/block-exploits.conf;
|
||||||
|
|
||||||
|
# HSTS
|
||||||
|
add_header Strict-Transport-Security $hsts_header always;
|
||||||
|
|
||||||
|
# Force SSL
|
||||||
|
set $trust_forwarded_proto "F";
|
||||||
|
include conf.d/include/force-ssl.conf;
|
||||||
|
|
||||||
|
proxy_set_header Upgrade $http_upgrade;
|
||||||
|
proxy_set_header Connection $http_connection;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
|
||||||
|
access_log /data/logs/proxy-host-{pid}_access.log proxy;
|
||||||
|
error_log /data/logs/proxy-host-{pid}_error.log warn;
|
||||||
|
|
||||||
|
location / {{
|
||||||
|
# HSTS
|
||||||
|
add_header Strict-Transport-Security $hsts_header always;
|
||||||
|
|
||||||
|
proxy_set_header Upgrade $http_upgrade;
|
||||||
|
proxy_set_header Connection $http_connection;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
|
||||||
|
# DSM Buffer Tuning
|
||||||
|
proxy_buffer_size 128k;
|
||||||
|
proxy_buffers 8 256k;
|
||||||
|
proxy_busy_buffers_size 256k;
|
||||||
|
proxy_read_timeout 3600s;
|
||||||
|
proxy_send_timeout 3600s;
|
||||||
|
proxy_ssl_verify off;
|
||||||
|
|
||||||
|
# Proxy!
|
||||||
|
include conf.d/include/proxy.conf;
|
||||||
|
}}
|
||||||
|
|
||||||
|
# Custom
|
||||||
|
include /data/nginx/custom/server_proxy[.]conf;
|
||||||
|
}}
|
||||||
|
"""
|
||||||
|
|
||||||
|
conf11 = make_dsm_conf(11, "sn-nas.snarfnet.net", 12, "14.10.10.71", 5001)
|
||||||
|
conf28 = make_dsm_conf(28, "snarfnas.snarfnet.net", 22, "14.10.10.72", 5001)
|
||||||
|
|
||||||
|
path11 = "/var/lib/docker/volumes/nginx-proxy-manager_nginx-data/_data/nginx/proxy_host/11.conf"
|
||||||
|
path28 = "/var/lib/docker/volumes/nginx-proxy-manager_nginx-data/_data/nginx/proxy_host/28.conf"
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command(f"cat << 'EOF' > {path11}\n{conf11}\nEOF\n")
|
||||||
|
stdout.channel.recv_exit_status()
|
||||||
|
print(f"Wrote {path11}")
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command(f"cat << 'EOF' > {path28}\n{conf28}\nEOF\n")
|
||||||
|
stdout.channel.recv_exit_status()
|
||||||
|
print(f"Wrote {path28}")
|
||||||
|
|
||||||
|
print("3. Testing Nginx configuration...")
|
||||||
|
stdin, stdout, stderr = c.exec_command("docker exec $(docker ps -q -f name=nginx-proxy-manager_app | head -n1) nginx -t")
|
||||||
|
out = stdout.read().decode()
|
||||||
|
err = stderr.read().decode()
|
||||||
|
print("nginx -t stdout:", out)
|
||||||
|
print("nginx -t stderr:", err)
|
||||||
|
|
||||||
|
if "successful" in out or "successful" in err:
|
||||||
|
print("4. Reloading Nginx Swarm service...")
|
||||||
|
stdin, stdout, stderr = c.exec_command("docker service update --force nginx-proxy-manager_app")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
print("Reload complete!")
|
||||||
|
else:
|
||||||
|
print("ERROR: Nginx config invalid!")
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
22
scripts/apply_web_vips.py
Normal file
22
scripts/apply_web_vips.py
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
import base64
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
|
||||||
|
def apply_manifest(host, file_path):
|
||||||
|
with open(file_path, "rb") as f:
|
||||||
|
b64 = base64.b64encode(f.read()).decode('ascii')
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect(host, username="root", key_filename=key_path, timeout=5)
|
||||||
|
stdin, stdout, stderr = c.exec_command(f"echo '{b64}' | base64 -d | k3s kubectl apply -f -")
|
||||||
|
print(f"Apply on {host}:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
c.close()
|
||||||
|
|
||||||
|
apply_manifest("140.44.4.11", r"C:\Users\Snarf\OneDrive\Desktop\code\SnarfCode\homelab-gitops-repo\manifests\aerodrome\dns\dns-stack.yaml")
|
||||||
|
apply_manifest("140.44.4.41", r"C:\Users\Snarf\OneDrive\Desktop\code\SnarfCode\homelab-gitops-repo\manifests\spaceport\dns\dns-stack.yaml")
|
||||||
|
|
||||||
30
scripts/audit_authentik.py
Normal file
30
scripts/audit_authentik.py
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.11", username="root", key_filename=key_path, timeout=5)
|
||||||
|
|
||||||
|
def run_query(title, q):
|
||||||
|
cmd = f"""k3s kubectl -n infra exec authentik-postgresql-0 -- bash -c "PGPASSWORD=\\$(cat /opt/bitnami/postgresql/secrets/password) psql -U authentik -d authentik -c \\"{q}\\"" """
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd)
|
||||||
|
print(f"=== {title} ===")
|
||||||
|
out = stdout.read().decode('utf-8')
|
||||||
|
err = stderr.read().decode('utf-8')
|
||||||
|
if out.strip(): print(out.strip())
|
||||||
|
if err.strip(): print("ERR:", err.strip())
|
||||||
|
print()
|
||||||
|
|
||||||
|
run_query("OUTPOST PROVIDERS", "SELECT outpost_id, provider_id FROM authentik_outposts_outpost_providers;")
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
c.close()
|
||||||
21
scripts/audit_cluster_workloads.py
Normal file
21
scripts/audit_cluster_workloads.py
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
def inspect_cluster(ip, name):
|
||||||
|
print(f"\n=======================================================")
|
||||||
|
print(f"=== {name} WORKLOADS ({ip}) ===")
|
||||||
|
print(f"=======================================================")
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect(ip, username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
cmd = "kubectl get deploy,sts,ds -A -o custom-columns=NAMESPACE:.metadata.namespace,KIND:.kind,NAME:.metadata.name,IMAGES:.spec.template.spec.containers[*].image"
|
||||||
|
_, out, _ = ssh.exec_command(cmd)
|
||||||
|
lines = out.read().decode().splitlines()
|
||||||
|
for l in lines:
|
||||||
|
if not any(k in l for k in ['kube-system', 'cattle-', 'metallb-system', 'longhorn-system']):
|
||||||
|
print(l)
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
|
inspect_cluster('140.44.4.11', 'AERODROME')
|
||||||
|
inspect_cluster('140.44.4.41', 'SPACEPORT')
|
||||||
|
|
||||||
40
scripts/audit_gitops_manifests.py
Normal file
40
scripts/audit_gitops_manifests.py
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
import os
|
||||||
|
import re
|
||||||
|
import paramiko
|
||||||
|
|
||||||
|
manifest_root = r"c:\Users\Snarf\OneDrive\Desktop\code\SnarfCode\homelab-gitops-repo\manifests"
|
||||||
|
|
||||||
|
print("=== GITOPS MANIFEST IMAGE AUDIT ===")
|
||||||
|
|
||||||
|
image_re = re.compile(r'image:\s*([^\s"\']+)', re.IGNORECASE)
|
||||||
|
chart_re = re.compile(r'chart:\s*([^\s"\']+)', re.IGNORECASE)
|
||||||
|
version_re = re.compile(r'version:\s*([^\s"\']+)', re.IGNORECASE)
|
||||||
|
|
||||||
|
manifest_data = {}
|
||||||
|
|
||||||
|
for root, dirs, files in os.walk(manifest_root):
|
||||||
|
for f in files:
|
||||||
|
if f.endswith('.yaml') or f.endswith('.yml'):
|
||||||
|
path = os.path.join(root, f)
|
||||||
|
rel_path = os.path.relpath(path, manifest_root)
|
||||||
|
with open(path, 'r', encoding='utf-8', errors='ignore') as fp:
|
||||||
|
content = fp.read()
|
||||||
|
images = image_re.findall(content)
|
||||||
|
charts = chart_re.findall(content)
|
||||||
|
versions = version_re.findall(content)
|
||||||
|
if images or charts:
|
||||||
|
manifest_data[rel_path] = {
|
||||||
|
'images': list(set(images)),
|
||||||
|
'charts': list(set(charts)),
|
||||||
|
'versions': list(set(versions))
|
||||||
|
}
|
||||||
|
|
||||||
|
for rel_path, data in sorted(manifest_data.items()):
|
||||||
|
print(f"\nFile: {rel_path}")
|
||||||
|
if data['images']:
|
||||||
|
print(" Images:", ", ".join(data['images']))
|
||||||
|
if data['charts']:
|
||||||
|
print(" Charts:", ", ".join(data['charts']))
|
||||||
|
if data['versions']:
|
||||||
|
print(" Versions:", ", ".join(data['versions']))
|
||||||
|
|
||||||
39
scripts/audit_spaceport_kernels.py
Normal file
39
scripts/audit_spaceport_kernels.py
Normal file
@@ -0,0 +1,39 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
spaceport_nodes = [
|
||||||
|
('sninfr0lnxctl01', '140.44.4.41'),
|
||||||
|
('sninfr0lnxctl02', '140.44.4.42'),
|
||||||
|
('sninfr0lnxctl03', '140.44.4.43'),
|
||||||
|
('sninfr0lnxctl04', '140.44.4.44'),
|
||||||
|
('sninfr0lnxwrk01', '140.44.4.51'),
|
||||||
|
('sninfr0lnxwrk02', '140.44.4.52'),
|
||||||
|
('sninfr0lnxwrk03', '140.44.4.53'),
|
||||||
|
('sninfr0lnxwrk04', '140.44.4.54'),
|
||||||
|
('sninfr0lnxwrk05', '140.44.4.55'),
|
||||||
|
('sninfr0lnxwrk06', '140.44.4.56'),
|
||||||
|
]
|
||||||
|
|
||||||
|
print(f"{'NODE':<18} {'IP':<15} {'RUNNING KERNEL':<22} {'LATEST MODULES':<25} {'VETH LOADED':<12} {'UPTIME'}")
|
||||||
|
print("="*105)
|
||||||
|
|
||||||
|
for name, ip in spaceport_nodes:
|
||||||
|
try:
|
||||||
|
client.connect(ip, username="root", key_filename=key_path, timeout=5)
|
||||||
|
stdin, stdout, stderr = client.exec_command("uname -r; ls -1 /lib/modules | sort -V | tail -n 1; lsmod | grep -q veth && echo YES || echo NO; uptime -p")
|
||||||
|
lines = [l.strip() for l in stdout.read().decode('utf-8').strip().split('\n') if l.strip()]
|
||||||
|
running_k = lines[0] if len(lines) > 0 else "?"
|
||||||
|
latest_mod = lines[1] if len(lines) > 1 else "?"
|
||||||
|
veth = lines[2] if len(lines) > 2 else "?"
|
||||||
|
uptime_str = lines[3] if len(lines) > 3 else "?"
|
||||||
|
print(f"{name:<18} {ip:<15} {running_k:<22} {latest_mod:<25} {veth:<12} {uptime_str}")
|
||||||
|
client.close()
|
||||||
|
except Exception as e:
|
||||||
|
print(f"{name:<18} {ip:<15} ERROR: {e}")
|
||||||
|
|
||||||
35
scripts/authorize_and_test.py
Normal file
35
scripts/authorize_and_test.py
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
pub_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6ycMke4AaV2qzWfULZCTZOKhSKf4pY2qRGYjYzs1MGKutJxt2jkYNK2aS0XffxLRR49v23zHbBwLYraNl2Mkb9rsKsyvKn8CAa1CEl9NxOci4LDRT8CeWHRnMZn8Mdu1pfeRwM26/Bjnd4RJrSKC7fyM0OlOpEp0DgnuPBqBtPMFGw+TCUT8eZ0pBrAI94wEan4l35kmuBcqm6RH3KuD1dDJKnPLzJ3/T21CViYL0GgFIDDor/hGI45VfOI/jKmkgM4YoudTCde6uPV5e2yaJpORZlR4nDxg9ntXp8HxbHGZkXYr55Ibsjem0lMU6nBh33jGbJVvat1Ay4oNCbygJsHfC/8ppe0bnLhnasxVAXhTvklZZL/sf7K/VS0ZifpfqB6DKtr1lziOm2wQeahv+Y1uoHkVNa3XY8Fx7xGPeT1BGd9lfcbUoweoTdMPMIPaSZlNGY32AHYkM4PShG9yI1coAqXzMby0+Z/Qwtr4l6cLKVvcGgqh5sd+z1+YvZbs= root@SNINFR2LNXCTL01"
|
||||||
|
|
||||||
|
# 1. Authorize on 140.44.4.71
|
||||||
|
c71 = paramiko.SSHClient()
|
||||||
|
c71.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c71.connect("140.44.4.71", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
cmd = f"""
|
||||||
|
mkdir -p ~/.ssh && chmod 700 ~/.ssh
|
||||||
|
grep -qF '{pub_key}' ~/.ssh/authorized_keys 2>/dev/null || echo '{pub_key}' >> ~/.ssh/authorized_keys
|
||||||
|
chmod 600 ~/.ssh/authorized_keys
|
||||||
|
"""
|
||||||
|
c71.exec_command(cmd)
|
||||||
|
c71.close()
|
||||||
|
print("Authorized 140.44.4.11 key on 140.44.4.71.")
|
||||||
|
|
||||||
|
# 2. Test SSH from 140.44.4.11 to 140.44.4.71
|
||||||
|
c11 = paramiko.SSHClient()
|
||||||
|
c11.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c11.connect("140.44.4.11", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c11.exec_command("ssh -o BatchMode=yes -o StrictHostKeyChecking=no root@140.44.4.71 hostname")
|
||||||
|
out = stdout.read().decode('utf-8').strip()
|
||||||
|
err = stderr.read().decode('utf-8').strip()
|
||||||
|
print("Result of ssh from 140.44.4.11 to 140.44.4.71:")
|
||||||
|
print("STDOUT:", out)
|
||||||
|
if err: print("STDERR:", err)
|
||||||
|
c11.close()
|
||||||
|
|
||||||
14
scripts/check_13_conf.py
Normal file
14
scripts/check_13_conf.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.71", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command('docker exec $(docker ps -q -f name=nginx-proxy-manager_app) cat /data/nginx/proxy_host/13.conf | grep server_name')
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
25
scripts/check_214_pods.py
Normal file
25
scripts/check_214_pods.py
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=key_path, timeout=15)
|
||||||
|
stdin, stdout, stderr = client.exec_command("KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n cattle-system get pods -o wide")
|
||||||
|
print("=== PODS IN CATTLE-SYSTEM ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
stdin, stdout, stderr = client.exec_command("KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n cattle-system rollout status deployment/rancher --timeout=30s")
|
||||||
|
print("=== ROLLOUT STATUS ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
print(stderr.read().decode('utf-8'))
|
||||||
|
|
||||||
|
stdin, stdout, stderr = client.exec_command("curl -k -I -s https://aerodrome.snarfnet.net | head -n 10")
|
||||||
|
print("=== ENDPOINT ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
client.close()
|
||||||
|
|
||||||
29
scripts/check_215_pods.py
Normal file
29
scripts/check_215_pods.py
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=key_path, timeout=15)
|
||||||
|
stdin, stdout, stderr = client.exec_command("KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n cattle-system get pods -o wide")
|
||||||
|
print("=== PODS IN CATTLE-SYSTEM ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
stdin, stdout, stderr = client.exec_command("KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n cattle-system rollout status deployment/rancher --timeout=30s")
|
||||||
|
print("=== ROLLOUT STATUS ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
print(stderr.read().decode('utf-8'))
|
||||||
|
|
||||||
|
stdin, stdout, stderr = client.exec_command("curl -k -I -s https://aerodrome.snarfnet.net | head -n 10")
|
||||||
|
print("=== ENDPOINT ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
stdin, stdout, stderr = client.exec_command("KUBECONFIG=/etc/rancher/k3s/k3s.yaml helm list -n cattle-system")
|
||||||
|
print("=== HELM LIST ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
client.close()
|
||||||
|
|
||||||
13
scripts/check_43_access.py
Normal file
13
scripts/check_43_access.py
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.71', username='snarf', key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("sudo docker exec $(sudo docker ps -q -f name=nginx-proxy-manager) tail -n 30 /data/logs/proxy-host-43_access.log")
|
||||||
|
print("=== Access log 43 (pihole2) around 11:00 ===")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
19
scripts/check_51.py
Normal file
19
scripts/check_51.py
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.51', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
def run(cmd):
|
||||||
|
print(f"=== {cmd} ===")
|
||||||
|
stdin, stdout, stderr = ssh.exec_command(cmd)
|
||||||
|
print(stdout.read().decode())
|
||||||
|
err = stderr.read().decode()
|
||||||
|
if err:
|
||||||
|
print("ERR:", err)
|
||||||
|
|
||||||
|
run("ss -tulpn | grep ':80 '")
|
||||||
|
run("ip a")
|
||||||
|
run("kubectl get pod -A -o wide | grep sninfr4lnxwrk01 | grep pihole || true")
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
18
scripts/check_51_nat.py
Normal file
18
scripts/check_51_nat.py
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.51', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
def run(cmd):
|
||||||
|
print(f"=== {cmd} ===")
|
||||||
|
stdin, stdout, stderr = ssh.exec_command(cmd)
|
||||||
|
print(stdout.read().decode())
|
||||||
|
err = stderr.read().decode()
|
||||||
|
if err:
|
||||||
|
print("ERR:", err)
|
||||||
|
|
||||||
|
run("iptables -t nat -S | grep -E '80\b|31905|31937'")
|
||||||
|
run("crictl ps")
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
17
scripts/check_55_and_43.py
Normal file
17
scripts/check_55_and_43.py
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.71', username='snarf', key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("sudo docker exec $(sudo docker ps -q -f name=nginx-proxy-manager) cat /data/nginx/proxy_host/55.conf")
|
||||||
|
print("=== 55.conf (pihole1) ===")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("sudo docker exec $(sudo docker ps -q -f name=nginx-proxy-manager) cat /data/nginx/proxy_host/43.conf")
|
||||||
|
print("=== 43.conf (pihole2) ===")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
14
scripts/check_71_containers.py
Normal file
14
scripts/check_71_containers.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.71", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("docker ps --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}'")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
15
scripts/check_aero_health.py
Normal file
15
scripts/check_aero_health.py
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = client.exec_command('k3s kubectl get pods -n cattle-system')
|
||||||
|
print("=== AERODROME CATTLE-SYSTEM PODS ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
14
scripts/check_aero_infra.py
Normal file
14
scripts/check_aero_infra.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.11", username="root", key_filename=key_path, timeout=5)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command("k3s kubectl -n infra get deploy,svc,pod")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
c.close()
|
||||||
|
|
||||||
12
scripts/check_aero_pihole_auth.py
Normal file
12
scripts/check_aero_pihole_auth.py
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.11', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command('k3s kubectl exec -n dns deployment/pihole -- curl -s -X POST "http://localhost/api/auth" -H "Content-Type: application/json" -d \'{"password":"snarfnetadmin"}\'')
|
||||||
|
print("Aerodrome API auth test:")
|
||||||
|
print(stdout.read().decode('utf-8', errors='ignore'))
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
14
scripts/check_aero_svc.py
Normal file
14
scripts/check_aero_svc.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.11", username="root", key_filename=key_path, timeout=5)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command("k3s kubectl -n infra get svc")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
c.close()
|
||||||
|
|
||||||
15
scripts/check_aerodrome_pods.py
Normal file
15
scripts/check_aerodrome_pods.py
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
s = paramiko.SSHClient()
|
||||||
|
s.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
s.connect('140.44.4.11', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
def run(cmd):
|
||||||
|
print(f"=== {cmd} ===")
|
||||||
|
_, out, _ = s.exec_command(cmd)
|
||||||
|
print(out.read().decode())
|
||||||
|
|
||||||
|
run("kubectl get pods -A | grep -v -E 'Running|Completed' || true")
|
||||||
|
run("kubectl get volumes.longhorn.io -n longhorn-system -o custom-columns=NAME:.metadata.name,ROBUSTNESS:.status.robustness,STATE:.status.state")
|
||||||
|
s.close()
|
||||||
|
|
||||||
19
scripts/check_aerodrome_preflight.py
Normal file
19
scripts/check_aerodrome_preflight.py
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.11', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
def run(cmd):
|
||||||
|
print(f"=== {cmd} ===")
|
||||||
|
stdin, stdout, stderr = ssh.exec_command(cmd)
|
||||||
|
print(stdout.read().decode())
|
||||||
|
err = stderr.read().decode()
|
||||||
|
if err:
|
||||||
|
print("ERR:", err)
|
||||||
|
|
||||||
|
run("kubectl get nodes")
|
||||||
|
run("kubectl get pods -A | grep -v -E 'Running|Completed' || true")
|
||||||
|
run("kubectl get volumes.longhorn.io -n longhorn-system -o custom-columns=NAME:.metadata.name,ROBUSTNESS:.status.robustness | grep -v healthy || true")
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
8
scripts/check_aerodrome_quick.py
Normal file
8
scripts/check_aerodrome_quick.py
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
import paramiko
|
||||||
|
ssh=paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.11', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
_,o,_=ssh.exec_command('kubectl get nodes')
|
||||||
|
print(o.read().decode())
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
29
scripts/check_after_reboot.py
Normal file
29
scripts/check_after_reboot.py
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
print("=== CHECKING VETH ON SNINFR0LNXCTL03 ===")
|
||||||
|
client.connect("140.44.4.43", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("modprobe veth; lsmod | grep veth; systemctl is-active k3s-server || systemctl is-active k3s")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
|
time.sleep(5)
|
||||||
|
|
||||||
|
print("=== CHECKING PODS IN CATTLE-SYSTEM ON SPACEPORT ===")
|
||||||
|
client.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl -n cattle-system get pods -o wide")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
print("=== CHECKING POD EVENTS ===")
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl -n cattle-system describe pod -l app=cattle-cluster-agent | grep -A 10 Events:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
client.close()
|
||||||
|
|
||||||
14
scripts/check_agent_deploy.py
Normal file
14
scripts/check_agent_deploy.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl -n cattle-system get deployment cattle-cluster-agent -o yaml")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
15
scripts/check_agent_logs.py
Normal file
15
scripts/check_agent_logs.py
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl -n cattle-system logs -l app=cattle-cluster-agent --tail=100")
|
||||||
|
print("=== SPACEPORT CATTLE-CLUSTER-AGENT LOGS ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
15
scripts/check_agent_logs_tail.py
Normal file
15
scripts/check_agent_logs_tail.py
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl -n cattle-system logs -l app=cattle-cluster-agent -c cluster-register --tail=20")
|
||||||
|
print("=== AGENT LOGS ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
14
scripts/check_agent_now.py
Normal file
14
scripts/check_agent_now.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl -n cattle-system logs -l app=cattle-cluster-agent -c cluster-register --tail=25")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
24
scripts/check_all_coder_snapshots.py
Normal file
24
scripts/check_all_coder_snapshots.py
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
nodes = [
|
||||||
|
("wrk01", "140.44.4.51"),
|
||||||
|
("wrk02", "140.44.4.52"),
|
||||||
|
("wrk03", "140.44.4.53"),
|
||||||
|
("wrk04", "140.44.4.54"),
|
||||||
|
("wrk05", "140.44.4.55"),
|
||||||
|
("wrk06", "140.44.4.56"),
|
||||||
|
]
|
||||||
|
|
||||||
|
for name, ip in nodes:
|
||||||
|
try:
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect(ip, username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=4)
|
||||||
|
_, out, _ = ssh.exec_command("find /var/lib/rancher/k3s/agent/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/ -path '*/home/coder/.config/coderv2' 2>/dev/null")
|
||||||
|
res = out.read().decode().strip()
|
||||||
|
if res:
|
||||||
|
print(f"[{name} - {ip}] Found:\n{res}")
|
||||||
|
ssh.close()
|
||||||
|
except Exception as e:
|
||||||
|
print(f"[{name}] err: {e}")
|
||||||
|
|
||||||
19
scripts/check_all_endpoints.py
Normal file
19
scripts/check_all_endpoints.py
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
import subprocess
|
||||||
|
|
||||||
|
endpoints = [
|
||||||
|
("Argo CD", "https://argo.snarfnet.net"),
|
||||||
|
("Rancher Manager", "https://aerodrome.snarfnet.net"),
|
||||||
|
("AWX Console", "https://awx.snarfnet.net"),
|
||||||
|
("Authentik SSO", "https://authentik.snarfnet.net"),
|
||||||
|
("Uptime Kuma", "https://uptime.snarfnet.net/dashboard"),
|
||||||
|
("Gitea GitOps", "http://140.44.4.51:90"),
|
||||||
|
("Zabbix Server", "http://140.44.4.51:8888"),
|
||||||
|
("Portainer Server", "https://140.44.4.71:9443")
|
||||||
|
]
|
||||||
|
|
||||||
|
print("=== ENDPOINT HEALTH CHECK ===")
|
||||||
|
for name, url in endpoints:
|
||||||
|
res = subprocess.run(f"curl.exe -k -s -o nul -w '%{{http_code}}' {url}", capture_output=True, text=True, shell=True)
|
||||||
|
code = res.stdout.strip()
|
||||||
|
print(f"[{code}] {name} ({url})")
|
||||||
|
|
||||||
25
scripts/check_all_npm_hosts.py
Normal file
25
scripts/check_all_npm_hosts.py
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.71", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
py_query = """
|
||||||
|
import sqlite3
|
||||||
|
conn = sqlite3.connect('/var/lib/docker/volumes/nginx-proxy-manager_nginx-data/_data/database.sqlite')
|
||||||
|
cur = conn.cursor()
|
||||||
|
cur.execute('SELECT id, domain_names, forward_host, forward_port, forward_scheme, enabled FROM proxy_host')
|
||||||
|
for r in cur.fetchall():
|
||||||
|
print(r)
|
||||||
|
conn.close()
|
||||||
|
"""
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command(f'python3 -c "{py_query}"')
|
||||||
|
print("All NPM hosts:\n", stdout.read().decode())
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
30
scripts/check_all_providers.py
Normal file
30
scripts/check_all_providers.py
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
import urllib.request
|
||||||
|
import ssl
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
ctx = ssl.create_default_context()
|
||||||
|
ctx.check_hostname = False
|
||||||
|
ctx.verify_mode = ssl.CERT_NONE
|
||||||
|
|
||||||
|
token = "SnarfNetPermanentAutomationToken2026"
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Bearer {token}",
|
||||||
|
"Content-Type": "application/json"
|
||||||
|
}
|
||||||
|
|
||||||
|
def api_call(url):
|
||||||
|
req = urllib.request.Request(url, headers=headers)
|
||||||
|
with urllib.request.urlopen(req, context=ctx, timeout=8) as resp:
|
||||||
|
return json.loads(resp.read().decode('utf-8'))
|
||||||
|
|
||||||
|
apps = api_call("https://authentik.snarfnet.net/api/v3/core/applications/?page_size=100")
|
||||||
|
for a in apps.get('results', []):
|
||||||
|
print(f"App: {a['name']:30} | Slug: {a['slug']:15} | Provider: {a.get('provider')}")
|
||||||
|
|
||||||
|
providers = api_call("https://authentik.snarfnet.net/api/v3/providers/proxy/?page_size=100")
|
||||||
|
for p in providers.get('results', []):
|
||||||
|
print(f"Proxy Provider: {p['name']:30} | PK: {p['pk']:4} | Mode: {p['mode']:15} | Cookie Domain: {p.get('cookie_domain')}")
|
||||||
|
|
||||||
27
scripts/check_all_tls.py
Normal file
27
scripts/check_all_tls.py
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
# Check certificates in Aerodrome across all namespaces
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("""
|
||||||
|
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl get secrets -A --field-selector=type=kubernetes.io/tls
|
||||||
|
""")
|
||||||
|
print("=== TLS SECRETS ON AERODROME ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
|
# Check certificates on Spaceport across all namespaces
|
||||||
|
client.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("""
|
||||||
|
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl get secrets -A --field-selector=type=kubernetes.io/tls
|
||||||
|
""")
|
||||||
|
print("=== TLS SECRETS ON SPACEPORT ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
12
scripts/check_api_status.py
Normal file
12
scripts/check_api_status.py
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.11', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command('k3s kubectl exec -n dns deployment/technitium-dns -- curl -s "http://localhost:5380/api/status"')
|
||||||
|
print("api/status response:")
|
||||||
|
print(stdout.read().decode('utf-8', errors='ignore'))
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
25
scripts/check_apis.py
Normal file
25
scripts/check_apis.py
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=key_path, timeout=15)
|
||||||
|
|
||||||
|
test_cmd = """
|
||||||
|
cd /tmp/rancher-test
|
||||||
|
KUBECONFIG=/etc/rancher/k3s/k3s.yaml helm template rancher ./rancher --namespace cattle-system --set hostname=aerodrome.snarfnet.net --set ingress.tls.source=secret | grep "apiVersion:" | sort | uniq -c
|
||||||
|
"""
|
||||||
|
|
||||||
|
stdin, stdout, stderr = client.exec_command(test_cmd)
|
||||||
|
out = stdout.read().decode('utf-8')
|
||||||
|
err = stderr.read().decode('utf-8')
|
||||||
|
print("APIs used:")
|
||||||
|
print(out)
|
||||||
|
if err:
|
||||||
|
print("STDERR:", err)
|
||||||
|
|
||||||
|
client.close()
|
||||||
12
scripts/check_apiurl.py
Normal file
12
scripts/check_apiurl.py
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.41', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command('k3s kubectl exec -n dns deployment/pihole -- curl -s "http://localhost/admin/login" | grep -i "data-apiurl"')
|
||||||
|
print("data-apiurl:")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
17
scripts/check_app_dll.py
Normal file
17
scripts/check_app_dll.py
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.11', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command('k3s kubectl exec -n dns deployment/technitium-dns -- ls -la /app')
|
||||||
|
print("/app contents:")
|
||||||
|
print(stdout.read().decode('utf-8', errors='ignore'))
|
||||||
|
|
||||||
|
# Search for AddAuthentication in dlls
|
||||||
|
stdin, stdout, stderr = ssh.exec_command('k3s kubectl exec -n dns deployment/technitium-dns -- strings /app/DnsServerApp.dll /app/DnsServerCore.dll 2>/dev/null | grep -i -C 3 "AddAuthentication"')
|
||||||
|
print("AddAuthentication strings:")
|
||||||
|
print(stdout.read().decode('utf-8', errors='ignore'))
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
14
scripts/check_argo_apps.py
Normal file
14
scripts/check_argo_apps.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.41', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("kubectl get applications -A -o custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name,SYNC:.status.sync.status,HEALTH:.status.health.status")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
err = stderr.read().decode()
|
||||||
|
if err:
|
||||||
|
print("ERR:", err)
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
22
scripts/check_argo_ing.py
Normal file
22
scripts/check_argo_ing.py
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.41', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
def run(cmd):
|
||||||
|
print(f"=== {cmd} ===")
|
||||||
|
stdin, stdout, stderr = ssh.exec_command(cmd)
|
||||||
|
print(stdout.read().decode())
|
||||||
|
err = stderr.read().decode()
|
||||||
|
if err:
|
||||||
|
print("ERR:", err)
|
||||||
|
|
||||||
|
run("kubectl get ingress argo-ing -n infrastructure -o yaml")
|
||||||
|
run("kubectl get svc pihole-web -n infrastructure -o yaml")
|
||||||
|
run("kubectl get svc -n infrastructure")
|
||||||
|
run("kubectl get svc -A | grep -E 'LoadBalancer|140.44.4.62'")
|
||||||
|
run("curl -k -I -H 'Host: argo.snarfnet.net' http://140.44.4.62")
|
||||||
|
run("curl -k -I -H 'Host: argo.snarfnet.net' https://140.44.4.62")
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
11
scripts/check_argocd_now.py
Normal file
11
scripts/check_argocd_now.py
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.41', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
_, out, _ = ssh.exec_command("kubectl get applications -n infrastructure -o custom-columns=NAME:.metadata.name,SYNC:.status.sync.status,HEALTH:.status.health.status")
|
||||||
|
print(out.read().decode())
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
26
scripts/check_auth_links.py
Normal file
26
scripts/check_auth_links.py
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.11", username="root", key_filename=key_path, timeout=5)
|
||||||
|
|
||||||
|
cmd = """
|
||||||
|
k3s kubectl -n infra exec authentik-postgresql-0 -- bash -c "PGPASSWORD=\\$(cat /opt/bitnami/postgresql/secrets/password) psql -U authentik -d authentik -c \\"SELECT provider_id, application_id FROM authentik_core_application_backchannel_providers;\\""
|
||||||
|
"""
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd)
|
||||||
|
print("Backchannel Providers:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
cmd2 = """
|
||||||
|
k3s kubectl -n infra exec authentik-postgresql-0 -- bash -c "PGPASSWORD=\\$(cat /opt/bitnami/postgresql/secrets/password) psql -U authentik -d authentik -c \\"SELECT * FROM authentik_providers_proxy_proxyprovider;\\""
|
||||||
|
"""
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd2)
|
||||||
|
print("Proxy Providers:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
27
scripts/check_auth_tables.py
Normal file
27
scripts/check_auth_tables.py
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.11", username="root", key_filename=key_path, timeout=5)
|
||||||
|
|
||||||
|
cmd = "k3s kubectl -n infra exec authentik-postgresql-0 -- bash -c 'PGPASSWORD=$(cat /opt/bitnami/postgresql/secrets/password) psql -U authentik -d authentik -c \"SELECT name, slug FROM authentik_core_application;\"'"
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd)
|
||||||
|
print("Apps:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
cmd2 = "k3s kubectl -n infra exec authentik-postgresql-0 -- bash -c 'PGPASSWORD=$(cat /opt/bitnami/postgresql/secrets/password) psql -U authentik -d authentik -c \"SELECT name, external_host FROM authentik_providers_proxy_proxyprovider;\"'"
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd2)
|
||||||
|
print("Proxy Providers:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
cmd3 = "k3s kubectl -n infra exec authentik-postgresql-0 -- bash -c 'PGPASSWORD=$(cat /opt/bitnami/postgresql/secrets/password) psql -U authentik -d authentik -c \"SELECT name, client_id FROM authentik_providers_oauth2_oauth2provider;\"'"
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd3)
|
||||||
|
print("OAuth2/OIDC Providers:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
27
scripts/check_authentik_apps.py
Normal file
27
scripts/check_authentik_apps.py
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.11", username="root", key_filename=key_path, timeout=5)
|
||||||
|
|
||||||
|
cmd = "k3s kubectl -n infra exec authentik-postgresql-0 -- psql -U authentik -d authentik -c 'SELECT id, name, authorization_flow_id FROM authentik_core_application;'"
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd)
|
||||||
|
print("Authentik Applications:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
cmd2 = "k3s kubectl -n infra exec authentik-postgresql-0 -- psql -U authentik -d authentik -c 'SELECT id, name, external_host FROM authentik_providers_proxy_proxyprovider;'"
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd2)
|
||||||
|
print("Authentik Proxy Providers:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
cmd3 = "k3s kubectl -n infra exec authentik-postgresql-0 -- psql -U authentik -d authentik -c 'SELECT id, name, client_id FROM authentik_providers_oauth2_oauth2provider;'"
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd3)
|
||||||
|
print("Authentik OAuth2/OIDC Providers:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
31
scripts/check_authentik_apps_full.py
Normal file
31
scripts/check_authentik_apps_full.py
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
import urllib.request
|
||||||
|
import ssl
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
ctx = ssl.create_default_context()
|
||||||
|
ctx.check_hostname = False
|
||||||
|
ctx.verify_mode = ssl.CERT_NONE
|
||||||
|
|
||||||
|
token = "SnarfNetPermanentAutomationToken2026"
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Bearer {token}",
|
||||||
|
"Content-Type": "application/json"
|
||||||
|
}
|
||||||
|
|
||||||
|
req = urllib.request.Request("https://authentik.snarfnet.net/api/v3/core/applications/?page_size=100", headers=headers)
|
||||||
|
with urllib.request.urlopen(req, context=ctx) as resp:
|
||||||
|
data = json.loads(resp.read().decode())
|
||||||
|
print(f"Total Applications: {len(data['results'])}")
|
||||||
|
for app in sorted(data['results'], key=lambda x: x['name']):
|
||||||
|
slug = app['slug']
|
||||||
|
if any(k in slug for k in ['dns', 'pihole', 'technitium', 'syno', 'nas', 'semaphore']):
|
||||||
|
print(f"- Name: {app['name']}")
|
||||||
|
print(f" Slug: {app['slug']}")
|
||||||
|
print(f" Group: {app.get('group')}")
|
||||||
|
print(f" Launch URL: {app.get('meta_launch_url')}")
|
||||||
|
print(f" Icon: {app.get('meta_icon')}")
|
||||||
|
print()
|
||||||
|
|
||||||
36
scripts/check_authentik_events.py
Normal file
36
scripts/check_authentik_events.py
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
import urllib.request
|
||||||
|
import ssl
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
ctx = ssl.create_default_context()
|
||||||
|
ctx.check_hostname = False
|
||||||
|
ctx.verify_mode = ssl.CERT_NONE
|
||||||
|
|
||||||
|
AK_BASE = "https://authentik.snarfnet.net"
|
||||||
|
AK_TOKEN = "SnarfNetPermanentAutomationToken2026"
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Bearer {AK_TOKEN}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"Accept": "application/json"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Fetch recent events
|
||||||
|
req = urllib.request.Request(f"{AK_BASE}/api/v3/events/events/?page_size=20", headers=headers)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req, timeout=10, context=ctx) as r:
|
||||||
|
data = json.loads(r.read().decode())
|
||||||
|
for ev in data.get("results", []):
|
||||||
|
action = ev.get("action")
|
||||||
|
user = ev.get("user", {}).get("username")
|
||||||
|
context = ev.get("context", {})
|
||||||
|
created = ev.get("created")
|
||||||
|
print(f"[{created}] Action: {action} | User: {user}")
|
||||||
|
if "message" in context or "redirect_uri" in str(context) or "error" in str(context) or "client_id" in str(context):
|
||||||
|
print(f" Context: {json.dumps(context, indent=2)}")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"Error querying Authentik: {e}")
|
||||||
|
|
||||||
20
scripts/check_authentik_ingress.py
Normal file
20
scripts/check_authentik_ingress.py
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.11", username="root", key_filename=key_path, timeout=5)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command("k3s kubectl get middleware.traefik.io,middleware.traefik.containo.us -A 2>/dev/null")
|
||||||
|
print("=== TRAEFIK MIDDLEWARES ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command("k3s kubectl get ingress -A")
|
||||||
|
print("=== ALL INGRESSES ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
19
scripts/check_authentik_logs.py
Normal file
19
scripts/check_authentik_logs.py
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.11', username='root', key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
# Check Authentik server logs
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("k3s kubectl logs -n infra deployment/authentik-server --tail=50")
|
||||||
|
print("Authentik server logs:")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
# Check Authentik outpost logs if any
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("k3s kubectl get pods -n infra")
|
||||||
|
print("Infra pods on Aerodrome:")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
20
scripts/check_authentik_outposts.py
Normal file
20
scripts/check_authentik_outposts.py
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
import urllib.request
|
||||||
|
import ssl
|
||||||
|
import json
|
||||||
|
|
||||||
|
ctx = ssl.create_default_context()
|
||||||
|
ctx.check_hostname = False
|
||||||
|
ctx.verify_mode = ssl.CERT_NONE
|
||||||
|
|
||||||
|
token = "SnarfNetPermanentAutomationToken2026"
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Bearer {token}",
|
||||||
|
"Content-Type": "application/json"
|
||||||
|
}
|
||||||
|
|
||||||
|
req = urllib.request.Request("https://authentik.snarfnet.net/api/v3/outposts/instances/", headers=headers)
|
||||||
|
with urllib.request.urlopen(req, context=ctx) as resp:
|
||||||
|
data = json.loads(resp.read().decode())
|
||||||
|
for o in data['results']:
|
||||||
|
print("Outpost:", o['name'], o['type'], o.get('providers'))
|
||||||
|
|
||||||
26
scripts/check_authentik_semaphore.py
Normal file
26
scripts/check_authentik_semaphore.py
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
import urllib.request
|
||||||
|
import ssl
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
ctx = ssl.create_default_context()
|
||||||
|
ctx.check_hostname = False
|
||||||
|
ctx.verify_mode = ssl.CERT_NONE
|
||||||
|
|
||||||
|
token = "SnarfNetPermanentAutomationToken2026"
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Bearer {token}",
|
||||||
|
"Content-Type": "application/json"
|
||||||
|
}
|
||||||
|
|
||||||
|
req = urllib.request.Request("https://authentik.snarfnet.net/api/v3/providers/oauth2/", headers=headers)
|
||||||
|
with urllib.request.urlopen(req, context=ctx) as resp:
|
||||||
|
data = json.loads(resp.read().decode())
|
||||||
|
for p in data['results']:
|
||||||
|
if 'semaphore' in p['name'].lower():
|
||||||
|
print(f"Provider ID: {p['pk']}, Name: {p['name']}")
|
||||||
|
print(f"Redirect URIs: {p.get('redirect_uris')}")
|
||||||
|
print(f"Client ID: {p.get('client_id')}")
|
||||||
|
|
||||||
20
scripts/check_authentik_users.py
Normal file
20
scripts/check_authentik_users.py
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
import urllib.request
|
||||||
|
import ssl
|
||||||
|
import json
|
||||||
|
|
||||||
|
ctx = ssl.create_default_context()
|
||||||
|
ctx.check_hostname = False
|
||||||
|
ctx.verify_mode = ssl.CERT_NONE
|
||||||
|
|
||||||
|
token = "SnarfNetPermanentAutomationToken2026"
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Bearer {token}",
|
||||||
|
"Content-Type": "application/json"
|
||||||
|
}
|
||||||
|
|
||||||
|
req = urllib.request.Request("https://authentik.snarfnet.net/api/v3/core/users/?page_size=20", headers=headers)
|
||||||
|
with urllib.request.urlopen(req, context=ctx) as resp:
|
||||||
|
data = json.loads(resp.read().decode())
|
||||||
|
for u in data['results']:
|
||||||
|
print(f"User: {u['username']}, Email: {u.get('email')}, Name: {u.get('name')}")
|
||||||
|
|
||||||
14
scripts/check_blkid.py
Normal file
14
scripts/check_blkid.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.43", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("blkid")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
14
scripts/check_boot_ctl03.py
Normal file
14
scripts/check_boot_ctl03.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.43", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("ls -la /boot; ls -la /boot/firmware 2>/dev/null | head -n 20")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
15
scripts/check_bridge_logs.py
Normal file
15
scripts/check_bridge_logs.py
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.71", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command("docker service logs --tail 10 gotify_gotify-discord-bridge")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
15
scripts/check_bridge_map.py
Normal file
15
scripts/check_bridge_map.py
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.71", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command("grep -n -A 10 'APP_WEBHOOK_MAP' /var/lib/docker/volumes/gotify_gotify_data/_data/discord_bridge.py")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
20
scripts/check_cattle_all.py
Normal file
20
scripts/check_cattle_all.py
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl -n cattle-system get pods -o wide")
|
||||||
|
print("=== SPACEPORT CATTLE-SYSTEM PODS ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl -n cattle-system get deployment cattle-cluster-agent")
|
||||||
|
print("=== DEPLOYMENT STATUS ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
client.close()
|
||||||
|
|
||||||
14
scripts/check_cattle_pods.py
Normal file
14
scripts/check_cattle_pods.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
def check_cattle(ip, cluster):
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect(ip, username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
print(f"\n[{cluster}] Checking cattle-system pods...")
|
||||||
|
_, out, _ = ssh.exec_command("kubectl get pods -n cattle-system -o wide")
|
||||||
|
print(out.read().decode().strip())
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
|
check_cattle('140.44.4.11', 'AERODROME')
|
||||||
|
check_cattle('140.44.4.41', 'SPACEPORT')
|
||||||
|
|
||||||
29
scripts/check_cattle_spaceport.py
Normal file
29
scripts/check_cattle_spaceport.py
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
cmds = [
|
||||||
|
"k3s kubectl -n cattle-system get pods -o wide",
|
||||||
|
"k3s kubectl -n cattle-system describe pod -l app=cattle-cluster-agent",
|
||||||
|
"k3s kubectl get events -A --sort-by='.lastTimestamp' | tail -n 20"
|
||||||
|
]
|
||||||
|
|
||||||
|
for cmd in cmds:
|
||||||
|
print(f"\n==========================================")
|
||||||
|
print(f">>> Running on Spaceport: {cmd}")
|
||||||
|
print(f"==========================================")
|
||||||
|
stdin, stdout, stderr = client.exec_command(cmd)
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
err = stderr.read().decode('utf-8')
|
||||||
|
if err:
|
||||||
|
print("STDERR:", err)
|
||||||
|
|
||||||
|
client.close()
|
||||||
|
|
||||||
14
scripts/check_cattle_spaceport_clean.py
Normal file
14
scripts/check_cattle_spaceport_clean.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl -n cattle-system get pods -o wide")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
14
scripts/check_cert1.py
Normal file
14
scripts/check_cert1.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.71", username="root", key_filename=key_path, timeout=5)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command("ls -la /var/lib/docker/volumes/nginx-proxy-manager_nginx-letsencrypt/_data/live/npm-1/")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
c.close()
|
||||||
|
|
||||||
21
scripts/check_cert_san.py
Normal file
21
scripts/check_cert_san.py
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.71", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
def check_cert(num):
|
||||||
|
cmd = f"docker exec $(docker ps -q -f name=nginx-proxy-manager_app | head -n1) openssl x509 -in /etc/letsencrypt/live/{num}/fullchain.pem -noout -text | grep -A 2 'Subject Alternative Name'"
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd)
|
||||||
|
print(f"Cert {num}:", stdout.read().decode().strip())
|
||||||
|
|
||||||
|
check_cert("npm-1")
|
||||||
|
check_cert("npm-33")
|
||||||
|
check_cert("npm-37")
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
15
scripts/check_certbot.py
Normal file
15
scripts/check_certbot.py
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
client.connect("140.44.4.71", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = client.exec_command("docker exec $(docker ps -q -f name=nginx-proxy-manager_app) certbot certificates")
|
||||||
|
print("OUT:", stdout.read().decode('utf-8'))
|
||||||
|
print("ERR:", stderr.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
22
scripts/check_chart_kubeversions.py
Normal file
22
scripts/check_chart_kubeversions.py
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
|
||||||
|
versions = [
|
||||||
|
"2.7.10", "2.8.5", "2.9.3", "2.10.3", "2.11.3", "2.12.3", "2.13.3", "2.14.3", "2.15.3"
|
||||||
|
]
|
||||||
|
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=key_path, timeout=15)
|
||||||
|
|
||||||
|
for ver in versions:
|
||||||
|
stdin, stdout, stderr = client.exec_command(f"KUBECONFIG=/etc/rancher/k3s/k3s.yaml helm show chart rancher-stable/rancher --version {ver} | grep kubeVersion")
|
||||||
|
out = stdout.read().decode('utf-8').strip()
|
||||||
|
print(f"Rancher {ver}: {out}")
|
||||||
|
|
||||||
|
client.close()
|
||||||
|
|
||||||
13
scripts/check_chart_versions.py
Normal file
13
scripts/check_chart_versions.py
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=key_path, timeout=15)
|
||||||
|
stdin, stdout, stderr = client.exec_command("KUBECONFIG=/etc/rancher/k3s/k3s.yaml helm search repo rancher-stable/rancher --versions > /tmp/rancher_versions.txt && head -n 30 /tmp/rancher_versions.txt")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
14
scripts/check_cluster_cond.py
Normal file
14
scripts/check_cluster_cond.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = client.exec_command('k3s kubectl get clusters.provisioning.cattle.io -A')
|
||||||
|
print("=== PROVISIONING CLUSTERS ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
13
scripts/check_cluster_details.py
Normal file
13
scripts/check_cluster_details.py
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=key_path, timeout=15)
|
||||||
|
stdin, stdout, stderr = client.exec_command('KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl get clusters.management.cattle.io -o wide')
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
16
scripts/check_cluster_ready.py
Normal file
16
scripts/check_cluster_ready.py
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("""
|
||||||
|
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl get clusters.management.cattle.io -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.displayName}{"\tConnected: "}{.status.agentImage}{"\tReady: "}{range .status.conditions[?(@.type=="Ready")]}{.status}{" - "}{.message}{end}{"\n"}{end}'
|
||||||
|
""")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
25
scripts/check_cluster_tokens.py
Normal file
25
scripts/check_cluster_tokens.py
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl get clusterregistrationtokens.management.cattle.io -A -o yaml")
|
||||||
|
data = yaml.safe_load(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
|
for item in data.get('items', []):
|
||||||
|
cluster = item.get('spec', {}).get('clusterName')
|
||||||
|
token = item.get('status', {}).get('token')
|
||||||
|
cmd = item.get('status', {}).get('command')
|
||||||
|
manifest_url = item.get('status', {}).get('manifestUrl')
|
||||||
|
print(f"\nCluster: {cluster}")
|
||||||
|
print(f"Token: {token}")
|
||||||
|
print(f"Command: {cmd}")
|
||||||
|
print(f"Manifest URL: {manifest_url}")
|
||||||
|
|
||||||
26
scripts/check_cluster_yaml.py
Normal file
26
scripts/check_cluster_yaml.py
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl get clusters.management.cattle.io -o yaml")
|
||||||
|
data = yaml.safe_load(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
|
for item in data.get('items', []):
|
||||||
|
name = item.get('metadata', {}).get('name')
|
||||||
|
disp = item.get('spec', {}).get('displayName')
|
||||||
|
print(f"\n==========================================")
|
||||||
|
print(f"Cluster: {disp} ({name})")
|
||||||
|
print(f"==========================================")
|
||||||
|
conditions = item.get('status', {}).get('conditions', [])
|
||||||
|
for c in conditions:
|
||||||
|
if c.get('status') != 'True' or c.get('type') in ['Ready', 'Connected']:
|
||||||
|
print(f" [{c.get('type')}]: status={c.get('status')} reason={c.get('reason')} msg={c.get('message')}")
|
||||||
|
|
||||||
20
scripts/check_cmdline.py
Normal file
20
scripts/check_cmdline.py
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.43", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("cat /proc/cmdline; dmesg | grep -i 'Kernel command line'")
|
||||||
|
print("CMDLINE:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
stdin, stdout, stderr = client.exec_command("ls -la /dev/disk/by-id/; ls -la /dev/disk/by-uuid/")
|
||||||
|
print("DISKS:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
client.close()
|
||||||
|
|
||||||
11
scripts/check_coder_logs.py
Normal file
11
scripts/check_coder_logs.py
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.41', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
_, out, _ = ssh.exec_command('kubectl logs -n infrastructure -l app.kubernetes.io/name=coder --tail=30')
|
||||||
|
print(out.read().decode())
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
14
scripts/check_coder_page.py
Normal file
14
scripts/check_coder_page.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.41', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
_, out, _ = ssh.exec_command("curl -s https://coder.snarfnet.net/api/v2/users/first")
|
||||||
|
print("First user status:", out.read().decode())
|
||||||
|
|
||||||
|
_, out2, _ = ssh.exec_command("curl -s https://coder.snarfnet.net/api/v2/experiments")
|
||||||
|
print("Experiments:", out2.read().decode())
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
14
scripts/check_coredns.py
Normal file
14
scripts/check_coredns.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl -n kube-system get configmap coredns -o yaml")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
14
scripts/check_coredns_logs.py
Normal file
14
scripts/check_coredns_logs.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl -n kube-system logs -l k8s-app=kube-dns --tail=30")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
18
scripts/check_ctl02_uptime.py
Normal file
18
scripts/check_ctl02_uptime.py
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
try:
|
||||||
|
client.connect("140.44.4.42", username="root", key_filename=key_path, timeout=5)
|
||||||
|
stdin, stdout, stderr = client.exec_command("uptime; uname -r; modprobe veth; lsmod | grep veth")
|
||||||
|
print("OUTPUT from ctl02:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
except Exception as e:
|
||||||
|
print("Connect error:", e)
|
||||||
|
|
||||||
33
scripts/check_ctl03.py
Normal file
33
scripts/check_ctl03.py
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
node_ip = "140.44.4.43"
|
||||||
|
print(f"Connecting to sninfr0lnxctl03 ({node_ip})...")
|
||||||
|
client.connect(node_ip, username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
cmds = [
|
||||||
|
"uname -r",
|
||||||
|
"ls -d /lib/modules/*",
|
||||||
|
"lsmod | grep -E 'veth|vxlan|br_netfilter'",
|
||||||
|
"modprobe veth",
|
||||||
|
"dmesg | tail -n 25"
|
||||||
|
]
|
||||||
|
|
||||||
|
for cmd in cmds:
|
||||||
|
print(f"\n>>> Running on {node_ip}: {cmd}")
|
||||||
|
stdin, stdout, stderr = client.exec_command(cmd)
|
||||||
|
out = stdout.read().decode('utf-8').strip()
|
||||||
|
err = stderr.read().decode('utf-8').strip()
|
||||||
|
if out:
|
||||||
|
print(out)
|
||||||
|
if err:
|
||||||
|
print("STDERR:", err)
|
||||||
|
|
||||||
|
client.close()
|
||||||
|
|
||||||
12
scripts/check_current_clients.py
Normal file
12
scripts/check_current_clients.py
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet", timeout=10)
|
||||||
|
|
||||||
|
cmd = 'k3s kubectl exec -n dns deploy/pihole -c pihole -- tail -n 25 /var/log/pihole/pihole.log'
|
||||||
|
stdin, stdout, stderr = client.exec_command(cmd)
|
||||||
|
print("Queries by client on Aerodrome in last 15 min:")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
print("Stderr:", stderr.read().decode())
|
||||||
|
client.close()
|
||||||
25
scripts/check_current_status.py
Normal file
25
scripts/check_current_status.py
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.11", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl get clusters.management.cattle.io -o yaml")
|
||||||
|
data = yaml.safe_load(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
|
for item in data.get('items', []):
|
||||||
|
name = item.get('metadata', {}).get('name')
|
||||||
|
disp = item.get('spec', {}).get('displayName')
|
||||||
|
print(f"\n==========================================")
|
||||||
|
print(f"Cluster: {disp} ({name})")
|
||||||
|
print(f"==========================================")
|
||||||
|
for c in item.get('status', {}).get('conditions', []):
|
||||||
|
if c.get('type') in ['Ready', 'Connected']:
|
||||||
|
print(f" [{c.get('type')}]: status={c.get('status')} reason={c.get('reason')} msg={c.get('message')}")
|
||||||
|
|
||||||
15
scripts/check_custom_coredns.py
Normal file
15
scripts/check_custom_coredns.py
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.41", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("k3s kubectl -n kube-system get configmap coredns-custom")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
print(stderr.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
14
scripts/check_diff_kernel.py
Normal file
14
scripts/check_diff_kernel.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.43", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("cmp /boot/vmlinuz-6.12.109+rpt-rpi-2712 /boot/firmware/kernel_2712.img && echo IDENTICAL || echo DIFFERENT")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
12
scripts/check_divloginsso.py
Normal file
12
scripts/check_divloginsso.py
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.11', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command('k3s kubectl exec -n dns deployment/technitium-dns -- grep -rn "divLoginSso" /opt/technitium/ 2>/dev/null || true')
|
||||||
|
print("grep divLoginSso:")
|
||||||
|
print(stdout.read().decode('utf-8', errors='ignore'))
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
14
scripts/check_dmesg_head.py
Normal file
14
scripts/check_dmesg_head.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.43", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("dmesg | head -n 10")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
17
scripts/check_dns_pods_live.py
Normal file
17
scripts/check_dns_pods_live.py
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
|
||||||
|
def check_cluster(ip, name):
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect(ip, username="root", key_filename=key_path, timeout=5)
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("k3s kubectl get pods -n dns -o wide")
|
||||||
|
print(f"Pods on {name} ({ip}):\n", stdout.read().decode())
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
|
check_cluster("140.44.4.11", "Aerodrome")
|
||||||
|
check_cluster("140.44.4.41", "Spaceport")
|
||||||
|
|
||||||
27
scripts/check_dns_query.py
Normal file
27
scripts/check_dns_query.py
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.11", username="root", key_filename=key_path, timeout=5)
|
||||||
|
|
||||||
|
cmd = "dig @140.44.4.36 google.com"
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd)
|
||||||
|
print("=== DIG FULL OUTPUT ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
cmd2 = "k3s kubectl -n dns logs deploy/pihole --tail=25"
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd2)
|
||||||
|
print("=== PIHOLE LOGS ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
cmd3 = "k3s kubectl -n dns logs deploy/technitium-dns --tail=25"
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd3)
|
||||||
|
print("=== TECHNITIUM LOGS ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
22
scripts/check_dns_resolutions.py
Normal file
22
scripts/check_dns_resolutions.py
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.71", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
def run(cmd):
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd)
|
||||||
|
return stdout.read().decode('utf-8', errors='ignore') + stderr.read().decode('utf-8', errors='ignore')
|
||||||
|
|
||||||
|
print("--- DNS check via ns1 (140.44.4.36) ---")
|
||||||
|
print(run("dig @140.44.4.36 +short pihole1.snarfnet.net pihole2.snarfnet.net dns1.snarfnet.net dns2.snarfnet.net"))
|
||||||
|
|
||||||
|
print("--- DNS check via ns2 (140.44.4.65) ---")
|
||||||
|
print(run("dig @140.44.4.65 +short pihole1.snarfnet.net pihole2.snarfnet.net dns1.snarfnet.net dns2.snarfnet.net"))
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
44
scripts/check_dns_status.py
Normal file
44
scripts/check_dns_status.py
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
def check_cluster(host, name):
|
||||||
|
print(f"=== Checking {name} ({host}) ===")
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect(host, username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command('k3s kubectl get pods -n dns -o wide')
|
||||||
|
print("Pods:")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command('k3s kubectl exec -n dns deployment/pihole -- pihole -v')
|
||||||
|
print("Pihole version:")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
# Set password explicitly
|
||||||
|
stdin, stdout, stderr = ssh.exec_command('k3s kubectl exec -n dns deployment/pihole -- pihole setpassword snarfnetadmin')
|
||||||
|
out = stdout.read().decode()
|
||||||
|
err = stderr.read().decode()
|
||||||
|
print("Set password with pihole setpassword:")
|
||||||
|
print(out)
|
||||||
|
if err:
|
||||||
|
print("ERR:", err)
|
||||||
|
|
||||||
|
if "Unknown" in out or "invalid" in out.lower() or err:
|
||||||
|
stdin, stdout, stderr = ssh.exec_command('k3s kubectl exec -n dns deployment/pihole -- pihole -a -p snarfnetadmin')
|
||||||
|
print("Fallback pihole -a -p:")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
# Check Technitium login via API
|
||||||
|
stdin, stdout, stderr = ssh.exec_command('k3s kubectl exec -n dns deployment/technitium-dns -- curl -s "http://localhost:5380/api/user/login?user=admin&pass=snarfnetadmin"')
|
||||||
|
login_res = stdout.read().decode()
|
||||||
|
print("Technitium API Login:", login_res)
|
||||||
|
|
||||||
|
# Check Technitium SSO config
|
||||||
|
stdin, stdout, stderr = ssh.exec_command("k3s kubectl exec -n dns deployment/technitium-dns -- cat /etc/dns/auth.config 2>/dev/null || echo 'No auth.config'")
|
||||||
|
print("Technitium auth.config:")
|
||||||
|
print(stdout.read().decode())
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
|
check_cluster('140.44.4.11', 'Aerodrome')
|
||||||
|
check_cluster('140.44.4.41', 'Spaceport')
|
||||||
22
scripts/check_dns_svcs.py
Normal file
22
scripts/check_dns_svcs.py
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.11", username="root", key_filename=key_path, timeout=5)
|
||||||
|
|
||||||
|
cmd = "k3s kubectl -n dns get pods,svc -o wide"
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd)
|
||||||
|
print("=== AERODROME DNS WORKLOADS ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
c.connect("140.44.4.41", username="root", key_filename=key_path, timeout=5)
|
||||||
|
stdin, stdout, stderr = c.exec_command(cmd)
|
||||||
|
print("=== SPACEPORT DNS WORKLOADS ===")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
15
scripts/check_docker_tail.py
Normal file
15
scripts/check_docker_tail.py
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
key_path = r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet'
|
||||||
|
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.71", username="root", key_filename=key_path, timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command("docker logs --tail 20 opencanary")
|
||||||
|
print("Docker logs:\n", stdout.read().decode())
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
14
scripts/check_dpkg_kernel.py
Normal file
14
scripts/check_dpkg_kernel.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.43", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("dpkg -l | grep -E 'linux-|raspberrypi-'")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
22
scripts/check_env_in_pod.py
Normal file
22
scripts/check_env_in_pod.py
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
c = paramiko.SSHClient()
|
||||||
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
c.connect("140.44.4.11", username="root", key_filename=key_path, timeout=5)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command("k3s kubectl -n dns exec deployment/technitium-dns -- env")
|
||||||
|
print("ENV IN POD:")
|
||||||
|
for line in stdout.read().decode('utf-8').splitlines():
|
||||||
|
if "DNS_SERVER" in line:
|
||||||
|
print(" ", line)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = c.exec_command("k3s kubectl -n dns exec deployment/technitium-dns -- ls -la /etc/dns/logs /etc/dns")
|
||||||
|
print("FILES:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
|
||||||
|
c.close()
|
||||||
|
|
||||||
12
scripts/check_etc_dns.py
Normal file
12
scripts/check_etc_dns.py
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
import paramiko
|
||||||
|
|
||||||
|
ssh = paramiko.SSHClient()
|
||||||
|
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
ssh.connect('140.44.4.11', username='root', key_filename=r'C:\Users\Snarf\.ssh\id_ed25519_snarfnet', timeout=10)
|
||||||
|
|
||||||
|
stdin, stdout, stderr = ssh.exec_command('k3s kubectl exec -n dns deployment/technitium-dns -- ls -la /etc/dns/')
|
||||||
|
print("/etc/dns listing:")
|
||||||
|
print(stdout.read().decode('utf-8', errors='ignore'))
|
||||||
|
|
||||||
|
ssh.close()
|
||||||
|
|
||||||
15
scripts/check_firmware_version.py
Normal file
15
scripts/check_firmware_version.py
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
import paramiko
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.stdout.reconfigure(encoding='utf-8')
|
||||||
|
|
||||||
|
key_path = r"C:\Users\Snarf\.ssh\id_ed25519_snarfnet"
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
|
||||||
|
client.connect("140.44.4.43", username="root", key_filename=key_path, timeout=10)
|
||||||
|
stdin, stdout, stderr = client.exec_command("zcat /boot/firmware/kernel_2712.img 2>/dev/null | strings | grep 'Linux version' | head -n 5")
|
||||||
|
print("kernel_2712.img version:")
|
||||||
|
print(stdout.read().decode('utf-8'))
|
||||||
|
client.close()
|
||||||
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user