SnarfCode/Zabbix-fix.md

# Zabbix Auto-Registration Deployment

Deployment scripts and documentation for Zabbix Agent 2 with PSK-encrypted auto-registration against `zabbix.snarfnet.net`.

## Overview

This project automates the end-to-end setup of Zabbix active agent auto-registration:

1. **Server-side:** Creates auto-registration actions via the Zabbix API so new agents are automatically assigned to host groups and linked to templates.
2. **Agent-side:** Installs and configures Zabbix Agent 2 with PSK encryption on Linux (x86_64 and ARM) and Windows hosts.

When an agent starts with `ServerActive` and `HostMetadata` configured, it reaches out to the Zabbix server on port 10051. The server matches the metadata against auto-registration action conditions and automatically adds the host.

## Scripts

| File | Purpose |
|------|---------|
| `configure_server_autoregistration.sh` | Creates host groups and auto-registration actions on the Zabbix server via API |
| `deploy_zabbix_agent_linux.sh` | Agent install for Linux x86_64 (RHEL, Debian, Ubuntu) |
| `deploy_zabbix_agent_linux_arm.sh` | Agent install for Linux ARM (aarch64, armhf, Raspberry Pi) |
| `deploy_zabbix_agent_windows.ps1` | Agent install for Windows x86_64 |

## Prerequisites

- **Zabbix Server 7.0** running and accessible
- **PSK encryption** already configured on the server (Administration → General → Autoregistration)
- **Port 10051/TCP** exposed and reachable from agent hosts (see [Kubernetes Exposure](#kubernetes-exposure) if running in k8s)
- `curl` and `jq` on the machine running the server config script
- `openssl` on agent hosts (for PSK key generation if not providing one)

---

## Step 1: Expose Zabbix Server Trapper Port (Kubernetes)

If your Zabbix server runs in Kubernetes, port 10051 must be exposed externally for agents to connect. The web UI (443) is not sufficient — agents need the trapper port.

### Ports Required

| Port | Service | Direction | Purpose |
|------|---------|-----------|---------|
| **10051/TCP** | zabbix-server | Inbound from agents | Active check-ins, auto-registration |
| 443/TCP | zabbix-web | Inbound from users | Web UI and API |

### Option A: LoadBalancer Service (recommended)

```yaml
apiVersion: v1
kind: Service
metadata:
  name: zabbix-server-trapper
  namespace: zabbix
spec:
  type: LoadBalancer
  selector:
    app: zabbix-server  # match your pod labels
  ports:
    - name: trapper
      port: 10051
      targetPort: 10051
      protocol: TCP
```

### Option B: NodePort Service

```yaml
apiVersion: v1
kind: Service
metadata:
  name: zabbix-server-trapper
  namespace: zabbix
spec:
  type: NodePort
  selector:
    app: zabbix-server  # match your pod labels
  ports:
    - name: trapper
      port: 10051
      targetPort: 10051
      nodePort: 30051
      protocol: TCP
```

With NodePort, update agent `ServerActive` to use `<node-ip>:30051` or put a load balancer in front.

### Option C: Nginx Ingress TCP Passthrough

Add to the ingress controller's TCP ConfigMap:

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: tcp-services
  namespace: ingress-nginx
data:
  "10051": "zabbix/zabbix-server:10051"
```

Ensure the ingress controller's Service also exposes port 10051.

### DNS Considerations

Make sure `zabbix.snarfnet.net` resolves to the IP where port 10051 is exposed. If the web UI and trapper are on different IPs, either:
- Point the main DNS record to the trapper LB and use a separate record for the web UI
- Or update `ServerActive` in agent configs to a dedicated trapper hostname

### Verify Connectivity

From an agent host:

```bash
nc -zv zabbix.snarfnet.net 10051
```

Expected: `Connection to zabbix.snarfnet.net 10051 port [tcp/*] succeeded!`

If you get "connection refused" — the port isn't exposed or the trapper process isn't running.

---

## Step 2: Configure Server Auto-Registration Actions

Run the server configuration script to create host groups and auto-registration actions:

```bash
bash configure_server_autoregistration.sh -u Admin -p 'your_zabbix_admin_password'
```

### What it does

1. Authenticates with the Zabbix API at `https://zabbix.snarfnet.net/api_jsonrpc.php`
2. Finds or creates host groups: `Linux servers`, `Windows servers`
3. Looks up templates: `Linux by Zabbix agent active`, `Windows by Zabbix agent active`
4. Creates two auto-registration actions (skips if they already exist)

### Actions Created

| Action | Condition | Operations |
|--------|-----------|------------|
| Auto-register Linux hosts | Host metadata contains `Linux` | Add to group `Linux servers`, link template `Linux by Zabbix agent active` |
| Auto-register Windows hosts | Host metadata contains `Windows` | Add to group `Windows servers`, link template `Windows by Zabbix agent active` |

### Options

```
-u    Zabbix API username (required)
-p    Zabbix API password (required)
-s    Zabbix API URL (default: https://zabbix.snarfnet.net/api_jsonrpc.php)
-h    Show help
```

### Notes

- The API user must have **Super admin** role to create actions
- PSK configuration is assumed to already be in place (Administration → General → Autoregistration)
- The script is idempotent — safe to run multiple times

---

## Step 3: Deploy Agents

### Generate a Shared PSK Key

All agents must use the same PSK key that's configured on the server:

```bash
openssl rand -hex 32
```

### Linux x86_64

```bash
# Auto-generate PSK (prints key at end)
sudo bash deploy_zabbix_agent_linux.sh

# With a specific PSK
sudo bash deploy_zabbix_agent_linux.sh "your_64_char_hex_psk_here"
```

**Supports:** RHEL/CentOS/Rocky/Alma 8+, Ubuntu, Debian

**What it does:**
1. Detects OS family (RHEL or Debian-based)
2. Adds the Zabbix 7.0 repository and installs `zabbix-agent2`
3. Writes PSK file with restricted permissions (640, root:zabbix)
4. Configures `ServerActive=zabbix.snarfnet.net`, `HostMetadata=Linux`, TLS PSK settings
5. Enables and starts the `zabbix-agent2` service

### Linux ARM (Raspberry Pi, aarch64, armhf)

```bash
# Auto-generate PSK
sudo bash deploy_zabbix_agent_linux_arm.sh

# With a specific PSK
sudo bash deploy_zabbix_agent_linux_arm.sh "your_64_char_hex_psk_here"
```

**Supports:** Raspberry Pi OS, Ubuntu ARM, Debian ARM, any aarch64/armhf/armv6l Linux with systemd

**What it does:**
1. Detects architecture (aarch64, armv7l, armv6l)
2. Tries package manager install (apt on Debian/Ubuntu/Raspbian)
3. Falls back to pre-compiled static binary tarball from Zabbix CDN
4. Creates systemd service unit for binary installs
5. Creates `zabbix` user if needed
6. Writes PSK file and agent configuration
7. Enables and starts the service

### Windows

```powershell
# Run as Administrator

# Auto-generate PSK
.\deploy_zabbix_agent_windows.ps1

# With a specific PSK
.\deploy_zabbix_agent_windows.ps1 -PskKey "your_64_char_hex_psk_here"
```

**Supports:** Windows Server 2016+, Windows 10/11 (x86_64)

**What it does:**
1. Downloads Zabbix Agent 2 MSI from official CDN
2. Installs silently to `C:\Program Files\Zabbix Agent 2`
3. Writes PSK file with ACL-restricted permissions (Administrators + SYSTEM only)
4. Writes agent config with `HostMetadata=Windows` and TLS PSK settings
5. Adds Windows Firewall rule for port 10050 inbound (Domain/Private profiles)
6. Sets service to automatic start and starts it

---

## Configuration Reference

| Setting | Value |
|---------|-------|
| Zabbix Server | `zabbix.snarfnet.net` |
| PSK Identity | `PSK_autoregister` |
| Host Metadata (Linux) | `Linux` |
| Host Metadata (Windows) | `Windows` |
| PSK File (Linux) | `/etc/zabbix/zabbix_agent2.psk` |
| PSK File (Windows) | `C:\Program Files\Zabbix Agent 2\zabbix_agent2.psk` |
| Agent Config (Linux) | `/etc/zabbix/zabbix_agent2.conf` |
| Agent Config (Windows) | `C:\Program Files\Zabbix Agent 2\zabbix_agent2.conf` |
| Trapper Port | 10051 (agent → server, active checks + registration) |
| Agent Port | 10050 (server → agent, passive checks) |

---

## Security Notes

- PSK key must be **identical** on the server and all agents using the same identity
- PSK files are permission-locked (640 on Linux, ACL-restricted on Windows)
- Use unique PSK identities per environment to segment (e.g., `PSK_prod`, `PSK_dev`)
- Rotate PSK keys by updating the server autoregistration config and redeploying agents
- The server config script does **not** modify PSK settings — manage those separately in the Zabbix UI

---

## Troubleshooting

### Connectivity Test

```bash
# From agent → server (must succeed for auto-registration)
nc -zv zabbix.snarfnet.net 10051
```

```powershell
Test-NetConnection -ComputerName zabbix.snarfnet.net -Port 10051
```

### Agent Logs

```bash
# Linux
journalctl -u zabbix-agent2 --since "5 minutes ago"
tail -f /var/log/zabbix/zabbix_agent2.log
grep -iE "error|failed|denied|psk|tls" /var/log/zabbix/zabbix_agent2.log
```

```powershell
# Windows
Get-Content "C:\Program Files\Zabbix Agent 2\zabbix_agent2.log" -Tail 50
Select-String -Path "C:\Program Files\Zabbix Agent 2\zabbix_agent2.log" -Pattern "error|failed|denied|psk|tls"
```

### Server Logs (on Zabbix server)

```bash
tail -f /var/log/zabbix/zabbix_server.log | grep -i "autoregistration\|psk\|tls\|cannot"
```

### Common Issues

| Symptom | Cause | Fix |
|---------|-------|-----|
| `connection refused` on 10051 | Port not exposed (Kubernetes) or trapper not running | Expose port 10051 via LoadBalancer/NodePort; check `StartTrappers` in server config |
| `connection timed out` on 10051 | Firewall blocking traffic | Open outbound 10051 on agent host; open inbound 10051 on server/cluster |
| `TLS handshake failed` | PSK key or identity mismatch | Verify key matches exactly; check for trailing newlines in PSK file |
| Agent connects but host doesn't appear | Auto-registration action missing or disabled | Run `configure_server_autoregistration.sh`; verify actions are enabled in UI |
| Action exists but doesn't trigger | HostMetadata doesn't match condition | Verify agent config has `HostMetadata=Linux` or `HostMetadata=Windows` |
| Hostname conflict | Host with same name already exists | Delete/rename existing host in Zabbix, or change `HostnameItem` |
| Script creates actions with invalid JSON | Log messages captured in variables | Fixed in current version — `log()` writes to stderr |

### Verify Agent Config

```bash
# Linux — confirm critical settings
grep -E "^Server=|^ServerActive=|^HostMetadata=|^TLS" /etc/zabbix/zabbix_agent2.conf

# Check PSK file has no trailing newline
cat -A /etc/zabbix/zabbix_agent2.psk
# Should end with $ immediately after hex string, no extra lines
```

### Verify Server Actions via API

```bash
# Get auth token
TOKEN=$(curl -s -X POST https://zabbix.snarfnet.net/api_jsonrpc.php \
  -H "Content-Type: application/json-rpc" \
  -d '{"jsonrpc":"2.0","method":"user.login","params":{"username":"Admin","password":"YOUR_PASS"},"id":1}' \
  | jq -r '.result')

# List autoregistration actions
curl -s -X POST https://zabbix.snarfnet.net/api_jsonrpc.php \
  -H "Content-Type: application/json-rpc" \
  -d "{\"jsonrpc\":\"2.0\",\"method\":\"action.get\",\"params\":{\"filter\":{\"eventsource\":\"2\"}},\"auth\":\"${TOKEN}\",\"id\":2}" \
  | jq '.result[] | {name, status}'
```

---

## Deployment Order Summary

1. **Expose port 10051** on your Kubernetes cluster (LoadBalancer/NodePort/Ingress TCP)
2. **Verify connectivity** from an agent host: `nc -zv zabbix.snarfnet.net 10051`
3. **Run server config script** to create auto-registration actions
4. **Deploy agents** with the shared PSK key
5. **Verify** hosts appear in Zabbix UI under their respective host groups