SnarfCode/docs/WALKTHROUGH.md

# IaC Reverse Engineering Tool — User Walkthrough

> Bring your unmanaged on-premises infrastructure under Terraform control in minutes.

---

## What This Tool Does

```
┌─────────────────────────────────────────────────────────────────────────┐
│                                                                         │
│   Your Live Infrastructure          Generated Terraform Output          │
│                                                                         │
│   ┌──────────────┐                  ┌──────────────────────┐           │
│   │ Kubernetes   │──┐               │ kubernetes_deploy.tf │           │
│   │ Cluster      │  │               │ kubernetes_svc.tf    │           │
│   └──────────────┘  │               │ docker_service.tf    │           │
│   ┌──────────────┐  │  iac-reverse  │ windows_service.tf   │           │
│   │ Docker Swarm │──┼──────────────▶│ synology_volume.tf   │           │
│   │ Cluster      │  │   generate    │ harvester_vm.tf      │           │
│   └──────────────┘  │               │ variables.tf         │           │
│   ┌──────────────┐  │               │ providers.tf         │           │
│   │ Windows      │──┤               │ terraform.tfstate    │           │
│   │ Servers      │  │               └──────────────────────┘           │
│   └──────────────┘  │                                                   │
│   ┌──────────────┐  │                                                   │
│   │ Synology NAS │──┤                                                   │
│   └──────────────┘  │                                                   │
│   ┌──────────────┐  │                                                   │
│   │ Harvester    │──┘                                                   │
│   │ HCI          │                                                      │
│   └──────────────┘                                                      │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘
```

---

## Table of Contents

1. [Prerequisites](#prerequisites)
2. [Installation](#installation)
3. [Quick Start](#quick-start)
4. [Creating Scan Profiles](#creating-scan-profiles)
5. [Running a Scan](#running-a-scan)
6. [Generating Terraform Code](#generating-terraform-code)
7. [Incremental Scanning (Diff)](#incremental-scanning-diff)
8. [Validating Output](#validating-output)
9. [Authentik SSO Login](#authentik-sso-login)
10. [Supported Providers](#supported-providers)
11. [Troubleshooting](#troubleshooting)

---

## Prerequisites

Before installing, make sure you have:

| Requirement | Version | Purpose |
|---|---|---|
| Python | 3.11+ | Runtime |
| pip | Latest | Package installation |
| Terraform | 1.5+ | Output validation (optional) |
| Git | Any | Cloning the repo |

**Platform-specific requirements for scanning:**

| Provider | Requirement |
|---|---|
| Kubernetes | Valid kubeconfig file |
| Docker Swarm | Docker daemon access (TCP or socket) |
| Windows | WinRM enabled on target machines |
| Synology | DSM admin credentials |
| Harvester | Harvester cluster kubeconfig |
| Bare Metal | BMC/iDRAC Redfish API access |

---

## Installation

### Step 1: Clone the repository

```bash
git clone https://github.com/your-org/SnarfCode.git
cd SnarfCode
```

### Step 2: Install the tool

```bash
# Install in development mode (recommended)
pip install -e ".[dev]"
```

### Step 3: Verify installation

```bash
iac-reverse --version
```

Expected output:
```
iac-reverse, version 0.1.0
```

```
┌─────────────────────────────────────────────────────────────┐
│  ✓ Installation Complete                                    │
│                                                             │
│  You now have access to:                                    │
│    • iac-reverse scan       Discover infrastructure         │
│    • iac-reverse generate   Full pipeline (scan → HCL)      │
│    • iac-reverse diff       Incremental change detection    │
│    • iac-reverse validate   Validate generated output       │
│    • iac-reverse login      Authentik SSO authentication    │
│                                                             │
└─────────────────────────────────────────────────────────────┘
```

---

## Quick Start

The fastest way to get going — scan your Kubernetes cluster and generate Terraform:

```bash
# 1. Create a scan profile
cat > my-cluster.yaml << EOF
provider: kubernetes
credentials:
  kubeconfig_path: ~/.kube/config
  context: my-cluster
endpoints:
  - https://k8s-api.internal.lab:6443
EOF

# 2. Generate Terraform code
iac-reverse generate --profile my-cluster.yaml --output-dir ./terraform-output

# 3. Check the results
ls terraform-output/
```

That's it! You now have Terraform HCL + state for your cluster.

---

## Creating Scan Profiles

Scan profiles are YAML files that tell the tool what to scan and how to authenticate.

### Profile Structure

```yaml
provider: <provider_type>        # Required: kubernetes, docker_swarm, synology, etc.
credentials:                     # Required: provider-specific auth
  key: value
endpoints:                       # Optional: API endpoints to scan
  - https://api.example.com:6443
resource_type_filters:           # Optional: limit which resource types to discover
  - kubernetes_deployment
  - kubernetes_service
authentik_token: <token>         # Optional: SSO token from Authentik
```

### Environment Variable Expansion

Use `${ENV_VAR}` syntax in credential fields to avoid hardcoding secrets:

```yaml
provider: synology
credentials:
  host: nas01.internal.lab
  port: "5001"
  username: "${SYNOLOGY_USER}"
  password: "${SYNOLOGY_PASSWORD}"
  use_ssl: "true"
```

You can also provide defaults with `${ENV_VAR:-default_value}`:

```yaml
credentials:
  port: "${SYNOLOGY_PORT:-5001}"
```

### Example Profiles

<details>
<summary><b>Kubernetes (Raspberry Pi Cluster)</b></summary>

```yaml
provider: kubernetes
credentials:
  kubeconfig_path: "${HOME}/.kube/config"
  context: "pi-cluster"
endpoints:
  - "https://k8s-api.internal.lab:6443"
resource_type_filters:
  - kubernetes_deployment
  - kubernetes_service
  - kubernetes_ingress
  - kubernetes_config_map
  - kubernetes_persistent_volume
  - kubernetes_namespace
```
</details>

<details>
<summary><b>Docker Swarm</b></summary>

```yaml
provider: docker_swarm
credentials:
  host: "tcp://swarm-manager.internal.lab:2376"
  tls_verify: "true"
  cert_path: "${HOME}/.docker/certs"
```
</details>

<details>
<summary><b>Windows Server</b></summary>

```yaml
provider: windows
credentials:
  host: "win-server-01.internal.lab"
  username: "${WINDOWS_USER}"
  password: "${WINDOWS_PASSWORD}"
  transport: "ntlm"
  port: "5986"
  use_ssl: "true"
resource_type_filters:
  - windows_service
  - windows_scheduled_task
  - windows_iis_site
  - windows_iis_app_pool
  - windows_feature
  - windows_hyperv_vm
```
</details>

<details>
<summary><b>Synology NAS</b></summary>

```yaml
provider: synology
credentials:
  host: "nas01.internal.lab"
  port: "5001"
  username: "${SYNOLOGY_USER}"
  password: "${SYNOLOGY_PASSWORD}"
  use_ssl: "true"
resource_type_filters:
  - synology_shared_folder
  - synology_volume
  - synology_storage_pool
```
</details>

<details>
<summary><b>SUSE Harvester (Dell PowerEdge)</b></summary>

```yaml
provider: harvester
credentials:
  kubeconfig_path: "${HOME}/.kube/harvester-config"
  context: "harvester-cluster"
endpoints:
  - "https://harvester.internal.lab:6443"
```
</details>

<details>
<summary><b>Bare Metal (IPMI/Redfish)</b></summary>

```yaml
provider: bare_metal
credentials:
  host: "bmc-server01.internal.lab"
  username: "${BMC_USER}"
  password: "${BMC_PASSWORD}"
  port: "443"
  use_ssl: "true"
```
</details>

<details>
<summary><b>Multi-Provider (scan everything at once)</b></summary>

```yaml
- provider: kubernetes
  credentials:
    kubeconfig_path: ~/.kube/config
    context: pi-cluster
  endpoints:
    - https://k8s-api.internal.lab:6443

- provider: synology
  credentials:
    host: nas01.internal.lab
    port: "5001"
    username: "${SYNOLOGY_USER}"
    password: "${SYNOLOGY_PASSWORD}"

- provider: windows
  credentials:
    host: win-server-01.internal.lab
    username: "${WINDOWS_USER}"
    password: "${WINDOWS_PASSWORD}"
    transport: ntlm
    port: "5986"
    use_ssl: "true"
```
</details>

---

## Running a Scan

The `scan` command discovers resources without generating any output files. Useful for previewing what the tool will find.

```bash
iac-reverse scan --profile my-cluster.yaml
```

**Example output:**

```
Loading scan profile: my-cluster.yaml
Provider: kubernetes
Creating plugin...
Starting scan...
  [1/6] Scanning kubernetes_deployment... (12 resources found)
  [2/6] Scanning kubernetes_service... (18 resources found)
  [3/6] Scanning kubernetes_ingress... (4 resources found)
  [4/6] Scanning kubernetes_config_map... (23 resources found)
  [5/6] Scanning kubernetes_persistent_volume... (6 resources found)
  [6/6] Scanning kubernetes_namespace... (5 resources found)

Scan complete: 68 resources discovered
```

```
┌─────────────────────────────────────────────────────────────┐
│                                                             │
│  Pipeline Flow: scan command                                │
│                                                             │
│  ┌──────────┐    ┌──────────┐    ┌──────────────────┐      │
│  │  Profile  │───▶│ Scanner  │───▶│ Resource Summary │      │
│  │  (YAML)  │    │          │    │   (terminal)     │      │
│  └──────────┘    └──────────┘    └──────────────────┘      │
│                                                             │
└─────────────────────────────────────────────────────────────┘
```

---

## Generating Terraform Code

The `generate` command runs the full pipeline: scan → resolve dependencies → generate HCL → build state → validate.

```bash
iac-reverse generate --profile my-cluster.yaml --output-dir ./terraform-output
```

**Example output:**

```
Loading scan profile: my-cluster.yaml
Step 1/5: Scanning infrastructure...
  [1/6] Scanning kubernetes_deployment... (12 resources found)
  [2/6] Scanning kubernetes_service... (18 resources found)
  ...
  Found 68 resources
Step 2/5: Resolving dependencies...
  Resolved 42 relationships, 0 cycles detected
Step 3/5: Generating Terraform code...
  Generated 6 resource files
Step 4/5: Building Terraform state...
  State file: 68 entries
Step 5/5: Validating output...
  ✓ Validation passed

Generation complete:
  Output directory: ./terraform-output
  Resource files: 6
  Total resources: 68
```

**Generated file structure:**

```
terraform-output/
├── kubernetes_deployment.tf      # All deployments
├── kubernetes_service.tf         # All services
├── kubernetes_ingress.tf         # All ingresses
├── kubernetes_config_map.tf      # All config maps
├── kubernetes_persistent_volume.tf
├── kubernetes_namespace.tf
├── variables.tf                  # Extracted shared variables
├── providers.tf                  # Provider configuration
└── terraform.tfstate             # State binding to live resources
```

```
┌─────────────────────────────────────────────────────────────────────┐
│                                                                     │
│  Pipeline Flow: generate command                                    │
│                                                                     │
│  ┌────────┐  ┌─────────┐  ┌──────────┐  ┌───────┐  ┌───────────┐ │
│  │Profile │─▶│ Scanner │─▶│ Resolver │─▶│ Code  │─▶│ State     │ │
│  │ YAML   │  │         │  │          │  │ Gen   │  │ Builder   │ │
│  └────────┘  └─────────┘  └──────────┘  └───────┘  └───────────┘ │
│                                                          │         │
│                                                          ▼         │
│                                                     ┌───────────┐  │
│                                                     │ Validator │  │
│                                                     └───────────┘  │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘
```

---

## Incremental Scanning (Diff)

After your initial scan, use `diff` to detect infrastructure changes without regenerating everything.

```bash
iac-reverse diff --profile my-cluster.yaml
```

**Example output:**

```
Loading scan profile: my-cluster.yaml
Loading previous snapshot...
  Previous snapshot: 2024-01-14T09:00:00Z (65 resources)
Scanning infrastructure...
  [1/6] Scanning kubernetes_deployment... (14 resources found)
  ...
Comparing with previous scan...
  Snapshot saved

Change Summary:
  Added:    3
  Removed:  1
  Modified: 2

  + kubernetes_deployment/new-api-service
  + kubernetes_service/new-api-svc
  + kubernetes_ingress/new-api-ingress
  - kubernetes_deployment/deprecated-worker
  ~ kubernetes_deployment/web-frontend (replicas: 3 → 5)
  ~ kubernetes_service/web-svc (port: 8080 → 9090)
```

```
┌─────────────────────────────────────────────────────────────┐
│                                                             │
│  Incremental Scan Flow                                      │
│                                                             │
│  ┌──────────┐    ┌──────────┐    ┌──────────────────┐      │
│  │ Previous │    │ Current  │    │                  │      │
│  │ Snapshot │───▶│  Scan    │───▶│  Change Summary  │      │
│  │ (JSON)   │    │          │    │  + Added         │      │
│  └──────────┘    └──────────┘    │  - Removed       │      │
│                                  │  ~ Modified      │      │
│                                  └──────────────────┘      │
│                                                             │
└─────────────────────────────────────────────────────────────┘
```

---

## Validating Output

Run standalone validation against existing Terraform output:

```bash
iac-reverse validate --dir ./terraform-output
```

**Example output (success):**

```
Validating: ./terraform-output

Validation Results:
  terraform init:     ✓
  terraform validate: ✓
  terraform plan:     ✓

✓ All validations passed - no drift detected
```

**Example output (drift detected):**

```
Validating: ./terraform-output

Validation Results:
  terraform init:     ✓
  terraform validate: ✓
  terraform plan:     ✗

Planned Changes (2):
  modify: kubernetes_deployment.web_frontend
  add: kubernetes_service.new_backend

⚠ Validation passed but drift detected
```

---

## Authentik SSO Login

If your infrastructure uses Authentik for identity management, authenticate first:

```bash
iac-reverse login \
  --url https://auth.internal.lab \
  --client-id iac-reverse-tool \
  --client-secret <your-secret>
```

**Example output:**

```
Authenticating with Authentik at https://auth.internal.lab...
✓ Authenticated as user: admin@internal.lab
  Groups: admins, infra-team
  Token stored in .iac-reverse/token
```

The stored token is automatically used by subsequent `scan` and `generate` commands when `authentik_token` is referenced in your profile.

---

## Supported Providers

```
┌─────────────────────────────────────────────────────────────────────────┐
│                                                                         │
│  Provider              Platform Type        Architecture    Resources   │
│  ─────────────────────────────────────────────────────────────────────  │
│  kubernetes            Container Orch.      ARM/AArch64     6 types     │
│  docker_swarm          Container Orch.      ARM/AArch64     5 types     │
│  synology              Storage Appliance    ARM/AMD64       5 types     │
│  harvester             HCI                  AMD64           4 types     │
│  bare_metal            Bare Metal           AMD64           4 types     │
│  windows               Windows              AMD64           13 types    │
│                                                                         │
│  Total: 37 resource types across 6 providers                            │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘
```

### Resource Types by Provider

| Provider | Resource Types |
|---|---|
| **Kubernetes** | deployment, service, ingress, config_map, persistent_volume, namespace |
| **Docker Swarm** | service, network, volume, config, secret |
| **Synology** | shared_folder, volume, storage_pool, replication_task, user |
| **Harvester** | virtualmachine, volume, image, network |
| **Bare Metal** | hardware, bmc_config, network_interface, raid_config |
| **Windows** | service, scheduled_task, iis_site, iis_app_pool, network_adapter, firewall_rule, installed_software, feature, hyperv_vm, hyperv_switch, dns_record, local_user, local_group |

---

## Troubleshooting

### Common Issues

| Problem | Solution |
|---|---|
| `Terraform binary not found` | Install Terraform and add to PATH |
| `Authentication failed for provider 'kubernetes'` | Check kubeconfig path and context name |
| `WinRM is not enabled or unreachable` | Enable WinRM on the target Windows machine |
| `Connection refused` for Docker | Verify Docker daemon is running and accessible |
| `Environment variable 'X' is not set` | Export the required env var or add a default in the profile |

### Enabling WinRM on Windows targets

```powershell
# Run on the target Windows machine (as Administrator)
Enable-PSRemoting -Force
winrm set winrm/config/service '@{AllowUnencrypted="false"}'
winrm set winrm/config/service/auth '@{Basic="true"}'
```

### Checking connectivity

```bash
# Test Kubernetes access
kubectl --kubeconfig ~/.kube/config --context my-cluster get nodes

# Test Docker Swarm access
docker -H tcp://swarm-manager:2376 node ls

# Test WinRM access (from Linux)
python -c "import winrm; s = winrm.Session('https://win-server:5986/wsman', auth=('user','pass'), transport='ntlm', server_cert_validation='ignore'); print(s.run_ps('hostname').std_out)"
```

### Getting help

```bash
# General help
iac-reverse --help

# Command-specific help
iac-reverse scan --help
iac-reverse generate --help
iac-reverse diff --help
iac-reverse validate --help
iac-reverse login --help
```

---

## Architecture Overview

```
┌─────────────────────────────────────────────────────────────────────────┐
│                          iac-reverse CLI                                 │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│  ┌─────────────────────────────────────────────────────────────────┐   │
│  │                      Pipeline Engine                             │   │
│  │                                                                  │   │
│  │  ┌─────────┐  ┌──────────┐  ┌─────────┐  ┌───────┐  ┌───────┐ │   │
│  │  │Scanner  │─▶│Dependency│─▶│  Code   │─▶│ State │─▶│Valida-│ │   │
│  │  │         │  │ Resolver │  │Generator│  │Builder│  │ tor   │ │   │
│  │  └────┬────┘  └──────────┘  └─────────┘  └───────┘  └───────┘ │   │
│  │       │                                                          │   │
│  └───────┼──────────────────────────────────────────────────────────┘   │
│          │                                                              │
│  ┌───────┼──────────────────────────────────────────────────────────┐   │
│  │       │         Provider Plugins                                  │   │
│  │  ┌────┴─────┬──────────┬──────────┬──────────┬──────────┐       │   │
│  │  │Kubernetes│  Docker  │ Synology │Harvester │  Windows │       │   │
│  │  │  Plugin  │  Plugin  │  Plugin  │  Plugin  │  Plugin  │       │   │
│  │  └──────────┴──────────┴──────────┴──────────┴──────────┘       │   │
│  │                                                                   │   │
│  └───────────────────────────────────────────────────────────────────┘   │
│                                                                         │
│  ┌───────────────────────────────────────────────────────────────────┐   │
│  │                   Incremental Scan Engine                         │   │
│  │  ┌──────────────┐  ┌────────────────┐  ┌─────────────────────┐  │   │
│  │  │Snapshot Store│  │Change Detector │  │Incremental Updater  │  │   │
│  │  └──────────────┘  └────────────────┘  └─────────────────────┘  │   │
│  └───────────────────────────────────────────────────────────────────┘   │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘
```

---

## Tips & Best Practices

1. **Start small** — Scan one provider at a time before combining into multi-provider profiles
2. **Use resource filters** — Limit initial scans to specific resource types to keep output manageable
3. **Store secrets in env vars** — Never hardcode passwords in profile YAML files
4. **Run diff regularly** — Set up a cron job or CI pipeline to detect infrastructure drift
5. **Review generated code** — The tool generates a starting point; review and customize before using in production
6. **Version control your profiles** — Keep scan profiles in git alongside your generated Terraform code

---

*Built with Python 3.11+ • Terraform HCL output • Property-based testing with Hypothesis*