Snarf/SnarfCode
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

User documentation

Browse Source
This commit is contained in:
p2913020
2026-05-22 21:46:41 -04:00
parent 1a11244fff
commit c564ccc2ea
1 changed files with 672 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

672
docs/WALKTHROUGH.md Normal file
View File

@@ -0,0 +1,672 @@
# IaC Reverse Engineering Tool — User Walkthrough
> Bring your unmanaged on-premises infrastructure under Terraform control in minutes.
---
## What This Tool Does
```
┌─────────────────────────────────────────────────────────────────────────┐
│ │
│ Your Live Infrastructure Generated Terraform Output │
│ │
│ ┌──────────────┐ ┌──────────────────────┐ │
│ │ Kubernetes │──┐ │ kubernetes_deploy.tf │ │
│ │ Cluster │ │ │ kubernetes_svc.tf │ │
│ └──────────────┘ │ │ docker_service.tf │ │
│ ┌──────────────┐ │ iac-reverse │ windows_service.tf │ │
│ │ Docker Swarm │──┼──────────────▶│ synology_volume.tf │ │
│ │ Cluster │ │ generate │ harvester_vm.tf │ │
│ └──────────────┘ │ │ variables.tf │ │
│ ┌──────────────┐ │ │ providers.tf │ │
│ │ Windows │──┤ │ terraform.tfstate │ │
│ │ Servers │ │ └──────────────────────┘ │
│ └──────────────┘ │ │
│ ┌──────────────┐ │ │
│ │ Synology NAS │──┤ │
│ └──────────────┘ │ │
│ ┌──────────────┐ │ │
│ │ Harvester │──┘ │
│ │ HCI │ │
│ └──────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘
```
---
## Table of Contents
1. [Prerequisites](#prerequisites)
2. [Installation](#installation)
3. [Quick Start](#quick-start)
4. [Creating Scan Profiles](#creating-scan-profiles)
5. [Running a Scan](#running-a-scan)
6. [Generating Terraform Code](#generating-terraform-code)
7. [Incremental Scanning (Diff)](#incremental-scanning-diff)
8. [Validating Output](#validating-output)
9. [Authentik SSO Login](#authentik-sso-login)
10. [Supported Providers](#supported-providers)
11. [Troubleshooting](#troubleshooting)
---
## Prerequisites
Before installing, make sure you have:
| Requirement | Version | Purpose |
|---|---|---|
| Python | 3.11+ | Runtime |
| pip | Latest | Package installation |
| Terraform | 1.5+ | Output validation (optional) |
| Git | Any | Cloning the repo |
**Platform-specific requirements for scanning:**
| Provider | Requirement |
|---|---|
| Kubernetes | Valid kubeconfig file |
| Docker Swarm | Docker daemon access (TCP or socket) |
| Windows | WinRM enabled on target machines |
| Synology | DSM admin credentials |
| Harvester | Harvester cluster kubeconfig |
| Bare Metal | BMC/iDRAC Redfish API access |
---
## Installation
### Step 1: Clone the repository
```bash
git clone https://github.com/your-org/SnarfCode.git
cd SnarfCode
```
### Step 2: Install the tool
```bash
# Install in development mode (recommended)
pip install -e ".[dev]"
```
### Step 3: Verify installation
```bash
iac-reverse --version
```
Expected output:
```
iac-reverse, version 0.1.0
```
```
┌─────────────────────────────────────────────────────────────┐
│ ✓ Installation Complete │
│ │
│ You now have access to: │
│ • iac-reverse scan Discover infrastructure │
│ • iac-reverse generate Full pipeline (scan → HCL) │
│ • iac-reverse diff Incremental change detection │
│ • iac-reverse validate Validate generated output │
│ • iac-reverse login Authentik SSO authentication │
│ │
└─────────────────────────────────────────────────────────────┘
```
---
## Quick Start
The fastest way to get going — scan your Kubernetes cluster and generate Terraform:
```bash
# 1. Create a scan profile
cat > my-cluster.yaml << EOF
provider: kubernetes
credentials:
kubeconfig_path: ~/.kube/config
context: my-cluster
endpoints:
- https://k8s-api.internal.lab:6443
EOF
# 2. Generate Terraform code
iac-reverse generate --profile my-cluster.yaml --output-dir ./terraform-output
# 3. Check the results
ls terraform-output/
```
That's it! You now have Terraform HCL + state for your cluster.
---
## Creating Scan Profiles
Scan profiles are YAML files that tell the tool what to scan and how to authenticate.
### Profile Structure
```yaml
provider: <provider_type> # Required: kubernetes, docker_swarm, synology, etc.
credentials: # Required: provider-specific auth
key: value
endpoints: # Optional: API endpoints to scan
- https://api.example.com:6443
resource_type_filters: # Optional: limit which resource types to discover
- kubernetes_deployment
- kubernetes_service
authentik_token: <token> # Optional: SSO token from Authentik
```
### Environment Variable Expansion
Use `${ENV_VAR}` syntax in credential fields to avoid hardcoding secrets:
```yaml
provider: synology
credentials:
host: nas01.internal.lab
port: "5001"
username: "${SYNOLOGY_USER}"
password: "${SYNOLOGY_PASSWORD}"
use_ssl: "true"
```
You can also provide defaults with `${ENV_VAR:-default_value}`:
```yaml
credentials:
port: "${SYNOLOGY_PORT:-5001}"
```
### Example Profiles
<details>
<summary><b>Kubernetes (Raspberry Pi Cluster)</b></summary>
```yaml
provider: kubernetes
credentials:
kubeconfig_path: "${HOME}/.kube/config"
context: "pi-cluster"
endpoints:
- "https://k8s-api.internal.lab:6443"
resource_type_filters:
- kubernetes_deployment
- kubernetes_service
- kubernetes_ingress
- kubernetes_config_map
- kubernetes_persistent_volume
- kubernetes_namespace
```
</details>
<details>
<summary><b>Docker Swarm</b></summary>
```yaml
provider: docker_swarm
credentials:
host: "tcp://swarm-manager.internal.lab:2376"
tls_verify: "true"
cert_path: "${HOME}/.docker/certs"
```
</details>
<details>
<summary><b>Windows Server</b></summary>
```yaml
provider: windows
credentials:
host: "win-server-01.internal.lab"
username: "${WINDOWS_USER}"
password: "${WINDOWS_PASSWORD}"
transport: "ntlm"
port: "5986"
use_ssl: "true"
resource_type_filters:
- windows_service
- windows_scheduled_task
- windows_iis_site
- windows_iis_app_pool
- windows_feature
- windows_hyperv_vm
```
</details>
<details>
<summary><b>Synology NAS</b></summary>
```yaml
provider: synology
credentials:
host: "nas01.internal.lab"
port: "5001"
username: "${SYNOLOGY_USER}"
password: "${SYNOLOGY_PASSWORD}"
use_ssl: "true"
resource_type_filters:
- synology_shared_folder
- synology_volume
- synology_storage_pool
```
</details>
<details>
<summary><b>SUSE Harvester (Dell PowerEdge)</b></summary>
```yaml
provider: harvester
credentials:
kubeconfig_path: "${HOME}/.kube/harvester-config"
context: "harvester-cluster"
endpoints:
- "https://harvester.internal.lab:6443"
```
</details>
<details>
<summary><b>Bare Metal (IPMI/Redfish)</b></summary>
```yaml
provider: bare_metal
credentials:
host: "bmc-server01.internal.lab"
username: "${BMC_USER}"
password: "${BMC_PASSWORD}"
port: "443"
use_ssl: "true"
```
</details>
<details>
<summary><b>Multi-Provider (scan everything at once)</b></summary>
```yaml
- provider: kubernetes
credentials:
kubeconfig_path: ~/.kube/config
context: pi-cluster
endpoints:
- https://k8s-api.internal.lab:6443
- provider: synology
credentials:
host: nas01.internal.lab
port: "5001"
username: "${SYNOLOGY_USER}"
password: "${SYNOLOGY_PASSWORD}"
- provider: windows
credentials:
host: win-server-01.internal.lab
username: "${WINDOWS_USER}"
password: "${WINDOWS_PASSWORD}"
transport: ntlm
port: "5986"
use_ssl: "true"
```
</details>
---
## Running a Scan
The `scan` command discovers resources without generating any output files. Useful for previewing what the tool will find.
```bash
iac-reverse scan --profile my-cluster.yaml
```
**Example output:**
```
Loading scan profile: my-cluster.yaml
Provider: kubernetes
Creating plugin...
Starting scan...
[1/6] Scanning kubernetes_deployment... (12 resources found)
[2/6] Scanning kubernetes_service... (18 resources found)
[3/6] Scanning kubernetes_ingress... (4 resources found)
[4/6] Scanning kubernetes_config_map... (23 resources found)
[5/6] Scanning kubernetes_persistent_volume... (6 resources found)
[6/6] Scanning kubernetes_namespace... (5 resources found)
Scan complete: 68 resources discovered
```
```
┌─────────────────────────────────────────────────────────────┐
│ │
│ Pipeline Flow: scan command │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │
│ │ Profile │───▶│ Scanner │───▶│ Resource Summary │ │
│ │ (YAML) │ │ │ │ (terminal) │ │
│ └──────────┘ └──────────┘ └──────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
```
---
## Generating Terraform Code
The `generate` command runs the full pipeline: scan → resolve dependencies → generate HCL → build state → validate.
```bash
iac-reverse generate --profile my-cluster.yaml --output-dir ./terraform-output
```
**Example output:**
```
Loading scan profile: my-cluster.yaml
Step 1/5: Scanning infrastructure...
[1/6] Scanning kubernetes_deployment... (12 resources found)
[2/6] Scanning kubernetes_service... (18 resources found)
...
Found 68 resources
Step 2/5: Resolving dependencies...
Resolved 42 relationships, 0 cycles detected
Step 3/5: Generating Terraform code...
Generated 6 resource files
Step 4/5: Building Terraform state...
State file: 68 entries
Step 5/5: Validating output...
✓ Validation passed
Generation complete:
Output directory: ./terraform-output
Resource files: 6
Total resources: 68
```
**Generated file structure:**
```
terraform-output/
├── kubernetes_deployment.tf # All deployments
├── kubernetes_service.tf # All services
├── kubernetes_ingress.tf # All ingresses
├── kubernetes_config_map.tf # All config maps
├── kubernetes_persistent_volume.tf
├── kubernetes_namespace.tf
├── variables.tf # Extracted shared variables
├── providers.tf # Provider configuration
└── terraform.tfstate # State binding to live resources
```
```
┌─────────────────────────────────────────────────────────────────────┐
│ │
│ Pipeline Flow: generate command │
│ │
│ ┌────────┐ ┌─────────┐ ┌──────────┐ ┌───────┐ ┌───────────┐ │
│ │Profile │─▶│ Scanner │─▶│ Resolver │─▶│ Code │─▶│ State │ │
│ │ YAML │ │ │ │ │ │ Gen │ │ Builder │ │
│ └────────┘ └─────────┘ └──────────┘ └───────┘ └───────────┘ │
│ │ │
│ ▼ │
│ ┌───────────┐ │
│ │ Validator │ │
│ └───────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘
```
---
## Incremental Scanning (Diff)
After your initial scan, use `diff` to detect infrastructure changes without regenerating everything.
```bash
iac-reverse diff --profile my-cluster.yaml
```
**Example output:**
```
Loading scan profile: my-cluster.yaml
Loading previous snapshot...
Previous snapshot: 2024-01-14T09:00:00Z (65 resources)
Scanning infrastructure...
[1/6] Scanning kubernetes_deployment... (14 resources found)
...
Comparing with previous scan...
Snapshot saved
Change Summary:
Added: 3
Removed: 1
Modified: 2
+ kubernetes_deployment/new-api-service
+ kubernetes_service/new-api-svc
+ kubernetes_ingress/new-api-ingress
- kubernetes_deployment/deprecated-worker
~ kubernetes_deployment/web-frontend (replicas: 3 → 5)
~ kubernetes_service/web-svc (port: 8080 → 9090)
```
```
┌─────────────────────────────────────────────────────────────┐
│ │
│ Incremental Scan Flow │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │
│ │ Previous │ │ Current │ │ │ │
│ │ Snapshot │───▶│ Scan │───▶│ Change Summary │ │
│ │ (JSON) │ │ │ │ + Added │ │
│ └──────────┘ └──────────┘ │ - Removed │ │
│ │ ~ Modified │ │
│ └──────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
```
---
## Validating Output
Run standalone validation against existing Terraform output:
```bash
iac-reverse validate --dir ./terraform-output
```
**Example output (success):**
```
Validating: ./terraform-output
Validation Results:
terraform init: ✓
terraform validate: ✓
terraform plan: ✓
✓ All validations passed - no drift detected
```
**Example output (drift detected):**
```
Validating: ./terraform-output
Validation Results:
terraform init: ✓
terraform validate: ✓
terraform plan: ✗
Planned Changes (2):
modify: kubernetes_deployment.web_frontend
add: kubernetes_service.new_backend
⚠ Validation passed but drift detected
```
---
## Authentik SSO Login
If your infrastructure uses Authentik for identity management, authenticate first:
```bash
iac-reverse login \
--url https://auth.internal.lab \
--client-id iac-reverse-tool \
--client-secret <your-secret>
```
**Example output:**
```
Authenticating with Authentik at https://auth.internal.lab...
✓ Authenticated as user: admin@internal.lab
Groups: admins, infra-team
Token stored in .iac-reverse/token
```
The stored token is automatically used by subsequent `scan` and `generate` commands when `authentik_token` is referenced in your profile.
---
## Supported Providers
```
┌─────────────────────────────────────────────────────────────────────────┐
│ │
│ Provider Platform Type Architecture Resources │
│ ───────────────────────────────────────────────────────────────────── │
│ kubernetes Container Orch. ARM/AArch64 6 types │
│ docker_swarm Container Orch. ARM/AArch64 5 types │
│ synology Storage Appliance ARM/AMD64 5 types │
│ harvester HCI AMD64 4 types │
│ bare_metal Bare Metal AMD64 4 types │
│ windows Windows AMD64 13 types │
│ │
│ Total: 37 resource types across 6 providers │
│ │
└─────────────────────────────────────────────────────────────────────────┘
```
### Resource Types by Provider
| Provider | Resource Types |
|---|---|
| **Kubernetes** | deployment, service, ingress, config_map, persistent_volume, namespace |
| **Docker Swarm** | service, network, volume, config, secret |
| **Synology** | shared_folder, volume, storage_pool, replication_task, user |
| **Harvester** | virtualmachine, volume, image, network |
| **Bare Metal** | hardware, bmc_config, network_interface, raid_config |
| **Windows** | service, scheduled_task, iis_site, iis_app_pool, network_adapter, firewall_rule, installed_software, feature, hyperv_vm, hyperv_switch, dns_record, local_user, local_group |
---
## Troubleshooting
### Common Issues
| Problem | Solution |
|---|---|
| `Terraform binary not found` | Install Terraform and add to PATH |
| `Authentication failed for provider 'kubernetes'` | Check kubeconfig path and context name |
| `WinRM is not enabled or unreachable` | Enable WinRM on the target Windows machine |
| `Connection refused` for Docker | Verify Docker daemon is running and accessible |
| `Environment variable 'X' is not set` | Export the required env var or add a default in the profile |
### Enabling WinRM on Windows targets
```powershell
# Run on the target Windows machine (as Administrator)
Enable-PSRemoting -Force
winrm set winrm/config/service '@{AllowUnencrypted="false"}'
winrm set winrm/config/service/auth '@{Basic="true"}'
```
### Checking connectivity
```bash
# Test Kubernetes access
kubectl --kubeconfig ~/.kube/config --context my-cluster get nodes
# Test Docker Swarm access
docker -H tcp://swarm-manager:2376 node ls
# Test WinRM access (from Linux)
python -c "import winrm; s = winrm.Session('https://win-server:5986/wsman', auth=('user','pass'), transport='ntlm', server_cert_validation='ignore'); print(s.run_ps('hostname').std_out)"
```
### Getting help
```bash
# General help
iac-reverse --help
# Command-specific help
iac-reverse scan --help
iac-reverse generate --help
iac-reverse diff --help
iac-reverse validate --help
iac-reverse login --help
```
---
## Architecture Overview
```
┌─────────────────────────────────────────────────────────────────────────┐
│ iac-reverse CLI │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Pipeline Engine │ │
│ │ │ │
│ │ ┌─────────┐ ┌──────────┐ ┌─────────┐ ┌───────┐ ┌───────┐ │ │
│ │ │Scanner │─▶│Dependency│─▶│ Code │─▶│ State │─▶│Valida-│ │ │
│ │ │ │ │ Resolver │ │Generator│ │Builder│ │ tor │ │ │
│ │ └────┬────┘ └──────────┘ └─────────┘ └───────┘ └───────┘ │ │
│ │ │ │ │
│ └───────┼──────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌───────┼──────────────────────────────────────────────────────────┐ │
│ │ │ Provider Plugins │ │
│ │ ┌────┴─────┬──────────┬──────────┬──────────┬──────────┐ │ │
│ │ │Kubernetes│ Docker │ Synology │Harvester │ Windows │ │ │
│ │ │ Plugin │ Plugin │ Plugin │ Plugin │ Plugin │ │ │
│ │ └──────────┴──────────┴──────────┴──────────┴──────────┘ │ │
│ │ │ │
│ └───────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌───────────────────────────────────────────────────────────────────┐ │
│ │ Incremental Scan Engine │ │
│ │ ┌──────────────┐ ┌────────────────┐ ┌─────────────────────┐ │ │
│ │ │Snapshot Store│ │Change Detector │ │Incremental Updater │ │ │
│ │ └──────────────┘ └────────────────┘ └─────────────────────┘ │ │
│ └───────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘
```
---
## Tips & Best Practices
1. **Start small** — Scan one provider at a time before combining into multi-provider profiles
2. **Use resource filters** — Limit initial scans to specific resource types to keep output manageable
3. **Store secrets in env vars** — Never hardcode passwords in profile YAML files
4. **Run diff regularly** — Set up a cron job or CI pipeline to detect infrastructure drift
5. **Review generated code** — The tool generates a starting point; review and customize before using in production
6. **Version control your profiles** — Keep scan profiles in git alongside your generated Terraform code
---
*Built with Python 3.11+ • Terraform HCL output • Property-based testing with Hypothesis*