From c564ccc2ea731b0244e22876cbcfc495330ff09b Mon Sep 17 00:00:00 2001 From: p2913020 Date: Fri, 22 May 2026 21:46:41 -0400 Subject: [PATCH] User documentation --- docs/WALKTHROUGH.md | 672 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 672 insertions(+) create mode 100644 docs/WALKTHROUGH.md diff --git a/docs/WALKTHROUGH.md b/docs/WALKTHROUGH.md new file mode 100644 index 0000000..ddbbc2e --- /dev/null +++ b/docs/WALKTHROUGH.md @@ -0,0 +1,672 @@ +# IaC Reverse Engineering Tool — User Walkthrough + +> Bring your unmanaged on-premises infrastructure under Terraform control in minutes. + +--- + +## What This Tool Does + +``` +┌─────────────────────────────────────────────────────────────────────────┐ +│ │ +│ Your Live Infrastructure Generated Terraform Output │ +│ │ +│ ┌──────────────┐ ┌──────────────────────┐ │ +│ │ Kubernetes │──┐ │ kubernetes_deploy.tf │ │ +│ │ Cluster │ │ │ kubernetes_svc.tf │ │ +│ └──────────────┘ │ │ docker_service.tf │ │ +│ ┌──────────────┐ │ iac-reverse │ windows_service.tf │ │ +│ │ Docker Swarm │──┼──────────────▶│ synology_volume.tf │ │ +│ │ Cluster │ │ generate │ harvester_vm.tf │ │ +│ └──────────────┘ │ │ variables.tf │ │ +│ ┌──────────────┐ │ │ providers.tf │ │ +│ │ Windows │──┤ │ terraform.tfstate │ │ +│ │ Servers │ │ └──────────────────────┘ │ +│ └──────────────┘ │ │ +│ ┌──────────────┐ │ │ +│ │ Synology NAS │──┤ │ +│ └──────────────┘ │ │ +│ ┌──────────────┐ │ │ +│ │ Harvester │──┘ │ +│ │ HCI │ │ +│ └──────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────────────────┘ +``` + +--- + +## Table of Contents + +1. [Prerequisites](#prerequisites) +2. [Installation](#installation) +3. [Quick Start](#quick-start) +4. [Creating Scan Profiles](#creating-scan-profiles) +5. [Running a Scan](#running-a-scan) +6. [Generating Terraform Code](#generating-terraform-code) +7. [Incremental Scanning (Diff)](#incremental-scanning-diff) +8. [Validating Output](#validating-output) +9. [Authentik SSO Login](#authentik-sso-login) +10. [Supported Providers](#supported-providers) +11. [Troubleshooting](#troubleshooting) + +--- + +## Prerequisites + +Before installing, make sure you have: + +| Requirement | Version | Purpose | +|---|---|---| +| Python | 3.11+ | Runtime | +| pip | Latest | Package installation | +| Terraform | 1.5+ | Output validation (optional) | +| Git | Any | Cloning the repo | + +**Platform-specific requirements for scanning:** + +| Provider | Requirement | +|---|---| +| Kubernetes | Valid kubeconfig file | +| Docker Swarm | Docker daemon access (TCP or socket) | +| Windows | WinRM enabled on target machines | +| Synology | DSM admin credentials | +| Harvester | Harvester cluster kubeconfig | +| Bare Metal | BMC/iDRAC Redfish API access | + +--- + +## Installation + +### Step 1: Clone the repository + +```bash +git clone https://github.com/your-org/SnarfCode.git +cd SnarfCode +``` + +### Step 2: Install the tool + +```bash +# Install in development mode (recommended) +pip install -e ".[dev]" +``` + +### Step 3: Verify installation + +```bash +iac-reverse --version +``` + +Expected output: +``` +iac-reverse, version 0.1.0 +``` + +``` +┌─────────────────────────────────────────────────────────────┐ +│ ✓ Installation Complete │ +│ │ +│ You now have access to: │ +│ • iac-reverse scan Discover infrastructure │ +│ • iac-reverse generate Full pipeline (scan → HCL) │ +│ • iac-reverse diff Incremental change detection │ +│ • iac-reverse validate Validate generated output │ +│ • iac-reverse login Authentik SSO authentication │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## Quick Start + +The fastest way to get going — scan your Kubernetes cluster and generate Terraform: + +```bash +# 1. Create a scan profile +cat > my-cluster.yaml << EOF +provider: kubernetes +credentials: + kubeconfig_path: ~/.kube/config + context: my-cluster +endpoints: + - https://k8s-api.internal.lab:6443 +EOF + +# 2. Generate Terraform code +iac-reverse generate --profile my-cluster.yaml --output-dir ./terraform-output + +# 3. Check the results +ls terraform-output/ +``` + +That's it! You now have Terraform HCL + state for your cluster. + +--- + +## Creating Scan Profiles + +Scan profiles are YAML files that tell the tool what to scan and how to authenticate. + +### Profile Structure + +```yaml +provider: # Required: kubernetes, docker_swarm, synology, etc. +credentials: # Required: provider-specific auth + key: value +endpoints: # Optional: API endpoints to scan + - https://api.example.com:6443 +resource_type_filters: # Optional: limit which resource types to discover + - kubernetes_deployment + - kubernetes_service +authentik_token: # Optional: SSO token from Authentik +``` + +### Environment Variable Expansion + +Use `${ENV_VAR}` syntax in credential fields to avoid hardcoding secrets: + +```yaml +provider: synology +credentials: + host: nas01.internal.lab + port: "5001" + username: "${SYNOLOGY_USER}" + password: "${SYNOLOGY_PASSWORD}" + use_ssl: "true" +``` + +You can also provide defaults with `${ENV_VAR:-default_value}`: + +```yaml +credentials: + port: "${SYNOLOGY_PORT:-5001}" +``` + +### Example Profiles + +
+Kubernetes (Raspberry Pi Cluster) + +```yaml +provider: kubernetes +credentials: + kubeconfig_path: "${HOME}/.kube/config" + context: "pi-cluster" +endpoints: + - "https://k8s-api.internal.lab:6443" +resource_type_filters: + - kubernetes_deployment + - kubernetes_service + - kubernetes_ingress + - kubernetes_config_map + - kubernetes_persistent_volume + - kubernetes_namespace +``` +
+ +
+Docker Swarm + +```yaml +provider: docker_swarm +credentials: + host: "tcp://swarm-manager.internal.lab:2376" + tls_verify: "true" + cert_path: "${HOME}/.docker/certs" +``` +
+ +
+Windows Server + +```yaml +provider: windows +credentials: + host: "win-server-01.internal.lab" + username: "${WINDOWS_USER}" + password: "${WINDOWS_PASSWORD}" + transport: "ntlm" + port: "5986" + use_ssl: "true" +resource_type_filters: + - windows_service + - windows_scheduled_task + - windows_iis_site + - windows_iis_app_pool + - windows_feature + - windows_hyperv_vm +``` +
+ +
+Synology NAS + +```yaml +provider: synology +credentials: + host: "nas01.internal.lab" + port: "5001" + username: "${SYNOLOGY_USER}" + password: "${SYNOLOGY_PASSWORD}" + use_ssl: "true" +resource_type_filters: + - synology_shared_folder + - synology_volume + - synology_storage_pool +``` +
+ +
+SUSE Harvester (Dell PowerEdge) + +```yaml +provider: harvester +credentials: + kubeconfig_path: "${HOME}/.kube/harvester-config" + context: "harvester-cluster" +endpoints: + - "https://harvester.internal.lab:6443" +``` +
+ +
+Bare Metal (IPMI/Redfish) + +```yaml +provider: bare_metal +credentials: + host: "bmc-server01.internal.lab" + username: "${BMC_USER}" + password: "${BMC_PASSWORD}" + port: "443" + use_ssl: "true" +``` +
+ +
+Multi-Provider (scan everything at once) + +```yaml +- provider: kubernetes + credentials: + kubeconfig_path: ~/.kube/config + context: pi-cluster + endpoints: + - https://k8s-api.internal.lab:6443 + +- provider: synology + credentials: + host: nas01.internal.lab + port: "5001" + username: "${SYNOLOGY_USER}" + password: "${SYNOLOGY_PASSWORD}" + +- provider: windows + credentials: + host: win-server-01.internal.lab + username: "${WINDOWS_USER}" + password: "${WINDOWS_PASSWORD}" + transport: ntlm + port: "5986" + use_ssl: "true" +``` +
+ +--- + +## Running a Scan + +The `scan` command discovers resources without generating any output files. Useful for previewing what the tool will find. + +```bash +iac-reverse scan --profile my-cluster.yaml +``` + +**Example output:** + +``` +Loading scan profile: my-cluster.yaml +Provider: kubernetes +Creating plugin... +Starting scan... + [1/6] Scanning kubernetes_deployment... (12 resources found) + [2/6] Scanning kubernetes_service... (18 resources found) + [3/6] Scanning kubernetes_ingress... (4 resources found) + [4/6] Scanning kubernetes_config_map... (23 resources found) + [5/6] Scanning kubernetes_persistent_volume... (6 resources found) + [6/6] Scanning kubernetes_namespace... (5 resources found) + +Scan complete: 68 resources discovered +``` + +``` +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ Pipeline Flow: scan command │ +│ │ +│ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ +│ │ Profile │───▶│ Scanner │───▶│ Resource Summary │ │ +│ │ (YAML) │ │ │ │ (terminal) │ │ +│ └──────────┘ └──────────┘ └──────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## Generating Terraform Code + +The `generate` command runs the full pipeline: scan → resolve dependencies → generate HCL → build state → validate. + +```bash +iac-reverse generate --profile my-cluster.yaml --output-dir ./terraform-output +``` + +**Example output:** + +``` +Loading scan profile: my-cluster.yaml +Step 1/5: Scanning infrastructure... + [1/6] Scanning kubernetes_deployment... (12 resources found) + [2/6] Scanning kubernetes_service... (18 resources found) + ... + Found 68 resources +Step 2/5: Resolving dependencies... + Resolved 42 relationships, 0 cycles detected +Step 3/5: Generating Terraform code... + Generated 6 resource files +Step 4/5: Building Terraform state... + State file: 68 entries +Step 5/5: Validating output... + ✓ Validation passed + +Generation complete: + Output directory: ./terraform-output + Resource files: 6 + Total resources: 68 +``` + +**Generated file structure:** + +``` +terraform-output/ +├── kubernetes_deployment.tf # All deployments +├── kubernetes_service.tf # All services +├── kubernetes_ingress.tf # All ingresses +├── kubernetes_config_map.tf # All config maps +├── kubernetes_persistent_volume.tf +├── kubernetes_namespace.tf +├── variables.tf # Extracted shared variables +├── providers.tf # Provider configuration +└── terraform.tfstate # State binding to live resources +``` + +``` +┌─────────────────────────────────────────────────────────────────────┐ +│ │ +│ Pipeline Flow: generate command │ +│ │ +│ ┌────────┐ ┌─────────┐ ┌──────────┐ ┌───────┐ ┌───────────┐ │ +│ │Profile │─▶│ Scanner │─▶│ Resolver │─▶│ Code │─▶│ State │ │ +│ │ YAML │ │ │ │ │ │ Gen │ │ Builder │ │ +│ └────────┘ └─────────┘ └──────────┘ └───────┘ └───────────┘ │ +│ │ │ +│ ▼ │ +│ ┌───────────┐ │ +│ │ Validator │ │ +│ └───────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────────────┘ +``` + +--- + +## Incremental Scanning (Diff) + +After your initial scan, use `diff` to detect infrastructure changes without regenerating everything. + +```bash +iac-reverse diff --profile my-cluster.yaml +``` + +**Example output:** + +``` +Loading scan profile: my-cluster.yaml +Loading previous snapshot... + Previous snapshot: 2024-01-14T09:00:00Z (65 resources) +Scanning infrastructure... + [1/6] Scanning kubernetes_deployment... (14 resources found) + ... +Comparing with previous scan... + Snapshot saved + +Change Summary: + Added: 3 + Removed: 1 + Modified: 2 + + + kubernetes_deployment/new-api-service + + kubernetes_service/new-api-svc + + kubernetes_ingress/new-api-ingress + - kubernetes_deployment/deprecated-worker + ~ kubernetes_deployment/web-frontend (replicas: 3 → 5) + ~ kubernetes_service/web-svc (port: 8080 → 9090) +``` + +``` +┌─────────────────────────────────────────────────────────────┐ +│ │ +│ Incremental Scan Flow │ +│ │ +│ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ +│ │ Previous │ │ Current │ │ │ │ +│ │ Snapshot │───▶│ Scan │───▶│ Change Summary │ │ +│ │ (JSON) │ │ │ │ + Added │ │ +│ └──────────┘ └──────────┘ │ - Removed │ │ +│ │ ~ Modified │ │ +│ └──────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## Validating Output + +Run standalone validation against existing Terraform output: + +```bash +iac-reverse validate --dir ./terraform-output +``` + +**Example output (success):** + +``` +Validating: ./terraform-output + +Validation Results: + terraform init: ✓ + terraform validate: ✓ + terraform plan: ✓ + +✓ All validations passed - no drift detected +``` + +**Example output (drift detected):** + +``` +Validating: ./terraform-output + +Validation Results: + terraform init: ✓ + terraform validate: ✓ + terraform plan: ✗ + +Planned Changes (2): + modify: kubernetes_deployment.web_frontend + add: kubernetes_service.new_backend + +⚠ Validation passed but drift detected +``` + +--- + +## Authentik SSO Login + +If your infrastructure uses Authentik for identity management, authenticate first: + +```bash +iac-reverse login \ + --url https://auth.internal.lab \ + --client-id iac-reverse-tool \ + --client-secret +``` + +**Example output:** + +``` +Authenticating with Authentik at https://auth.internal.lab... +✓ Authenticated as user: admin@internal.lab + Groups: admins, infra-team + Token stored in .iac-reverse/token +``` + +The stored token is automatically used by subsequent `scan` and `generate` commands when `authentik_token` is referenced in your profile. + +--- + +## Supported Providers + +``` +┌─────────────────────────────────────────────────────────────────────────┐ +│ │ +│ Provider Platform Type Architecture Resources │ +│ ───────────────────────────────────────────────────────────────────── │ +│ kubernetes Container Orch. ARM/AArch64 6 types │ +│ docker_swarm Container Orch. ARM/AArch64 5 types │ +│ synology Storage Appliance ARM/AMD64 5 types │ +│ harvester HCI AMD64 4 types │ +│ bare_metal Bare Metal AMD64 4 types │ +│ windows Windows AMD64 13 types │ +│ │ +│ Total: 37 resource types across 6 providers │ +│ │ +└─────────────────────────────────────────────────────────────────────────┘ +``` + +### Resource Types by Provider + +| Provider | Resource Types | +|---|---| +| **Kubernetes** | deployment, service, ingress, config_map, persistent_volume, namespace | +| **Docker Swarm** | service, network, volume, config, secret | +| **Synology** | shared_folder, volume, storage_pool, replication_task, user | +| **Harvester** | virtualmachine, volume, image, network | +| **Bare Metal** | hardware, bmc_config, network_interface, raid_config | +| **Windows** | service, scheduled_task, iis_site, iis_app_pool, network_adapter, firewall_rule, installed_software, feature, hyperv_vm, hyperv_switch, dns_record, local_user, local_group | + +--- + +## Troubleshooting + +### Common Issues + +| Problem | Solution | +|---|---| +| `Terraform binary not found` | Install Terraform and add to PATH | +| `Authentication failed for provider 'kubernetes'` | Check kubeconfig path and context name | +| `WinRM is not enabled or unreachable` | Enable WinRM on the target Windows machine | +| `Connection refused` for Docker | Verify Docker daemon is running and accessible | +| `Environment variable 'X' is not set` | Export the required env var or add a default in the profile | + +### Enabling WinRM on Windows targets + +```powershell +# Run on the target Windows machine (as Administrator) +Enable-PSRemoting -Force +winrm set winrm/config/service '@{AllowUnencrypted="false"}' +winrm set winrm/config/service/auth '@{Basic="true"}' +``` + +### Checking connectivity + +```bash +# Test Kubernetes access +kubectl --kubeconfig ~/.kube/config --context my-cluster get nodes + +# Test Docker Swarm access +docker -H tcp://swarm-manager:2376 node ls + +# Test WinRM access (from Linux) +python -c "import winrm; s = winrm.Session('https://win-server:5986/wsman', auth=('user','pass'), transport='ntlm', server_cert_validation='ignore'); print(s.run_ps('hostname').std_out)" +``` + +### Getting help + +```bash +# General help +iac-reverse --help + +# Command-specific help +iac-reverse scan --help +iac-reverse generate --help +iac-reverse diff --help +iac-reverse validate --help +iac-reverse login --help +``` + +--- + +## Architecture Overview + +``` +┌─────────────────────────────────────────────────────────────────────────┐ +│ iac-reverse CLI │ +├─────────────────────────────────────────────────────────────────────────┤ +│ │ +│ ┌─────────────────────────────────────────────────────────────────┐ │ +│ │ Pipeline Engine │ │ +│ │ │ │ +│ │ ┌─────────┐ ┌──────────┐ ┌─────────┐ ┌───────┐ ┌───────┐ │ │ +│ │ │Scanner │─▶│Dependency│─▶│ Code │─▶│ State │─▶│Valida-│ │ │ +│ │ │ │ │ Resolver │ │Generator│ │Builder│ │ tor │ │ │ +│ │ └────┬────┘ └──────────┘ └─────────┘ └───────┘ └───────┘ │ │ +│ │ │ │ │ +│ └───────┼──────────────────────────────────────────────────────────┘ │ +│ │ │ +│ ┌───────┼──────────────────────────────────────────────────────────┐ │ +│ │ │ Provider Plugins │ │ +│ │ ┌────┴─────┬──────────┬──────────┬──────────┬──────────┐ │ │ +│ │ │Kubernetes│ Docker │ Synology │Harvester │ Windows │ │ │ +│ │ │ Plugin │ Plugin │ Plugin │ Plugin │ Plugin │ │ │ +│ │ └──────────┴──────────┴──────────┴──────────┴──────────┘ │ │ +│ │ │ │ +│ └───────────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌───────────────────────────────────────────────────────────────────┐ │ +│ │ Incremental Scan Engine │ │ +│ │ ┌──────────────┐ ┌────────────────┐ ┌─────────────────────┐ │ │ +│ │ │Snapshot Store│ │Change Detector │ │Incremental Updater │ │ │ +│ │ └──────────────┘ └────────────────┘ └─────────────────────┘ │ │ +│ └───────────────────────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────────────────┘ +``` + +--- + +## Tips & Best Practices + +1. **Start small** — Scan one provider at a time before combining into multi-provider profiles +2. **Use resource filters** — Limit initial scans to specific resource types to keep output manageable +3. **Store secrets in env vars** — Never hardcode passwords in profile YAML files +4. **Run diff regularly** — Set up a cron job or CI pipeline to detect infrastructure drift +5. **Review generated code** — The tool generates a starting point; review and customize before using in production +6. **Version control your profiles** — Keep scan profiles in git alongside your generated Terraform code + +--- + +*Built with Python 3.11+ • Terraform HCL output • Property-based testing with Hypothesis*