Troubleshooting Guide

Quick Diagnostics

Initial Health Check

# Run comprehensive diagnostics
terraback doctor

# Expected output for healthy system:
✅ Terraform detected: v1.5.7
✅ Configuration valid
✅ Storage accessible
✅ Network connectivity: OK
✅ Permissions verified
✅ No issues detected

Common Quick Fixes

# Fix most common issues automatically
terraback doctor --fix

# Reset to clean state (preserves backups)
terraback reset --soft

# Complete reset (use with caution)
terraback reset --hard --confirm

Common Issues and Solutions

Installation Issues

Problem: Command not found after installation

Symptoms:

terraback: command not found

Solutions:

Check if binary is installed:

# Linux/macOS
ls -la /usr/local/bin/terraback

# Windows
where terraback

Add to PATH:

# Linux/macOS
export PATH=$PATH:/usr/local/bin
echo 'export PATH=$PATH:/usr/local/bin' >> ~/.bashrc

# Windows PowerShell
[Environment]::SetEnvironmentVariable("Path", $env:Path + ";C:\Program Files\Terraback", [EnvironmentVariableTarget]::User)

Verify installation:

# Reinstall if necessary
curl -sSL https://terraback.io/install.sh | bash

Problem: Permission denied during installation

Symptoms:

Error: Permission denied while installing to /usr/local/bin

Solutions:

# Option 1: Use sudo
sudo curl -sSL https://terraback.io/install.sh | sudo bash

# Option 2: Install to user directory
export TERRABACK_INSTALL_DIR=$HOME/.local/bin
curl -sSL https://terraback.io/install.sh | bash

# Option 3: Fix permissions
sudo chown -R $(whoami) /usr/local/bin

Authentication Issues

Problem: API key not recognized

Symptoms:

Error: Authentication failed: Invalid API key

Solutions:

Verify API key:

# Check current key
echo $TERRABACK_API_KEY

# Re-authenticate
terraback auth logout
terraback auth login

Check key format:

# Ensure no extra spaces or quotes
export TERRABACK_API_KEY="tbk_abc123..."  # Correct
export TERRABACK_API_KEY=tbk_abc123...    # Also correct
export TERRABACK_API_KEY='tbk_abc123...'  # Correct

Verify API endpoint:

# Check if using correct API
terraback config get api.url

# Test connectivity
curl -H "Authorization: Bearer $TERRABACK_API_KEY" \
  https://api.terraback.io/v1/status

Problem: MFA token expired or invalid

Symptoms:

Error: MFA verification failed: Token expired

Solutions:

# Re-initialize MFA
terraback auth mfa reset

# Sync time (token expiry is time-sensitive)
# Linux
sudo ntpdate -s time.nist.gov

# macOS
sudo sntp -sS time.apple.com

# Disable MFA temporarily (if authorized)
terraback auth mfa disable --emergency-code <code>

Storage Issues

Problem: S3 access denied

Symptoms:

Error: Access Denied: Unable to access S3 bucket

Solutions:

Check AWS credentials:

# Verify credentials
aws sts get-caller-identity

# Check specific bucket access
aws s3 ls s3://your-bucket-name/

Verify IAM permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:GetBucketVersioning",
        "s3:PutBucketVersioning"
      ],
      "Resource": [
        "arn:aws:s3:::terraback-bucket",
        "arn:aws:s3:::terraback-bucket/*"
      ]
    }
  ]
}

Check bucket configuration:

# Verify bucket exists
aws s3api head-bucket --bucket terraback-bucket

# Check bucket region
aws s3api get-bucket-location --bucket terraback-bucket

# Update Terraback configuration
terraback config set storage.s3.region us-east-1

Problem: Storage quota exceeded

Symptoms:

Error: Storage quota exceeded: 95% capacity reached

Solutions:

# 1. Check storage usage
terraback storage stats

# 2. Clean up old backups
terraback backup prune --older-than 30d --confirm

# 3. Enable compression
terraback config set backup.compression.enabled true

# 4. Implement lifecycle policies
terraback storage lifecycle create \
  --transition-to-ia 30d \
  --transition-to-glacier 90d \
  --expire 365d

# 5. Increase storage quota
terraback storage resize --size 500GB

Backup Issues

Problem: Backup failing with timeout

Symptoms:

Error: Backup operation timed out after 300 seconds

Solutions:

Increase timeout:

# Increase operation timeout
terraback config set timeouts.backup "30m"

# For large state files
terraback config set performance.chunk_size "50MB"
terraback config set performance.parallel_uploads 10

Optimize state file:

# Remove unnecessary resources
terraform state list
terraform state rm <unnecessary-resource>

# Split large state
terraback state split --max-size 100MB

Use incremental backup:

# Enable incremental backups
terraback config set backup.incremental true
terraback backup create --incremental

Problem: Backup corruption detected

Symptoms:

Error: Backup integrity check failed: Checksum mismatch

Solutions:

# 1. Verify specific backup
terraback backup verify <backup-id>

# 2. List all corrupted backups
terraback backup verify --all --report

# 3. Attempt repair
terraback backup repair <backup-id>

# 4. If unrepairable, quarantine
terraback backup quarantine <backup-id>

# 5. Create fresh backup
terraback backup create --name "recovery-$(date +%Y%m%d)"

Restoration Issues

Problem: Restore failing with state lock

Symptoms:

Error: State is locked by another process

Solutions:

# 1. Check who holds the lock
terraform state show -lock=false

# 2. Force unlock (use cautiously)
terraform force-unlock <lock-id>

# 3. Wait and retry
terraback restore apply <backup-id> \
  --retry 3 \
  --retry-delay 30s

# 4. Use Terraback's lock handling
terraback restore apply <backup-id> \
  --force-unlock \
  --backup-current

Problem: Partial restore failure

Symptoms:

Error: Restore partially applied: 45/50 resources restored

Solutions:

# 1. Get detailed failure report
terraback restore status --detailed

# 2. Retry failed resources only
terraback restore retry --failed-only

# 3. Selective restore of working resources
terraback restore selective <backup-id> \
  --exclude-failed \
  --verify

# 4. Manual intervention for specific resources
terraform import <resource-type>.<name> <resource-id>

Drift Detection Issues

Problem: False positive drift alerts

Symptoms:

Warning: Drift detected in resources that haven't changed

Solutions:

Configure ignore patterns:

# .terraback/config.yml
drift:
  ignore:
    # Ignore timestamp changes
    - "*.updated_at"
    - "*.modified_date"

    # Ignore computed fields
    - "*.arn"
    - "*.id"

    # Ignore specific resources
    - "aws_instance.temp_*"

Tune sensitivity:

# Reduce sensitivity for non-production
terraback drift config \
  --sensitivity low \
  --workspace development

# Ignore known drift
terraback drift acknowledge <drift-id> \
  --reason "Known configuration, will fix in next sprint"

Problem: Drift detection performance issues

Symptoms:

Drift detection taking longer than 10 minutes

Solutions:

# 1. Enable parallel checking
terraback config set drift.parallel_checks true
terraback config set drift.max_workers 10

# 2. Reduce check frequency
terraback drift config --interval 30m

# 3. Limit scope
terraback drift config \
  --include "critical_resources_only" \
  --exclude "development_workspace"

# 4. Use caching
terraback config set drift.cache.enabled true
terraback config set drift.cache.ttl 5m

Network Issues

Problem: Connection timeouts

Symptoms:

Error: Connection timeout: Unable to reach api.terraback.io

Solutions:

Check connectivity:

# Test API endpoint
curl -I https://api.terraback.io/health

# Test storage endpoint
nslookup s3.amazonaws.com

# Check firewall rules
sudo iptables -L -n | grep 443

Configure proxy:

# Set proxy environment
export HTTP_PROXY=http://proxy.company.com:8080
export HTTPS_PROXY=http://proxy.company.com:8080
export NO_PROXY=localhost,127.0.0.1

# Configure Terraback proxy
terraback config set network.proxy.http_proxy $HTTP_PROXY
terraback config set network.proxy.https_proxy $HTTPS_PROXY

Increase timeouts:

terraback config set network.timeouts.connection 60s
terraback config set network.timeouts.read 5m
terraback config set network.retry.max_attempts 5

Problem: SSL certificate errors

Symptoms:

Error: SSL certificate verification failed

Solutions:

# 1. Update CA certificates
# Linux
sudo update-ca-certificates

# macOS
brew install ca-certificates

# 2. For self-signed certificates
terraback config set network.tls.verify false  # Not recommended for production

# 3. Add custom CA
terraback config set network.tls.ca_bundle /path/to/ca-bundle.crt

# 4. Debug SSL issues
openssl s_client -connect api.terraback.io:443 -servername api.terraback.io

Performance Issues

Problem: Slow backup operations

Symptoms:

Backup taking > 10 minutes for 50MB state file

Solutions:

Profile the operation:

# Enable profiling
terraback backup create --profile

# Analyze bottlenecks
terraback debug profile --last-operation

Optimize configuration:

performance:
  compression:
    algorithm: "lz4"  # Faster than gzip
    level: 1  # Fastest compression

  parallelism:
    uploads: 10
    downloads: 10
    workers: 8

  caching:
    enabled: true
    size: "512MB"

Network optimization:

# Enable bandwidth optimization
terraback config set network.optimization.enabled true

# Use regional endpoints
terraback config set storage.s3.endpoint "s3.us-east-1.amazonaws.com"

Problem: High memory usage

Symptoms:

terraback process using > 2GB RAM

Solutions:

# 1. Limit memory usage
terraback config set performance.memory.max_heap "1GB"

# 2. Enable streaming for large files
terraback config set performance.streaming.enabled true
terraback config set performance.streaming.threshold "10MB"

# 3. Disable caching
terraback config set performance.cache.enabled false

# 4. Use disk-based operations
terraback config set performance.use_disk_buffer true

Error Messages Reference

Critical Errors

Error Code	Message	Solution
TBK-001	Configuration file corrupted	Run `terraback config repair`
TBK-002	Storage backend unreachable	Check network and credentials
TBK-003	Invalid state file format	Validate with `terraform validate`
TBK-004	Encryption key missing	Set `TERRABACK_ENCRYPTION_KEY`
TBK-005	License expired	Renew at terraback.io/account

Warning Messages

Warning Code	Message	Action Required
TBK-W001	Backup older than retention policy	Will be auto-deleted
TBK-W002	Storage approaching quota	Increase quota or cleanup
TBK-W003	Drift detected	Review and remediate
TBK-W004	Update available	Run `terraback update`
TBK-W005	Deprecated configuration	Update configuration

Debug Mode

Enable Detailed Logging

# Set debug environment
export TERRABACK_DEBUG=true
export TERRABACK_LOG_LEVEL=debug

# Or via config
terraback config set log.level debug
terraback config set log.verbose true

# Run with debug flag
terraback backup create --debug

# View debug logs
terraback logs show --level debug --tail 100

Generate Debug Report

# Comprehensive debug report
terraback debug report --output debug-$(date +%Y%m%d).tar.gz

# Contents include:
# - System information
# - Configuration (sanitized)
# - Recent logs
# - Network diagnostics
# - Storage connectivity
# - Performance metrics

Recovery Procedures

Emergency Recovery Mode

#!/bin/bash
# emergency-recovery.sh

# Enter emergency mode
terraback emergency --enable

# Bypass all checks (dangerous!)
export TERRABACK_EMERGENCY_MODE=true
export TERRABACK_SKIP_VALIDATION=true

# Attempt recovery
terraback restore apply \
  --latest-known-good \
  --force \
  --skip-checks \
  --no-verify

# Exit emergency mode
terraback emergency --disable

Data Recovery from Corrupted Storage

# 1. List all available backups including corrupted
terraback backup list --include-corrupted

# 2. Attempt repair
terraback storage repair --deep-scan

# 3. Extract partial data
terraback backup extract <backup-id> \
  --partial \
  --output ./recovered/

# 4. Rebuild from fragments
terraback backup rebuild \
  --from-fragments ./recovered/ \
  --output reconstructed-backup

Getting Support

Before Contacting Support

Run diagnostics:

terraback doctor --comprehensive > diagnostics.txt

Collect logs:

terraback support bundle --last-7-days

Document the issue:
- Steps to reproduce
- Expected behavior
- Actual behavior
- Error messages
- Environment details

Support Channels

Community Support

Slack: terraback.io/slack
GitHub Issues: github.com/terraback/terraback/issues
Forum: community.terraback.io

Professional Support

Email: support@terraback.io
Priority Support: support-priority@terraback.io (Pro/Enterprise)
Phone: +1-555-BACKUP-1 (Enterprise only)

Submitting a Support Ticket

# Automated ticket creation
terraback support ticket create \
  --title "Backup fails with timeout" \
  --priority high \
  --attach-diagnostics \
  --attach-logs

# Manual submission with all data
terraback support bundle \
  --comprehensive \
  --encrypt \
  --upload

FAQ

General Issues

Q: How do I completely reset Terraback?

# Soft reset (keeps backups)
terraback reset --soft

# Hard reset (removes everything)
terraback reset --hard --confirm --i-understand-this-will-delete-everything

Q: Can I recover if I lose my encryption key? A: Unfortunately, no. Without the encryption key, encrypted backups cannot be decrypted. Always store encryption keys securely and maintain multiple copies.

Q: Why is my backup size larger than my state file? A: Backups include metadata, versioning information, and sometimes multiple snapshots. Enable compression to reduce size:

terraback config set backup.compression.enabled true
terraback config set backup.compression.level 9

Q: How do I migrate from another backup solution?

# Import existing backups
terraback import \
  --from terraform-cloud \
  --workspace all \
  --map-structure

# Or from local backups
terraback import \
  --from-directory /old/backups/ \
  --format terraform-state

Performance Issues

Q: Terraback is using too much CPU

# Limit CPU usage
terraback config set performance.cpu.max_cores 2
terraback config set performance.nice_level 10

Q: Operations are slow in CI/CD

# Optimize for CI/CD
terraback config set ci_mode true
terraback config set performance.minimal_output true
terraback config set performance.skip_animations true

Security Concerns

Q: How do I rotate encryption keys?

# Rotate keys safely
terraback security rotate-keys \
  --create-backup \
  --verify-all \
  --gradual

Q: Can I audit all operations?

# Enable comprehensive auditing
terraback audit enable --all
terraback audit export --format compliance-report

Quick Diagnostics​

Initial Health Check​

Common Quick Fixes​

Common Issues and Solutions​

Installation Issues​

Problem: Command not found after installation​

Problem: Permission denied during installation​

Authentication Issues​

Problem: API key not recognized​

Problem: MFA token expired or invalid​

Storage Issues​

Problem: S3 access denied​

Problem: Storage quota exceeded​

Backup Issues​

Problem: Backup failing with timeout​

Problem: Backup corruption detected​

Restoration Issues​

Problem: Restore failing with state lock​

Problem: Partial restore failure​

Drift Detection Issues​

Problem: False positive drift alerts​

Problem: Drift detection performance issues​

Network Issues​

Problem: Connection timeouts​

Problem: SSL certificate errors​

Performance Issues​

Problem: Slow backup operations​

Problem: High memory usage​

Error Messages Reference​

Critical Errors​

Warning Messages​

Debug Mode​

Enable Detailed Logging​

Generate Debug Report​

Recovery Procedures​

Emergency Recovery Mode​

Data Recovery from Corrupted Storage​

Getting Support​

Before Contacting Support​

Support Channels​

Community Support​

Professional Support​

Submitting a Support Ticket​

FAQ​

General Issues​

Performance Issues​

Security Concerns​

Quick Diagnostics

Initial Health Check

Common Quick Fixes

Common Issues and Solutions

Installation Issues

Problem: Command not found after installation

Problem: Permission denied during installation

Authentication Issues

Problem: API key not recognized

Problem: MFA token expired or invalid

Storage Issues

Problem: S3 access denied

Problem: Storage quota exceeded

Backup Issues

Problem: Backup failing with timeout

Problem: Backup corruption detected

Restoration Issues

Problem: Restore failing with state lock

Problem: Partial restore failure

Drift Detection Issues

Problem: False positive drift alerts

Problem: Drift detection performance issues

Network Issues

Problem: Connection timeouts

Problem: SSL certificate errors

Performance Issues

Problem: Slow backup operations

Problem: High memory usage

Error Messages Reference

Critical Errors

Warning Messages

Debug Mode

Enable Detailed Logging

Generate Debug Report

Recovery Procedures

Emergency Recovery Mode

Data Recovery from Corrupted Storage

Getting Support

Before Contacting Support

Support Channels

Community Support

Professional Support

Submitting a Support Ticket

FAQ

General Issues

Performance Issues

Security Concerns