Lab 5: Simulación de Incidente

No hay código Terraform nuevo en este lab — simulamos la caída utilizando la AWS CLI, obteniendo los parámetros necesarios a partir de los outputs exportados por los módulos de Terraform.

Paso 1: Cargar variables desde el estado Terraform

source ~/route53-arc-tf/global.env

# Outputs del Lab 3 (ARC)
cd ~/route53-arc-tf/03_arc
export PRIMARY_RC_ARN=$(terraform output -raw primary_rc_arn)
export SECONDARY_RC_ARN=$(terraform output -raw secondary_rc_arn)
export CLUSTER_ENDPOINT=$(terraform output -json arc_cluster_endpoints \
  | python3 -c "import json,sys; print(json.load(sys.stdin)[0]['Endpoint'])")

# Outputs del Lab 1 (EC2)
cd ~/route53-arc-tf/01_primary
export EC2_ID=$(terraform output -raw ec2_instance_id)
export PRIMARY_ALB_DNS=$(terraform output -raw alb_dns_name)

# Outputs del Lab 4 (Automation)
cd ~/route53-arc-tf/04_automation
export ALARM_NAME=$(terraform output -raw alarm_name)
export LAMBDA_NAME=$(terraform output -raw lambda_name)

echo "=== Estado inicial ==="
echo "ALB DNS:  $PRIMARY_ALB_DNS"
echo "EC2 ID:   $EC2_ID"
echo "Alarm:    $ALARM_NAME"

# Verificar Routing Controls
echo ""
aws route53-recovery-cluster get-routing-control-state \
  --routing-control-arn $PRIMARY_RC_ARN \
  --endpoint-url $CLUSTER_ENDPOINT --region $AWS_REGION \
  --query 'RoutingControlState' --output text | xargs -I{} echo "Primario:   {}"

aws route53-recovery-cluster get-routing-control-state \
  --routing-control-arn $SECONDARY_RC_ARN \
  --endpoint-url $CLUSTER_ENDPOINT --region $AWS_REGION \
  --query 'RoutingControlState' --output text | xargs -I{} echo "Secundario: {}"

Simulación A: Forzar alarma CloudWatch (inmediato)

echo "=== SIMULACIÓN A: Forzando alarma CloudWatch ==="
echo "Hora: $(date)"

aws cloudwatch set-alarm-state \
  --alarm-name $ALARM_NAME \
  --state-value ALARM \
  --state-reason "Simulación de incidente – AWS Workshop Terraform Workshop" \
  --region $AWS_REGION

echo "⏳ Esperando que Lambda reaccione..."
sleep 15

# Verificar resultado
echo ""
echo "--- Routing Controls post-failover ---"
aws route53-recovery-cluster get-routing-control-state \
  --routing-control-arn $PRIMARY_RC_ARN \
  --endpoint-url $CLUSTER_ENDPOINT --region $AWS_REGION \
  --query 'RoutingControlState' --output text | xargs -I{} echo "Primario:   {} (esperado: Off)"

aws route53-recovery-cluster get-routing-control-state \
  --routing-control-arn $SECONDARY_RC_ARN \
  --endpoint-url $CLUSTER_ENDPOINT --region $AWS_REGION \
  --query 'RoutingControlState' --output text | xargs -I{} echo "Secundario: {} (esperado: On)"

Ver logs de Lambda:

LOG_GROUP="/aws/lambda/$LAMBDA_NAME"
STREAM=$(aws logs describe-log-streams \
  --log-group-name $LOG_GROUP \
  --order-by LastEventTime --descending --max-items 1 \
  --region $AWS_REGION \
  --query 'logStreams[0].logStreamName' --output text)

aws logs get-log-events \
  --log-group-name $LOG_GROUP \
  --log-stream-name "$STREAM" \
  --region $AWS_REGION \
  --query 'events[*].message' --output text | tail -15

Simulación B: Detener nginx (método real)

Primero restaurá el estado normal después del método A:

aws route53-recovery-cluster update-routing-control-states \
  --update-routing-control-state-entries \
    "[{\"RoutingControlArn\":\"$PRIMARY_RC_ARN\",\"RoutingControlState\":\"On\"},
      {\"RoutingControlArn\":\"$SECONDARY_RC_ARN\",\"RoutingControlState\":\"Off\"}]" \
  --endpoint-url $CLUSTER_ENDPOINT --region $AWS_REGION

aws cloudwatch set-alarm-state \
  --alarm-name $ALARM_NAME \
  --state-value OK \
  --state-reason "Estado restaurado" --region $AWS_REGION

echo "✅ Estado normal restaurado"

Ahora detené nginx via SSM:

COMMAND_ID=$(aws ssm send-command \
  --instance-ids $EC2_ID \
  --document-name AWS-RunShellScript \
  --parameters 'commands=["systemctl stop nginx"]' \
  --region $AWS_REGION \
  --query 'Command.CommandId' --output text)

echo "Comando SSM: $COMMAND_ID"
echo "Monitoreando Health Check (puede tardar ~2-3 min)..."
echo ""

for i in $(seq 1 20); do
  PRIMARY_STATE=$(aws route53-recovery-cluster get-routing-control-state \
    --routing-control-arn $PRIMARY_RC_ARN \
    --endpoint-url $CLUSTER_ENDPOINT --region $AWS_REGION \
    --query 'RoutingControlState' --output text 2>/dev/null)

  HTTP=$(curl -s -o /dev/null -w "%{http_code}" \
    --connect-timeout 3 --max-time 5 http://$PRIMARY_ALB_DNS/ 2>/dev/null)

  echo "[$(date '+%H:%M:%S')] ALB: HTTP$HTTP | Primary RC: $PRIMARY_STATE"

  if [ "$PRIMARY_STATE" = "Off" ]; then
    echo ""; echo "🎉 FAILOVER detectado automáticamente"
    break
  fi
  sleep 30
done

✅ Verificación del Lab 5

echo "=== Verificación Lab 5 ==="

P=$(aws route53-recovery-cluster get-routing-control-state \
  --routing-control-arn $PRIMARY_RC_ARN \
  --endpoint-url $CLUSTER_ENDPOINT --region $AWS_REGION \
  --query 'RoutingControlState' --output text 2>/dev/null)

S=$(aws route53-recovery-cluster get-routing-control-state \
  --routing-control-arn $SECONDARY_RC_ARN \
  --endpoint-url $CLUSTER_ENDPOINT --region $AWS_REGION \
  --query 'RoutingControlState' --output text 2>/dev/null)

[ "$P" = "Off" ] && echo "✅ Primario: $P" || echo "⚠️  Primario: $P"
[ "$S" = "On"  ] && echo "✅ Secundario: $S" || echo "⚠️  Secundario: $S"

echo "📬 Revisá tu email para la notificación SNS"

Siguiente paso → Lab 6: Failback