Lab 4: SNS + Lambda Failover

Lab 4: SNS + Lambda Failover

Estructura del módulo

04_automation/
├── main.tf
├── variables.tf
├── outputs.tf
└── lambda/
    ├── handler.py
    └── failback_handler.py

Paso 1: Setup

source ~/route53-arc-tf/global.env
cd ~/route53-arc-tf/04_automation
mkdir -p lambda

# Verificar que NOTIFICATION_EMAIL está configurado
# (se definió en global.env en el Lab 0 — editalo ahí si no lo hiciste)
echo "📧 Email de notificación: $NOTIFICATION_EMAIL"
[ -z "$NOTIFICATION_EMAIL" ] || [ "$NOTIFICATION_EMAIL" = "tu-email@ejemplo.com" ] \
  && echo "⚠️  EDITÁ el NOTIFICATION_EMAIL en ~/route53-arc-tf/global.env antes de continuar" \
  && echo "     Luego ejecutá: source ~/route53-arc-tf/global.env"

# export NOTIFICATION_PHONE="+54911XXXXXXXX"  # opcional

Paso 2: Código Lambda

lambda/handler.py

¿Para qué sirve este archivo?
Es el código de la función Python que ejecutará la conmutación automática de tráfico (failover). Cuando se activa por una notificación de alerta SNS:

  1. Parsea el mensaje recibido de CloudWatch.
  2. Se conecta al endpoint del cluster ARC mediante boto3.
  3. Cambia el estado del Routing Control Primario a Off y el Secundario a On.
  4. Envía un correo electrónico/SMS notificando al equipo de ingeniería que el failover fue ejecutado.
cat > lambda/handler.py << 'EOF'
"""
AWS Route 53 ARC Workshop (Terraform)
Lambda: Failover Handler & Recovery Notifier
"""
import json, os, boto3, logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

PRIMARY_RC_ARN   = os.environ["PRIMARY_RC_ARN"]
SECONDARY_RC_ARN = os.environ["SECONDARY_RC_ARN"]
CLUSTER_ENDPOINT = os.environ["CLUSTER_ENDPOINT"]
SNS_TOPIC_ARN    = os.environ["SNS_TOPIC_ARN"]
FAILBACK_URL     = os.environ.get("FAILBACK_URL", "")
REGION           = os.environ.get("AWS_REGION", "us-east-1")

def arc_client():
    return boto3.client("route53-recovery-cluster",
                        endpoint_url=CLUSTER_ENDPOINT, region_name=REGION)

def get_state(client, arn):
    return client.get_routing_control_state(
        RoutingControlArn=arn)["RoutingControlState"]

def execute_failover(client):
    client.update_routing_control_states(
        UpdateRoutingControlStateEntries=[
            {"RoutingControlArn": PRIMARY_RC_ARN,   "RoutingControlState": "Off"},
            {"RoutingControlArn": SECONDARY_RC_ARN, "RoutingControlState": "On"},
        ]
    )
    logger.info("Failover automático ejecutado")

def notify(subject, message):
    boto3.client("sns", region_name=REGION).publish(
        TopicArn=SNS_TOPIC_ARN, Subject=subject, Message=message)

def lambda_handler(event, context):
    logger.info(json.dumps(event))
    alarm_state = "UNKNOWN"
    try:
        for r in event.get("Records", []):
            if r.get("EventSource") == "aws:sns":
                msg_body = json.loads(r["Sns"]["Message"])
                alarm_state = msg_body.get("NewStateValue", "UNKNOWN")
    except Exception as e:
        logger.warning(f"Parse error: {e}")
        alarm_state = event.get("alarm_state", "UNKNOWN")

    arc = arc_client()

    # 🟢 CASO 1: Sitio primario se recuperó (Estado OK) -> Notificar para Failback Manual
    if alarm_state == "OK":
        if get_state(arc, PRIMARY_RC_ARN) == "Off":
            logger.info("Sitio primario recuperado (OK). Enviando alerta para Failback Manual...")
            notify(
                subject="🟢 [SITIO PRIMARIO RECUPERADO] Listo para Failback Manual",
                message=(
                    "AVISO DE RECUPERACIÓN:\n"
                    "El sitio primario vuelve a estar SALUDABLE (Health Check OK).\n\n"
                    "⚠️ El tráfico SE MANTIENE en el sitio de contingencia (Amplify) por política de seguridad.\n\n"
                    "Acción requerida por el operador:\n"
                    f"Para devolver el tráfico al sitio primario, hacé clic en este enlace (1-Click Failback):\n"
                    f"  👉 {FAILBACK_URL}\n\n"
                    "O ejecutá el comando de Failback Manual (Lab 6):\n"
                    "  aws route53-recovery-cluster update-routing-control-states ...\n\n"
                    "AWS Route 53 ARC Workshop (Terraform)"
                ),
            )
            return {"statusCode": 200, "body": "RECOVERY_NOTIFICATION_SENT"}
        else:
            logger.info("Sitio primario OK y ya activo. Sin acción requerida.")
            return {"statusCode": 200, "body": "PRIMARY_ALREADY_ON"}

    # 🚨 CASO 2: Sitio primario caído (Estado ALARM) -> Ejecutar Failover Automático
    if alarm_state == "ALARM":
        if get_state(arc, PRIMARY_RC_ARN) == "Off":
            logger.info("Ya en failover, sin cambios.")
            return {"statusCode": 200, "body": "Already failed over"}

        execute_failover(arc)
        notify(
            subject="🚨 [FAILOVER ACTIVADO] Sitio primario caído",
            message=(
                "ALERTA DE INCIDENTE:\n"
                "Failover automático ejecutado por caída del sitio primario.\n"
                "- Routing Control Primario: OFF\n"
                "- Routing Control Secundario: ON\n\n"
                "El tráfico se redirigió automáticamente al sitio de contingencia (Amplify).\n"
                "AWS Route 53 ARC Workshop (Terraform)"
            ),
        )
        return {"statusCode": 200, "body": "FAILOVER_EXECUTED"}

    return {"statusCode": 200, "body": f"No action for state: {alarm_state}"}
EOF

### `lambda/failback_handler.py`

> **¿Para qué sirve este archivo?**  
> Es el código de la función Python que restablecerá el tráfico al primario mediante un click.

```bash
cat > lambda/failback_handler.py << 'EOF'
"""
AWS Route 53 ARC Workshop (Terraform)
Lambda: 1-Click Failback Handler (Function URL / HTTP Endpoint)
"""
import json, os, boto3, logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

PRIMARY_RC_ARN   = os.environ["PRIMARY_RC_ARN"]
SECONDARY_RC_ARN = os.environ["SECONDARY_RC_ARN"]
CLUSTER_ENDPOINT = os.environ["CLUSTER_ENDPOINT"]
ALARM_NAME       = os.environ["ALARM_NAME"]
REGION           = os.environ.get("AWS_REGION", "us-east-1")

def arc_client():
    return boto3.client("route53-recovery-cluster",
                        endpoint_url=CLUSTER_ENDPOINT, region_name=REGION)

def execute_failback():
    arc = arc_client()
    arc.update_routing_control_states(
        UpdateRoutingControlStateEntries=[
            {"RoutingControlArn": PRIMARY_RC_ARN,   "RoutingControlState": "On"},
            {"RoutingControlArn": SECONDARY_RC_ARN, "RoutingControlState": "Off"},
        ]
    )
    # Restablecer estado de alarma CloudWatch a OK
    cw = boto3.client("cloudwatch", region_name=REGION)
    cw.set_alarm_state(
        AlarmName=ALARM_NAME,
        StateValue="OK",
        StateReason="Failback 1-Click completado exitosamente por el operador"
    )
    logger.info("Failback manual ejecutado con éxito")

def lambda_handler(event, context):
    logger.info("Solicitud de Failback 1-Click recibida")
    try:
        execute_failback()
        html_response = """<!DOCTYPE html>
<html lang="es">
<head>
  <meta charset="UTF-8">
  <title>Failback Completado – AWS Workshop</title>
  <style>
    body { font-family: 'Segoe UI', Arial, sans-serif; background: #0f172a; color: #e2e8f0; text-align: center; padding: 60px 20px; }
    .card { background: #1e293b; max-width: 550px; margin: 0 auto; padding: 40px; border-radius: 12px; box-shadow: 0 4px 20px rgba(0,0,0,0.5); }
    h1 { color: #22c55e; font-size: 2.2em; margin-bottom: 16px; }
    p { font-size: 1.1em; line-height: 1.6; color: #94a3b8; }
    .badge { display: inline-block; background: #22c55e; color: #0f172a; font-weight: bold; padding: 8px 24px; border-radius: 20px; margin: 20px 0; }
    .footer { margin-top: 30px; font-size: 0.85em; color: #64748b; }
  </style>
</head>
<body>
  <div class="card">
    <h1>✅ Failback Exitoso</h1>
    <div class="badge">SITIO PRIMARIO ONLINE</div>
    <p>El tráfico ha sido devuelto exitosamente al <strong>Sitio Primario</strong> en AWS (EC2 + ALB).</p>
    <p>Routing Controls:<br>• Primario: <strong>ON</strong><br>• Secundario: <strong>OFF</strong></p>
    <div class="footer">AWS Route 53 ARC Workshop – 1-Click Operations</div>
  </div>
</body>
</html>"""
        return {
            "statusCode": 200,
            "headers": {"Content-Type": "text/html; charset=utf-8"},
            "body": html_response
        }
    except Exception as e:
        logger.error(f"Error al ejecutar failback: {e}")
        return {
            "statusCode": 500,
            "headers": {"Content-Type": "text/html; charset=utf-8"},
            "body": f"<h1>❌ Error al ejecutar Failback</h1><p>{str(e)}</p>"
        }
EOF

cd lambda
zip -q ../failover.zip handler.py
zip -q ../failback.zip failback_handler.py
cd ..
echo "✅ Lambda ZIP: $(ls -lh failover.zip | awk '{print $5}')"

Paso 3: Archivos Terraform

1. variables.tf

¿Para qué sirve este archivo?
Define los parámetros de entrada para la automatización (región AWS, email de destino de alertas y teléfono opcional para SMS).

cat > variables.tf << 'EOF'
variable "aws_region" {
  type    = string
  default = "us-east-1"
}
variable "notification_email" {
  type = string
}
variable "notification_phone" {
  type    = string
  default = ""
}
variable "project_tag" {
  type    = string
  default = "route53-arc"
}
EOF

2. main.tf

¿Para qué sirve este archivo?
Orquesta los recursos de notificación y conmutación automática:

  • Remote State de Lab 3: Importa los ARNs de los Routing Controls y la Alarma base vía data "terraform_remote_state".
  • Tópico SNS & Suscripciones: Despliega el canal de comunicación y suscribe tu email/SMS.
  • Bucket S3 para código: Almacena temporalmente el ZIP del script Python para la función Lambda (force_destroy = true).
  • Rol e IAM Policy de Lambda: Concede permisos para consultar y modificar el estado del cluster ARC (route53-recovery-cluster) y publicar en SNS.
  • Recurso Lambda & Trigger SNS: Despliega la función en Python 3.12, configura sus variables de entorno y conecta la suscripción SNS a la invocación de Lambda.
  • Alarma CloudWatch conectada: Asocia la acción de alarma del Health Check para publicar directamente en el tópico SNS.
cat > main.tf << 'EOF'
terraform {
  required_version = ">= 1.5"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = var.aws_region
  default_tags {
    tags = {
      Workshop  = var.project_tag
      ManagedBy = "Terraform"
    }
  }
}

data "aws_caller_identity" "current" {}

# Leer outputs del Lab 3
data "terraform_remote_state" "arc" {
  backend = "local"
  config  = {
    path = "../03_arc/terraform.tfstate"
  }
}

locals {
  primary_rc_arn   = data.terraform_remote_state.arc.outputs.primary_rc_arn
  secondary_rc_arn = data.terraform_remote_state.arc.outputs.secondary_rc_arn
  # Extraemos el endpoint específico de la región us-east-1 para evitar errores de firma IAM en boto3
  cluster_endpoint = [for e in data.terraform_remote_state.arc.outputs.arc_cluster_endpoints : e.endpoint if e.region == "us-east-1"][0]
  hc_id            = data.terraform_remote_state.arc.outputs.health_check_id
  cw_alarm_base    = data.terraform_remote_state.arc.outputs.cloudwatch_alarm_name
}

# ── SNS Topic ──────────────────────────────────────────────────────────────────
resource "aws_sns_topic" "alerts" {
  name         = "route53-arc-alerts"
  display_name = "Route53 ARC – Alertas"
}

resource "aws_sns_topic_subscription" "email" {
  topic_arn = aws_sns_topic.alerts.arn
  protocol  = "email"
  endpoint  = var.notification_email
}

resource "aws_sns_topic_subscription" "sms" {
  count     = var.notification_phone != "" ? 1 : 0
  topic_arn = aws_sns_topic.alerts.arn
  protocol  = "sms"
  endpoint  = var.notification_phone
}

# ── S3 para código Lambda ──────────────────────────────────────────────────────
resource "aws_s3_bucket" "lambda_code" {
  bucket        = "route53-arc-lambda-${data.aws_caller_identity.current.account_id}"
  force_destroy = true
}

resource "aws_s3_object" "failover_zip" {
  bucket = aws_s3_bucket.lambda_code.id
  key    = "failover-handler.zip"
  source = "${path.module}/failover.zip"
  etag   = filemd5("${path.module}/failover.zip")
}

resource "aws_s3_object" "failback_zip" {
  bucket = aws_s3_bucket.lambda_code.id
  key    = "failback-handler.zip"
  source = "${path.module}/failback.zip"
  etag   = filemd5("${path.module}/failback.zip")
}

# ── IAM Role Lambda ────────────────────────────────────────────────────────────
resource "aws_iam_role" "lambda" {
  name = "route53-arc-lambda-failover-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{ Effect = "Allow"
      Principal = { Service = "lambda.amazonaws.com" }
      Action    = "sts:AssumeRole"
    }]
  })
}

resource "aws_iam_role_policy_attachment" "basic" {
  role       = aws_iam_role.lambda.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_iam_role_policy" "arc_sns" {
  name = "arc-sns-policy"
  role = aws_iam_role.lambda.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      { Effect = "Allow"
        Action = ["route53-recovery-cluster:GetRoutingControlState",
                  "route53-recovery-cluster:UpdateRoutingControlStates"]
        Resource = "*" },
      { Effect = "Allow"
        Action   = "sns:Publish"
        Resource = aws_sns_topic.alerts.arn },
      { Effect = "Allow"
        Action   = "cloudwatch:SetAlarmState"
        Resource = "arn:aws:cloudwatch:${var.aws_region}:${data.aws_caller_identity.current.account_id}:alarm:${local.cw_alarm_base}-with-sns" }
    ]
  })
}

# ── Lambda Function ────────────────────────────────────────────────────────────
resource "aws_lambda_function" "failover" {
  function_name = "route53-arc-failover-handler"
  runtime       = "python3.12"
  handler       = "handler.lambda_handler"
  role          = aws_iam_role.lambda.arn
  timeout       = 60
  memory_size   = 128

  s3_bucket        = aws_s3_bucket.lambda_code.id
  s3_key           = aws_s3_object.failover_zip.key
  source_code_hash = filebase64sha256("${path.module}/failover.zip")

  environment {
    variables = {
      PRIMARY_RC_ARN   = local.primary_rc_arn
      SECONDARY_RC_ARN = local.secondary_rc_arn
      CLUSTER_ENDPOINT = local.cluster_endpoint
      SNS_TOPIC_ARN    = aws_sns_topic.alerts.arn
      FAILBACK_URL     = aws_apigatewayv2_api.failback_api.api_endpoint
    }
  }
}

# ── Lambda Failback (1-Click) ──────────────────────────────────────────────────
resource "aws_lambda_function" "failback" {
  function_name = "route53-arc-failback-handler"
  runtime       = "python3.12"
  handler       = "failback_handler.lambda_handler"
  role          = aws_iam_role.lambda.arn
  timeout       = 30
  memory_size   = 128

  s3_bucket        = aws_s3_bucket.lambda_code.id
  s3_key           = aws_s3_object.failback_zip.key
  source_code_hash = filebase64sha256("${path.module}/failback.zip")

  environment {
    variables = {
      PRIMARY_RC_ARN   = local.primary_rc_arn
      SECONDARY_RC_ARN = local.secondary_rc_arn
      CLUSTER_ENDPOINT = local.cluster_endpoint
      ALARM_NAME       = "${local.cw_alarm_base}-with-sns"
      SNS_TOPIC_ARN    = aws_sns_topic.alerts.arn
    }
  }
}

resource "aws_apigatewayv2_api" "failback_api" {
  name          = "route53-arc-failback-api"
  protocol_type = "HTTP"
  target        = aws_lambda_function.failback.arn
}

resource "aws_lambda_permission" "apigw" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.failback.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "${aws_apigatewayv2_api.failback_api.execution_arn}/*/*"
}

resource "aws_lambda_permission" "sns_invoke" {
  statement_id  = "AllowSNSInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.failover.function_name
  principal     = "sns.amazonaws.com"
  source_arn    = aws_sns_topic.alerts.arn
}

resource "aws_sns_topic_subscription" "lambda_sub" {
  topic_arn = aws_sns_topic.alerts.arn
  protocol  = "lambda"
  endpoint  = aws_lambda_function.failover.arn
}

# ── CloudWatch Alarm con acción SNS ───────────────────────────────────────────
resource "aws_cloudwatch_metric_alarm" "with_action" {
  alarm_name          = "${local.cw_alarm_base}-with-sns"
  alarm_description   = "Sitio primario caído – dispara SNS + Lambda failover"
  namespace           = "AWS/Route53"
  metric_name         = "HealthCheckStatus"
  statistic           = "Minimum"
  period              = 60
  evaluation_periods  = 3
  threshold           = 1
  comparison_operator = "LessThanThreshold"
  treat_missing_data  = "breaching"
  dimensions          = { HealthCheckId = local.hc_id }
  alarm_actions       = [aws_sns_topic.alerts.arn]
  ok_actions          = [aws_sns_topic.alerts.arn]
}
EOF

3. outputs.tf

¿Para qué sirve este archivo?
Exporta el ARN del tópico SNS, el nombre de la función Lambda y el nombre de la alarma CloudWatch para usarlos en el laboratorio de simulación (Lab 5).

cat > outputs.tf << 'EOF'
output "sns_topic_arn" { value = aws_sns_topic.alerts.arn }
output "lambda_name"   { value = aws_lambda_function.failover.function_name }
output "alarm_name"    { value = aws_cloudwatch_metric_alarm.with_action.alarm_name }
output "failback_lambda_name" { value = aws_lambda_function.failback.function_name }
EOF

Paso 4: Desplegar

terraform init

terraform apply \
  -var="notification_email=$NOTIFICATION_EMAIL" \
  -auto-approve
Aviso

📬 Acción obligatoria antes de continuar al Lab 5:
AWS te enviará un email a la dirección configurada en $NOTIFICATION_EMAIL con asunto “AWS Notification - Subscription Confirmation”.
Hacé clic en el enlace “Confirm subscription” de ese email.
Si no lo confirmás, la Lambda de failover no podrá enviarte alertas y el Lab 5 no funcionará correctamente.

Podés verificar el estado de la suscripción con este comando:

SNS_ARN=$(terraform output -raw sns_topic_arn)
SUB_STATUS=$(aws sns list-subscriptions-by-topic \
  --topic-arn $SNS_ARN \
  --query "Subscriptions[?Endpoint=='$NOTIFICATION_EMAIL'].SubscriptionArn" \
  --output text --region $AWS_REGION 2>/dev/null)

if echo "$SUB_STATUS" | grep -q "PendingConfirmation"; then
  echo "⚠️  Suscripción PENDIENTE — revisá tu email y confirmá antes de continuar"
else
  echo "✅ Suscripción activa: $SUB_STATUS"
fi

✅ Verificación del Lab 4

echo "=== Verificación Lab 4 ==="
LAMBDA_NAME=$(terraform output -raw lambda_name)

aws lambda get-function --function-name $LAMBDA_NAME --region $AWS_REGION \
  --query 'Configuration.State' --output text \
  | grep -q Active && echo "✅ Lambda activa" || echo "❌ Lambda no activa"

aws lambda invoke \
  --function-name $LAMBDA_NAME \
  --payload '{"alarm_state":"OK"}' \
  --cli-binary-format raw-in-base64-out \
  /tmp/lambda-test.json --region $AWS_REGION
echo "Respuesta:"; cat /tmp/lambda-test.json; echo

Siguiente paso → Lab 5: Simulación