Read Run Summary Lambda

Reads _SUCCESS marker JSON for a given ETL run and returns validation summary for Step Functions to gate promotion.

Purpose

This Lambda function is invoked by Step Functions after ETL completion to check validation outcomes (quarantine rate, condemned rows, critical errors) before allowing promotion to the current/ prefix.

Input

{
  "silver_bucket": "ohpen-silver",
  "silver_prefix": "silver/mortgages/transactions",
  "run_key": "execution-name-from-step-functions",
  "schema_version": "v1"
}

Output

{
  "run_id": "...",
  "quarantined_rows": 123,
  "quarantine_rate": 5.2,
  "condemned_rows": 10,
  "input_rows": 1000,
  "has_critical_errors": false,
  "promotion_allowed": true,
  "quarantine_by_reason": {...}
}

Business Rules

Promotion is allowed if:

• Quarantine rate < 5%
• Condemned rows < 100
• No critical validation errors (business duplicates, circuit breaker)

Deployment

Deployed via Terraform as part of the ETL orchestration infrastructure.