© 2026 Stephen Adei. All rights reserved. All content on this site is the intellectual property of Stephen Adei. See License for terms of use and attribution.
ADR-007: Explicit S3 Lifecycle Rules
Status
Accepted
Context
The system needs to optimize storage costs by transitioning old data to cheaper storage classes (Glacier) while maintaining predictable access patterns.
The following options were considered:
- Explicit lifecycle rules (chosen)
- Intelligent-Tiering (rejected)
- No lifecycle policies (rejected - high storage costs)
Decision
Use explicit S3 lifecycle rules instead of Intelligent-Tiering:
- Silver layer: Transition to Glacier after 90 days
- Quarantine/Condemned: Transition to Glacier after 5 years
Rationale
- Predictable access patterns: Recent data (hot), old data (cold) - explicit rules match usage
- Low object counts: ~50 files/month/bucket - Intelligent-Tiering overhead ($0.0025/1000 objects) not cost-effective
- Cost optimization: Explicit rules provide predictable cost reduction
- Control: Manual tuning enables optimization for specific access patterns
- Simplicity: Explicit rules are easier to understand and audit
Consequences
Positive:
- Predictable costs: Explicit rules provide clear cost optimization
- Control: Manual tuning for specific access patterns
- Simplicity: Easier to understand and audit than Intelligent-Tiering
- Cost-effective: No per-object overhead at current scale
Negative:
- Manual tuning: Requires periodic review and adjustment
- Less adaptive: Doesn't automatically optimize for changing access patterns
Alternatives Considered
Intelligent-Tiering
- Why rejected: Adds $0.0025 per 1000 objects overhead. At current scale (~50 files/month), overhead exceeds benefit. Explicit rules more cost-effective.
No Lifecycle Policies
- Why rejected: High storage costs for old data. Glacier transition reduces costs by 80-90%.
Related Decisions
- Design Decisions Summary - Complete trade-off analysis for this decision
- ADR-001: Parquet-only Format - Lifecycle rules apply to Parquet files
- ADR-004: Quarantine + Condemned Layers - Error layers have longer retention (5 years)
Implementation Evidence
- Code: Terraform lifecycle rules in
main.tflines 184-258 - Documentation: Data Lake Architecture - Lifecycle Policies - Policy design
- Design Decisions: Design Decisions Summary - Trade-off analysis