Skip to main content

Module: Security & Compliance

Purpose​

This module provides security and compliance information for stakeholder communications. It includes security controls, compliance alignment, and audit capabilities.

Use in​

  • CIO communication (security overview)
  • Legal/Compliance communication (compliance alignment)
  • Security review meetings
  • Compliance documentation

Security Posture​

Data Encryption​

  • In Transit: TLS for all API calls (S3, Glue, Athena)
  • At Rest: S3 encryption with SSE-S3 (AES256) for standard buckets; SSE-KMS with customer-managed keys (CMK) for sensitive buckets (gold, quarantine)
  • Key Management: Customer-managed keys (KMS) implemented for sensitive data with automatic rotation enabled

Access Control​

  • IAM Roles: Least-privilege permissions per layer
  • Prefix-Scoped Access: Bronze/Silver/Gold layers isolated
  • No Public Access: S3 bucket policies enforce private access
  • Audit Logs: CloudTrail logs for all data access

Data Classification​

  • Financial Transaction Data: Sensitive classification
  • PII Handling: Minimal in raw layer, mask/tokenize in curated layers
  • Restricted Access: Raw/quarantine layers limited to Platform + Compliance teams

Compliance Alignment​

Regulatory Requirements​

  • Audit Trail: Immutable Bronze layer (full audit trail)
  • Reproducibility: Run isolation enables reproducible reporting
  • Data Retention: Configurable lifecycle policies (S3)
  • Change Management: Schema versioning, approval workflows

Compliance Features​

  • Immutable Raw Data: Bronze layer never overwritten
  • Run Isolation: Each processing run tracked via run_id
  • Access Control: Least-privilege IAM roles, audit logs
  • Data Classification: Financial data (sensitive), restricted access

Compliance Risk Assessment​

Overall Risk​

Security Controls by Layer​

Bronze Layer (Raw)​

  • Access: Platform team only (write), Compliance team (read)
  • Encryption: S3 encryption at rest
  • Audit: CloudTrail logs all access

Silver Layer (Validated)​

  • Access: Platform team (write), Domain teams (read)
  • Encryption: S3 encryption at rest
  • Audit: CloudTrail logs all access

Gold Layer (Business Reporting)​

  • Access: Platform team (write), Business teams (read)
  • Encryption: S3 encryption at rest
  • Audit: CloudTrail logs all access
  • Scope: Architecture, SQL Breakdown.

Quarantine Layer​

  • Access: Platform team (write), Data Quality + Compliance teams (read)
  • Encryption: S3 encryption at rest
  • Audit: CloudTrail logs all access

Security Review Process​

  1. Architecture Review: Security team reviews architecture design
  2. IAM Policy Review: Security team reviews access control model
  3. Compliance Validation: Legal/Compliance validates audit trail design
  4. Ongoing Audits: Quarterly security audits

Last Updated​

January 2026​

Owner​

Data Platform Team​

Communication Modules​

Task Documentation​

Technical Documentation​

© 2026 Stephen Adei•CC BY 4.0