Analytics em escala: Redshift, EMR, Athena, Lake Formation

Ingestion:
  Kinesis Data Streams / Firehose    — eventos real-time
  DMS                                — CDC de DBs transacionais
  DataSync / Transfer Family         — bulk batch

Storage (data lake):
  S3 (parquet/iceberg, particionado por data)
  S3 Lifecycle: Standard → IA → Glacier

Catalog & governança:
  Glue Data Catalog                   — schemas e partições
  Lake Formation                      — permissões finas + audit

Processing:
  Glue ETL (Spark serverless)        — batch ETL padrão
  EMR Serverless / EMR on EC2        — Spark/Flink customizado
  Lambda                              — ETL leve event-driven
  Step Functions                      — orquestração com retry

Query/analytics:
  Athena (serverless SQL)            — ad-hoc em S3
  Redshift + Spectrum                — warehouse + leitura S3
  QuickSight / Tableau / Looker      — BI layer

Real-time:
  Kinesis Data Analytics (Flink)     — stream processing
  Managed Service for Apache Kafka   — event backbone

-- Lake Formation: fine-grained access
GRANT SELECT ON "sales"."orders"
  TO DATA_LAKE_PRINCIPAL "analyst-role"
  WITH GRANT OPTION;

-- Row-level filter: vendedor só vê orders dele
CREATE DATA FILTER vendor_filter
  ON DATABASE sales
  TABLE orders
  ROW FILTER "vendor_id = current_user_vendor_id()";

-- Column masking para PII
GRANT SELECT (order_id, total, vendor_id) ON "sales"."orders"
  TO "analyst-role";
-- colunas omitidas (customer_email, customer_cpf) ficam invisíveis

-- Tag-based policies
CREATE LF_TAG confidentiality = ['public', 'internal', 'restricted'];
ASSIGN LF_TAG confidentiality='restricted' ON "sales"."orders"."customer_cpf";
GRANT SELECT ON columns with LF_TAG confidentiality='public'
  TO "analyst-role";