Alexanderamiri · Oct 26, 2026 · Oct 26, 2026
@@ -82,6 +82,7 @@ Migration happens later per-app at developer's pace — apps move from old ALB/E
 | `docs/org-runbook.md` | AWS Organizations setup procedure |
 | `docs/cognito-google-setup.md` | Cognito + Google Workspace IdP setup |
 | `docs/apply-gate.md` | Apply gate: credential broker, HMAC overrides, security model |
+| `docs/cost-management.md` | Budgets, anomaly detection, cost-allocation tags, CUR, Config recording, log retention |
 
 ### Terraform — Platform (CI-applied)
 ```
@@ -225,6 +226,8 @@ EventBridge ──► javabin-security SNS ──► slack-alert Lambda:
   IAM / resource / login events   ──► #javabin-infra-alerts
 Cost Anomaly ──► javabin-alerts SNS ──► slack-alert Lambda ──► #javabin-cost-alerts
 
+Account budget (FORECASTED >100%, ACTUAL >100%) ──► javabin-alerts SNS ──► #javabin-cost-alerts
+
 Scheduled:
   Monday 08:00 UTC ──► cost-report ──► #javabin-cost-alerts
   Monday 08:00 UTC ──► securityhub-summary ──► #platform-security-alerts
@@ -316,7 +319,7 @@ Team group OWNERs are admin-managed and never removed by the sync; [email protected]
 - **App databases share one network path**: service-rds allows ingress from the shared ECS tasks SG, so every team's tasks can reach every team's DB port (password still required). Fix: per-app security groups
 - **Team provisioner Lambda**: All sync functions working (Google/GitHub/Budget/Cognito/Identity Center). Password-set flow deployed.
 - **`registered_app_repos` manually managed**: Being replaced with team-scoped IAM roles (repo→team resolved via GitHub API at runtime)
-- **Cost allocation tags pending activation**: `repo`, `created-by`, `commit` tags need activation in Billing console (requires billing data to appear first)
+- **Cost allocation tags**: `team`, `service`, `environment`, `managed-by`, `repo` are active (Terraform). `created-by` and `commit` stay inactive on purpose (no cost signal)
 - **Platform-test-app naming migration**: Existing resources have old `javabin-` prefix names, needs state migration to `{team}-{service}` naming
 
 ## Agent Guidelines

@@ -0,0 +1,119 @@
+# Cost Management
+
+How the platform watches AWS spend, and where each piece lives. All of it is
+platform Terraform or a platform Lambda; nothing here is configured by hand.
+
+Steady state (2026-10): about **$550/month including 25% VAT** (~$440 before
+tax), flat since April 2026. Most of it is fixed cost: load balancers, public
+IPv4 addresses, the NAT gateway, AWS Config, Security Hub and the legacy
+pre-platform stacks. Budgets and Cost Explorer report cost **with tax**.
+
+## Alerts
+
+All cost alerts go to the `javabin-alerts` SNS topic, which the `slack-alert`
+Lambda posts to **#javabin-cost-alerts**.
+
+| Signal | Where | Fires when |
+|---|---|---|
+| Account budget `javabin-account-monthly` | `terraform/platform/cost-analytics/main.tf` | Forecast for the month > 100%; actual > 100%; actual > 150% |
+| Cost Anomaly Detection `javabin-anomaly-monitor` | `terraform/platform/monitoring/main.tf` | Per-service anomaly with total impact >= $10 (IMMEDIATE) |
+| Team budgets `javabin-team-<team>` | `team-provisioner` Lambda (from registry `budget_nok`, default 500 NOK) | Actual > 80% (Slack); actual > 200% triggers `budget-enforcer` |
+| `budget-enforcer` | `terraform/lambda-src/budget_enforcer` | Scales the team's ECS services to `desired_count=0` (nothing is deleted) |
+| `daily-cost-check` / `cost-report` | `terraform/lambda-src/` | Daily spike check; weekly breakdown every Monday |
+
+The account budget limit is `account_budget_usd` in `terraform/platform/main.tf`
+(currently $600, just above steady state). **Lower it as the legacy cleanup
+lands**, otherwise it stops catching drift. A budget that alerts every month is
+noise; one set far above spend is useless.
+
+There is deliberately no CloudWatch `EstimatedCharges` billing alarm. That
+metric needs "Receive Billing Alerts" enabled on the account and only exists in
+us-east-1; the old alarm sat in `INSUFFICIENT_DATA` with an SNS topic nobody
+subscribed to. The budget's FORECASTED notification is the early warning.
+
+A per-team anomaly monitor was considered and skipped: a tag-based
+`aws_ce_anomaly_monitor` needs every team value listed explicitly and is
+replaced whenever the list changes, and per-team budgets already cover it.
+
+## Attribution: cost-allocation tags
+
+`team`, `service`, `environment`, `managed-by` and `repo` are activated as
+cost-allocation tags (`aws_ce_cost_allocation_tag.tags` in
+`monitoring/main.tf`). Apps deployed through `app.yaml` get them from the
+generated module's `default_tags`, and ECS tasks inherit them through
+`propagate_tags = "SERVICE"`.
+
+Spend without a `team` tag is expected for: tax, AWS Config, Security Hub,
+public IPv4 charges, the shared NAT gateway and ALB, and every legacy
+pre-platform stack. In Cost Explorer group by `Tag: team` and read the
+"No tag key" bucket as "platform overhead + legacy".
+
+## Detailed analysis: CUR + Athena
+
+CUR 2.0 (daily, with resource IDs) is exported to `javabin-cur-<account>`,
+crawled by Glue into the `javabin_cur` database, and queryable from the
+`javabin-cost-analytics` Athena workgroup (100 MB scan limit per query).
+Use it for per-resource questions Cost Explorer cannot answer.
+
+## Built-in cost controls
+
+| Control | Where | Setting |
+|---|---|---|
+| Platform Lambda log retention | `terraform/platform/lambdas/log-groups.tf`, `identity/log-groups.tf` | 365 days. Every new Lambda must be added to `local.platform_lambda_names` (`scripts/tests/test_lambda_log_groups.py` enforces it) |
+| App log retention | `terraform/modules/ecs-service` | 30 days |
+| ECR image lifecycle | `terraform/modules/ecr-repo`, CI repos in `compute` | Expire old/untagged images |
+| AWS Config recording | `terraform/platform/monitoring/main.tf` | All supported types; `AWS::EC2::NetworkInterface`, VPC and Subnet recorded daily. See below |
+| Config history bucket | `monitoring/main.tf` | Objects expire after 730 days |
+| CUR bucket | `cost-analytics/main.tf` | IA after 90 days, expire after 365 |
+| Athena results | `cost-analytics/main.tf` | Expire after 7 days |
+
+### Why Config records ENIs daily
+
+AWS Config charges per configuration item ($0.003 each). Every Fargate task
+start/stop creates and deletes an ENI in the platform private subnets, and
+each change re-records the ENI plus its VPC and subnet. In Sept 2026 the
+account recorded ~13k items ($39); ENI, VPC and Subnet were ~7.3k of them
+(~$22), almost all caused by one crash-looping service (`testteam-platform-test-app`,
+~80 task starts a day).
+
+ENIs are **not** excluded from recording, because that would weaken Security
+Hub:
+
+- **EC2.180** (ENI source/destination check) is enabled and its Config rule is
+  scoped to `AWS::EC2::NetworkInterface`. Without ENI recording it goes stale.
+- **Config.1** fails when Config does not record every resource type an
+  enabled control needs. (It already fails today for IAM global types and the
+  custom recorder role; excluding ENIs would add a third reason.)
+
+Instead ENI, VPC and Subnet use a DAILY recording override: at most one item
+per changed resource per day, and EC2.180 still evaluates every ENI that
+exists at the daily snapshot. Short-lived ENIs created and deleted within the
+same day may not be recorded at all. Security groups stay continuous because
+SG changes are security-relevant.
+
+The saving depends on churn. With the crash loop it is roughly $15–20/month;
+once the crash loop is fixed, steady-state churn is a few dollars a month at
+most. Fixing the crashing service is the bigger lever.
+
+After apply, check for 7 days:
+
+- `AWS/Config` `ConfigurationItemsRecorded` per `ResourceType`. If
+  `AWS::EC2::SecurityGroup` dominates, consider adding it to the override.
+- Security Hub **EC2.180** still has fresh evaluations (not NO_DATA).
+- Security Hub **Config.1** has no new reason mentioning
+  `AWS::EC2::NetworkInterface`.
+
+### Account budget
+
+`account_budget_usd` is set about 10% above steady-state spend, with
+FORECASTED >100% and ACTUAL >100% notifications. Lower it each time a cleanup
+from the cost audit lands, or the budget stops catching real overruns.
+
+## Known savings outside platform Terraform
+
+The 2026-10 cost audit (kept outside the repo, in the maintainers'
+`explore/cost-audit-report.md`) lists legacy resources that cost money and are
+not owned by platform Terraform: idle Elastic Beanstalk environments, the
+legacy `Default-ALB` and its public IPs, the Coolify host, the NAT gateway
+design, and Savings Plan timing. Those are cleanup decisions for their owners,
+not things the platform changes automatically.
@@ -2,6 +2,7 @@
 
 Source code in `terraform/lambda-src/{function}/handler.py`. Shared module in `terraform/lambda-src/shared/`.
 Terraform in `terraform/platform/lambdas/`.
+Log groups `/aws/lambda/javabin-{function}` are declared in `lambdas/log-groups.tf` (365-day retention); add every new function to `local.platform_lambda_names` (enforced by `scripts/tests/test_lambda_log_groups.py`).
 
 ## slack-alert
 

@@ -78,10 +78,11 @@ SNS topics, EventBridge rules, Config, GuardDuty, Security Hub, account guardrai
 | `javabin-alerts` SNS | Cost anomalies, budgets |
 | `javabin-security` SNS | IAM changes, GuardDuty, Config, Security Hub |
 | EventBridge rules | IAM key creation, root login, console sign-in, resource creation, security tampering (CloudTrail/Config/GuardDuty/Security Hub/KMS/guardrail disables, plus GuardDuty filters, Security Hub automation rules and control changes, Config recorder changes). `UpdateDetector`, `UpdateServiceSetting` and the filter/rule events also fire on benign changes, including platform applies |
-| AWS Config | Configuration recorder (service-linked role, includes global IAM types) + S3 delivery (TLS-only, 2-year expiry) |
+| AWS Config | Configuration recorder (service-linked role, includes global IAM types; ENI/VPC/Subnet recorded daily) + S3 delivery (TLS-only, 2-year expiry) |
 | Account guardrails | EBS snapshot block-public-access (block all), AMI block-public-access (block new), EBS encryption by default, SSM document public sharing off |
 | IAM Access Analyzer | `javabin-external-access`, external-access findings only (free) |
 | Inspector | ECR + Lambda package scanning (EC2 and Lambda code scanning off). Enabling it switches ECR registry scanning from BASIC to ENHANCED, so `scan_on_push` on repositories (`compute/`, `modules/ecr-repo`) no longer has an effect. Expect a first burst of findings into Security Hub, mostly from legacy Lambdas on deprecated runtimes |
+| Cost Anomaly Detection | SERVICE monitor, >= $10 impact → `javabin-alerts` |
 | GuardDuty | Threat detection |
 | Security Hub | Findings aggregation |
 | `javabin-alert-dedup` DynamoDB | Deduplication table used by slack-alert Lambda |

@@ -0,0 +1,93 @@
+"""Every platform Lambda must have a Terraform-managed log group with retention.
+
+Lambda auto-creates /aws/lambda/<name> with "Never expire" if the group is not
+declared, so a new function added anywhere under terraform/platform without a
+matching log group (local.platform_lambda_names in lambdas/log-groups.tf, or an
+explicit aws_cloudwatch_log_group in its own module) would log forever.
+
+Run: python3 -m unittest discover -s scripts/tests
+"""
+
+import glob
+import os
+import re
+import unittest
+
+ROOT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "..", "terraform", "platform")
+
+
+def _read(*parts):
+    with open(os.path.join(ROOT, *parts)) as f:
+        return f.read()
+
+
+def _toset_names(block):
+    return set(re.findall(r'^\s*"([a-z0-9-]+)",\s*$', block, flags=re.M))
+
+
+def _platform_tf_sources():
+    for path in sorted(glob.glob(os.path.join(ROOT, "**", "*.tf"), recursive=True)):
+        with open(path) as f:
+            yield path, f.read()
+
+
+FUNCTION_NAME = re.compile(r'function_name\s*=\s*"\$\{var\.project\}-([a-z0-9-]+)"')
+EXPLICIT_LOG_GROUP = re.compile(r'name\s*=\s*"/aws/lambda/\$\{var\.project\}-([a-z0-9-]+)"')
+
+
+def lambda_function_names():
+    """Short names of every Lambda declared under terraform/platform."""
+    names = set()
+    for _, src in _platform_tf_sources():
+        names |= set(FUNCTION_NAME.findall(src))
+    return names
+
+
+def shared_log_group_names():
+    """Names in local.platform_lambda_names (the for_each in lambdas/log-groups.tf)."""
+    src = _read("lambdas", "log-groups.tf")
+    block = re.search(r"platform_lambda_names\s*=\s*toset\(\[(.*?)\]\)", src, flags=re.S).group(1)
+    return _toset_names(block)
+
+
+def log_group_names():
+    """Shared for_each names plus explicit /aws/lambda/${var.project}-<name> groups in any module."""
+    names = shared_log_group_names()
+    for _, src in _platform_tf_sources():
+        names |= set(EXPLICIT_LOG_GROUP.findall(src))
+    return names
+
+
+def imported_names():
+    src = _read("imports.tf")
+    block = re.search(
+        r"import\s*\{\s*for_each\s*=\s*toset\(\[(.*?)\]\)\s*to\s*=\s*module\.lambdas\.aws_cloudwatch_log_group\.lambda",
+        src,
+        flags=re.S,
+    ).group(1)
+    return _toset_names(block)
+
+
+class LambdaLogGroupTests(unittest.TestCase):
+    def test_every_function_has_a_log_group(self):
+        functions = lambda_function_names()
+        self.assertTrue(functions, "no aws_lambda_function found; did the naming pattern change?")
+        self.assertIn("auth-claims", functions, "identity module Lambda not discovered; scan is too narrow")
+        missing = functions - log_group_names()
+        self.assertFalse(
+            missing,
+            "declare a log group with retention (local.platform_lambda_names in lambdas/log-groups.tf, "
+            f"or an aws_cloudwatch_log_group in the function's module): {sorted(missing)}",
+        )
+
+    def test_no_log_group_without_function(self):
+        stale = log_group_names() - lambda_function_names()
+        self.assertFalse(stale, f"log groups for functions that no longer exist: {sorted(stale)}")
+
+    def test_imports_only_reference_declared_groups(self):
+        unknown = imported_names() - shared_log_group_names()
+        self.assertFalse(unknown, f"import blocks for undeclared log groups: {sorted(unknown)}")
+
+
+if __name__ == "__main__":
+    unittest.main()
@@ -7,8 +7,11 @@
 #   Glue database       Catalog for CUR data
 #   Glue crawler        Auto-discovers CUR Parquet schema daily
 #   Athena workgroup    Scoped workgroup for cost queries
-#   Billing alarm       CloudWatch alarm on EstimatedCharges (~4-6h delay)
-#   Account budget      Monthly budget with notifications at 80% and 100%
+#   Account budget      Monthly budget: forecast >100%, actual >100%, actual >150%
+#
+# There is no CloudWatch EstimatedCharges alarm: the AWS/Billing metric is not
+# enabled for this account (alarm sat in INSUFFICIENT_DATA from 2026-03 until it
+# was removed). The budget's FORECASTED notification is the early warning.
 ################################################################################
 
 terraform {
@@ -322,40 +325,6 @@ resource "aws_athena_workgroup" "cur" {
   }
 }
 
-################################################################################
-# Billing protection — CloudWatch billing alarm (us-east-1)
-################################################################################
-
-# SNS topic in us-east-1 — CloudWatch billing alarms can only target same-region SNS.
-# A Lambda forwarder (us-east-1 → eu-central-1) can be added later for Slack integration.
-# For now, the budget notifications below go directly to the main alerts topic.
-resource "aws_sns_topic" "billing_alarm" {
-  provider = aws.us_east_1
-  name     = "${var.project}-billing-alarm"
-}
-
-resource "aws_cloudwatch_metric_alarm" "billing" {
-  provider = aws.us_east_1
-
-  alarm_name          = "${var.project}-billing-alarm"
-  alarm_description   = "Account estimated charges exceeded $${var.billing_alarm_threshold_usd}"
-  comparison_operator = "GreaterThanThreshold"
-  evaluation_periods  = 1
-  metric_name         = "EstimatedCharges"
-  namespace           = "AWS/Billing"
-  period              = 21600 # 6 hours
-  statistic           = "Maximum"
-  threshold           = var.billing_alarm_threshold_usd
-  treat_missing_data  = "missing"
-
-  dimensions = {
-    Currency = "USD"
-  }
-
-  alarm_actions = [aws_sns_topic.billing_alarm.arn]
-  ok_actions    = [aws_sns_topic.billing_alarm.arn]
-}
-
 ################################################################################
 # Billing protection — account-level AWS Budget (notifications only)
 ################################################################################
@@ -367,21 +336,30 @@ resource "aws_budgets_budget" "account" {
   limit_unit   = "USD"
   time_unit    = "MONTHLY"
 
-  # 80% warning
+  # Early warning: AWS forecasts the month will end above the budget.
   notification {
     comparison_operator       = "GREATER_THAN"
-    notification_type         = "ACTUAL"
-    threshold                 = 80
+    notification_type         = "FORECASTED"
+    threshold                 = 100
     threshold_type            = "PERCENTAGE"
     subscriber_sns_topic_arns = [var.alerts_topic_arn]
   }
 
-  # 100% critical
+  # Budget exceeded.
   notification {
     comparison_operator       = "GREATER_THAN"
     notification_type         = "ACTUAL"
     threshold                 = 100
     threshold_type            = "PERCENTAGE"
     subscriber_sns_topic_arns = [var.alerts_topic_arn]
   }
+
+  # Critical: spend is 50% over budget. Something is running away.
+  notification {
+    comparison_operator       = "GREATER_THAN"
+    notification_type         = "ACTUAL"
+    threshold                 = 150
+    threshold_type            = "PERCENTAGE"
+    subscriber_sns_topic_arns = [var.alerts_topic_arn]
+  }
 }
@@ -18,16 +18,10 @@ variable "alerts_topic_arn" {
   type        = string
 }
 
-variable "billing_alarm_threshold_usd" {
-  description = "CloudWatch billing alarm threshold in USD"
-  type        = number
-  default     = 200
-}
-
 variable "account_budget_usd" {
-  description = "Account-level monthly budget in USD — notifications at 80% and 100%"
+  description = "Account-level monthly budget in USD, tax included (Cost Explorer UnblendedCost incl. 25% VAT). Notifies on forecast >100%, actual >100% and actual >150%. Steady state is ~$550/mo (2026-10); lower this as the legacy cleanup in docs/cost-management.md lands."
   type        = number
-  default     = 500
+  default     = 600
 }
 
 variable "org_boundary_arn" {

@@ -0,0 +1,13 @@
+# Bounded retention for the auth-claims Lambda's log group (auto-created by
+# Lambda with "Never expire"; adopted by terraform/platform/imports.tf).
+resource "aws_cloudwatch_log_group" "auth_claims" {
+  name              = "/aws/lambda/${var.project}-auth-claims"
+  retention_in_days = 365
+}
+
+# The external pool's trigger (javabin-auth-claims-external) had not logged
+# yet, so its group is created fresh rather than imported.
+resource "aws_cloudwatch_log_group" "auth_claims_external" {
+  name              = "/aws/lambda/${var.project}-auth-claims-external"
+  retention_in_days = 365
+}