A cloud cost spike is different from most operational incidents. A service outage triggers alarms immediately — latency goes up, error rates spike, on-call engineers get paged. A cost spike is invisible to operational monitoring. Usage goes up, the meter runs, and finance finds out weeks later when the invoice arrives.
The cost of this delay is not abstract. A misconfigured auto-scaling policy, a runaway data transfer job, or an accidental data scan on a production table can generate $10,000, $50,000, or $100,000 in unexpected charges before anyone notices. The charges are real and are owed to the cloud provider regardless of whether the underlying cause was intentional.
The most common causes of cost spikes
Auto-scaling misconfiguration — An auto-scaling policy with incorrect maximum limits can launch dozens or hundreds of instances in response to a traffic spike or a misconfigured health check. If the health check is incorrectly reporting instances as unhealthy, auto-scaling will keep launching new instances to replace them, consuming compute at a rate that can reach thousands of dollars per hour.
Runaway data transfer — AWS charges for data leaving a region (egress), and for data transferred between services in certain configurations. A misconfigured replication job, an accidental infinite loop in an application, or a traffic routing error that sends all traffic across region boundaries can generate massive data transfer charges.
Unintended data scans — AWS Athena and Amazon Redshift Spectrum charge by the amount of data scanned. A query against an unpartitioned table can scan terabytes of data in minutes. If this query runs repeatedly (as part of an automated job), the charges accumulate rapidly.
Developer or CI/CD mistakes — A developer running a large-scale operation in production (a data backfill, a table rebuild, a large batch job) without understanding the cost implications. A CI/CD pipeline that accidentally targets production with a resource-intensive operation.
Third-party integration failures — An integration with an external service that enters a retry loop after the external service returns errors. Each retry might invoke AWS services (Lambda, API Gateway, SQS), and a retry storm at scale can generate significant charges.
Building anomaly detection that alerts in hours
AWS Cost Anomaly Detection (available via Cost Explorer) is the built-in tool for this. Configure it to monitor total account spend and specific service spend, with a threshold that represents a meaningful departure from normal. An alert when daily spend is 30% above the trailing 7-day average, delivered via SNS to your Slack channel, will catch most spikes within hours.
The key configuration decisions:
Alert threshold — Too low and you get noise from normal variation. Too high and real spikes slip through. For most environments, 20–30% above the 7-day average for a single day is a reasonable threshold. For mission-critical spend categories (compute, data transfer), you can set more sensitive alerts.
Alert routing — Alerts need to reach someone who can act on them in the moment. This means the on-call engineer or the FinOps owner, not a finance team distribution list. The initial response is operational — find what’s causing the spike — not financial.
Granularity — Total account alerts catch big problems but can’t distinguish between a single runaway job and a legitimate traffic increase. Service-level alerts (separate monitors for EC2, data transfer, Athena) provide faster diagnosis by isolating which service is spiking.
The incident response process
When a cost spike alert fires, the response follows the same logic as any operational incident: contain, diagnose, remediate, document.
Contain — Identify the most likely cause and take action to stop the bleeding, even without full diagnosis. Suspected auto-scaling issue: lower the maximum instance count. Suspected runaway job: find and stop it. Suspected data transfer: check traffic routing and block if abnormal.
Diagnose — Use Cost Explorer with daily granularity to identify which service and which resource is driving the spike. For compute spikes, check the instance list in the affected region. For data transfer spikes, check VPC Flow Logs.
Remediate — Fix the underlying cause and verify that the daily cost returns to normal. Don’t close the incident until you have a day of post-fix data confirming the spike has ended.
Document — Write a brief incident summary: what happened, how it was detected, how long it ran, the estimated cost impact, and what change prevents recurrence. File this with the same rigor as a production incident. Finance will need the explanation for the month’s invoice.
What to do if charges have already been incurred
AWS does occasionally waive charges for the first occurrence of a major accidental cost event — particularly for new accounts or when the charge is clearly the result of a misconfiguration rather than intentional use. Contact AWS Support with a clear explanation of what happened, the estimated charge, and what has been done to prevent recurrence. There is no guarantee of a credit, but it’s worth asking for significant charges.
More importantly: don’t let the first indication of a spike be the monthly invoice. The detection-response gap is a process gap, not a technology gap. The tools to detect spikes in near-real-time are available from AWS for free. Building and operating them is the work.
CostDefender surfaces cost anomalies across your AWS footprint — identifying spikes, attributing them to specific services and resources, and alerting owners before the damage compounds.