The fix for the bill that keeps climbing
The previous article in this series walked through why S3 storage costs keep increasing even when files appear deleted — versioning leaves noncurrent versions and delete markers behind, and they accumulate silently. The cleanup question follows immediately: how do you actually keep this under control without running scripts on a calendar?
The answer is lifecycle policies. They are the part of S3 configuration that most teams set up once at bucket creation, forget, and then years later wish they had set up properly. The good news is the configuration is small. The cost of not having it is large. This article walks through the patterns that work in production.
What a lifecycle policy actually does
A lifecycle policy is a set of rules attached to a bucket that tells S3 to perform actions on objects when they reach certain ages. Each rule specifies:
- A scope (the whole bucket, or objects matching a prefix or tag)
- An action (transition to a different storage class, expire the object, abort incomplete uploads, remove noncurrent versions)
- A trigger (an age in days, or a specific date)
The actions are applied by S3 automatically, every day. There is no Lambda function to maintain, no cron job to monitor, no script to keep current. Once the policy is written and applied, the storage management runs in the background for the life of the bucket.
The catch is that lifecycle policies are deceptively simple to write and easy to write incorrectly. A misconfigured policy can leave the bill exactly where it was, or in rare cases, delete data that should have been retained. The patterns below are the ones we deploy on every production bucket.
Pattern 1: bound the lifetime of noncurrent versions
For any version-enabled bucket, this is the single highest-impact rule. It tells S3 to permanently remove noncurrent versions after a defined retention window — the window where mistakes are typically caught and rollbacks happen.
{
"ID": "expire-noncurrent-versions-after-90-days",
"Status": "Enabled",
"Filter": {},
"NoncurrentVersionExpiration": {
"NoncurrentDays": 90,
"NewerNoncurrentVersions": 5
}
}
NoncurrentDays: 90 removes noncurrent versions older than 90 days. NewerNoncurrentVersions: 5 keeps the 5 most recent noncurrent versions regardless of age, in case the most recent version turns out to be the broken one.
The exact window is a business decision. Document storage with regulatory requirements might keep noncurrent versions for years. A bucket holding daily-rotated application state might only need 7 days. The pattern is the same; the number changes.
Pattern 2: clean up incomplete multipart uploads
Multipart uploads that never complete leave fragments in the bucket that are invisible in the console listing and silently billed. On any bucket that ever receives uploads larger than 5 MB, this rule is non-negotiable.
{
"ID": "abort-incomplete-multipart-uploads",
"Status": "Enabled",
"Filter": {},
"AbortIncompleteMultipartUpload": {
"DaysAfterInitiation": 7
}
}
Seven days is the default we use — long enough that a legitimate large upload retried after a network failure still has time to complete, short enough that genuinely abandoned uploads do not accumulate.
The savings here are small per incident and large in aggregate. A bucket receiving thousands of uploads per day with even a small failure rate accumulates significant invisible storage over a year without this rule.
Pattern 3: transition cold data to cheaper storage classes
S3 Standard is fast and expensive. S3 Standard-IA (infrequent access) is roughly half the storage cost with a retrieval fee. S3 Glacier Instant Retrieval is roughly a fifth of the storage cost. Glacier Flexible Retrieval and Deep Archive go cheaper still, at the cost of retrieval latency measured in minutes or hours.
For any bucket holding data that ages out of frequent access — application logs, document archives, video originals, customer-uploaded files — a transition policy moves objects between classes automatically:
{
"ID": "transition-cold-data",
"Status": "Enabled",
"Filter": { "Prefix": "logs/" },
"Transitions": [
{ "Days": 30, "StorageClass": "STANDARD_IA" },
{ "Days": 90, "StorageClass": "GLACIER_IR" },
{ "Days": 365, "StorageClass": "DEEP_ARCHIVE" }
]
}
Two things to know before deploying transitions.
Minimum storage duration. Each cheaper class has a minimum billing duration — 30 days for Standard-IA, 90 for Glacier IR, 180 for Deep Archive. If you transition an object and then delete it before the minimum, you still pay for the minimum. The default lifecycle pattern (30 → 90 → 365 days) respects these minimums; custom configurations need to check them.
Small objects do not benefit. Standard-IA charges a per-object overhead that makes the transition economics worse than Standard for objects under ~128 KB. The rule above is right for an application that stores meaningful blobs; it is wrong for a bucket of small files. Filter by size if necessary, or batch small files into archives before storage.
For data with unpredictable access patterns, S3 Intelligent-Tiering is the alternative — it monitors access and moves objects between tiers automatically, with no retrieval fees. It is slightly more expensive than getting Standard-IA right manually; it is dramatically cheaper than getting it wrong.
Pattern 4: expire data that has a defined retention period
For data that has a regulatory or business expiry — logs after 7 years, transactional records after 10 years, audit trails after the compliance window — expiration removes the objects permanently when they reach the threshold.
{
"ID": "expire-after-retention-window",
"Status": "Enabled",
"Filter": { "Tag": { "Key": "retention", "Value": "7-years" } },
"Expiration": { "Days": 2555 }
}
The tag filter is the cleaner pattern than prefix filters. It lets the application set retention on the object at write time, and the lifecycle policy handles every retention class in one place. New retention requirements become new tag values rather than new prefixes.
Pattern 5: clean up delete markers without underlying versions
This one is small but worth including. When a delete marker is the only remaining "version" of an object — because the noncurrent versions underneath have all expired — the delete marker itself stays around indefinitely. Each one is tiny; in a large bucket with heavy delete activity, they accumulate.
{
"ID": "remove-expired-delete-markers",
"Status": "Enabled",
"Filter": {},
"Expiration": { "ExpiredObjectDeleteMarker": true }
}
This is set-and-forget housekeeping. It does nothing on a small bucket and quietly cleans up tens of thousands of orphaned markers on a large one.
The patterns combined
A typical production bucket runs four to six rules together. The combined policy might look like:
- Abort incomplete multipart uploads after 7 days
- Transition current objects to Standard-IA after 30 days
- Transition current objects to Glacier IR after 90 days
- Expire noncurrent versions after 90 days (keeping the 5 most recent)
- Expire delete markers when no versions remain
- Expire current objects with retention tag after the regulatory window
Each rule operates independently. S3 applies them daily. The cost curve flattens within a few months as the historical accumulation gets cleaned up and new data gets managed from the moment it arrives.
What lifecycle policies do not solve
Two important boundaries.
They only apply going forward. A lifecycle policy applied to a bucket with five years of accumulated historical versions will start cleaning them up — but at the rate the policy specifies, applied to whichever objects newly cross the threshold. A bucket with terabytes of orphaned noncurrent versions older than the new retention window will see them expire on the next daily run; one with billions of small objects will see the cleanup take a noticeable amount of time. For bulk historical cleanup at scale, S3 Batch Operations is the better tool — covered in the next article.
They do not undo a bad architectural choice. A bucket that should never have had versioning enabled, on data that does not need version history, is better fixed by suspending versioning and removing the existing versions than by indefinitely paying for them. Lifecycle policies make the cost predictable; they do not always make it correct.
The thing to internalize
S3 cost management is not a project. It is a configuration. The teams that have a predictable storage bill have lifecycle policies on every bucket from the day it is created, written deliberately to match the data's lifecycle, and updated when the business retention requirements change. The teams that have a surprising storage bill almost always have one thing in common: the lifecycle policy is empty, or set to defaults nobody chose.
Five rules on a new bucket take 20 minutes to write. Skipping them costs more than you would guess and less than the next article will save.