DevOps4UA

Save Money on AWS S3: Automate Storage Tiering with Terraform

Jenkins

Source: Dev.to | Jan 30, 2026 (UTC)

If you’re storing data in AWS S3, you’ve probably noticed that storage costs can add up quickly. The good news? You can significantly reduce these costs by automatically moving objects that haven’t been accessed in a while to cheaper storage tiers. In this article, I’ll show you how to implement S3 lifecycle policies using Terraform to do exactly that.

Understanding S3 Storage Classes

Before diving into the code, let’s quickly review the S3 storage classes and their use cases:

  • S3 Standard: For frequently accessed data (~$0.023/GB)
  • S3 Standard-IA (Infrequent Access): For data accessed less frequently (~$0.0125/GB)
  • S3 Intelligent-Tiering: Automatically moves data between tiers based on access patterns
  • S3 Glacier Instant Retrieval: For archive data with instant retrieval (~$0.004/GB)
  • S3 Glacier Flexible Retrieval: For archive data with retrieval times from minutes to hours (~$0.0036/GB)
  • S3 Glacier Deep Archive: Lowest cost for long-term archive (~$0.00099/GB)

The cost savings are substantial. Moving from Standard to Standard-IA saves about 46%, and moving to Glacier Deep Archive saves over 95%!

The Problem: Forgotten Data Costing You Money

Many organizations have S3 buckets filled with objects that were uploaded once and rarely (or never) accessed again. Examples include:

  • Old application logs
  • Backup files
  • Historical reports
  • Archived user uploads
  • Legacy project files

Without lifecycle policies, all this data sits in S3 Standard storage, costing you money unnecessarily.

The Solution: Terraform Lifecycle Policies

Let’s create a Terraform configuration that automatically transitions objects to cheaper storage tiers based on their age.

Basic Example: Simple Tiering Strategy

Here’s a straightforward lifecycle policy that moves objects through different storage tiers:

resource "aws_s3_bucket" "cost_optimized_bucket" { bucket = "my-cost-optimized-bucket" } resource "aws_s3_bucket_lifecycle_configuration" "bucket_lifecycle" { bucket = aws_s3_bucket.cost_optimized_bucket.id rule { id = "transition-old-objects" status = "Enabled" transition { days = 30 storage_class = "STANDARD_IA" } transition { days = 90 storage_class = "GLACIER_IR" } transition { days = 180 storage_class = "DEEP_ARCHIVE" } } }

Enter fullscreen mode

Exit fullscreen mode

This configuration:

Advanced Example: Multiple Rules for Different Prefixes

Often, you’ll want different policies for different types of data. Here’s a more sophisticated example:

resource "aws_s3_bucket" "application_data" { bucket = "my-application-data-bucket" } resource "aws_s3_bucket_lifecycle_configuration" "advanced_lifecycle" { bucket = aws_s3_bucket.application_data.id # Rule for application logs rule { id = "archive-logs" status = "Enabled" filter { prefix = "logs/" } transition { days = 7 storage_class = "STANDARD_IA" } transition { days = 30 storage_class = "GLACIER_IR" } transition { days = 90 storage_class = "DEEP_ARCHIVE" } expiration { days = 365 # Delete logs after 1 year } } # Rule for backups rule { id = "archive-backups" status = "Enabled" filter { prefix = "backups/" } transition { days = 1 storage_class = "GLACIER_IR" } transition { days = 30 storage_class = "DEEP_ARCHIVE" } expiration { days = 2555 # Delete after 7 years (compliance requirement) } } # Rule for user uploads rule { id = "transition-user-content" status = "Enabled" filter { prefix = "uploads/" } transition { days = 60 storage_class = "STANDARD_IA" } transition { days = 180 storage_class = "GLACIER_IR" } } # Rule for temporary files rule { id = "cleanup-temp-files" status = "Enabled" filter { prefix = "temp/" } expiration { days = 7 # Delete temporary files after 7 days } } }

Enter fullscreen mode

Exit fullscreen mode

Using Intelligent-Tiering for Unknown Access Patterns

If you’re unsure about access patterns, S3 Intelligent-Tiering automatically optimizes costs:

resource "aws_s3_bucket" "intelligent_bucket" { bucket = "my-intelligent-bucket" } resource "aws_s3_bucket_lifecycle_configuration" "intelligent_lifecycle" { bucket = aws_s3_bucket.intelligent_bucket.id rule { id = "enable-intelligent-tiering" status = "Enabled" transition { days = 0 storage_class = "INTELLIGENT_TIERING" } } } # Configure Intelligent-Tiering archive access tiers resource "aws_s3_bucket_intelligent_tiering_configuration" "intelligent_config" { bucket = aws_s3_bucket.intelligent_bucket.id name = "EntireBucket" tiering { access_tier = "ARCHIVE_ACCESS" days = 90 } tiering { access_tier = "DEEP_ARCHIVE_ACCESS" days = 180 } }

Enter fullscreen mode

Exit fullscreen mode

Filtering by Tags

You can also create lifecycle rules based on object tags:

resource "aws_s3_bucket_lifecycle_configuration" "tag_based_lifecycle" { bucket = aws_s3_bucket.cost_optimized_bucket.id rule { id = "archive-by-tag" status = "Enabled" filter { tag { key = "archive" value = "true" } } transition { days = 1 storage_class = "GLACIER_IR" } } rule { id = "delete-temporary" status = "Enabled" filter { and { prefix = "temp/" tags = { type = "temporary" } } } expiration { days = 30 } } }

Enter fullscreen mode

Exit fullscreen mode

Complete Example with Best Practices

Here’s a complete, production-ready example incorporating versioning and cleanup:

# S3 Bucket resource "aws_s3_bucket" "production_data" { bucket = "my-production-data-${var.environment}" tags = { Environment = var.environment ManagedBy = "Terraform" Purpose = "Cost-optimized storage" } } # Enable versioning resource "aws_s3_bucket_versioning" "production_versioning" { bucket = aws_s3_bucket.production_data.id versioning_configuration { status = "Enabled" } } # Lifecycle configuration resource "aws_s3_bucket_lifecycle_configuration" "production_lifecycle" { bucket = aws_s3_bucket.production_data.id # Current version lifecycle rule { id = "transition-current-versions" status = "Enabled" transition { days = 30 storage_class = "STANDARD_IA" } transition { days = 90 storage_class = "GLACIER_IR" } transition { days = 180 storage_class = "DEEP_ARCHIVE" } } # Noncurrent version lifecycle rule { id = "transition-noncurrent-versions" status = "Enabled" noncurrent_version_transition { noncurrent_days = 7 storage_class = "STANDARD_IA" } noncurrent_version_transition { noncurrent_days = 30 storage_class = "GLACIER_IR" } noncurrent_version_expiration { noncurrent_days = 90 } } # Clean up incomplete multipart uploads rule { id = "cleanup-incomplete-uploads" status = "Enabled" abort_incomplete_multipart_upload { days_after_initiation = 7 } } # Clean up delete markers rule { id = "cleanup-delete-markers" status = "Enabled" expiration { expired_object_delete_marker = true } } }

Enter fullscreen mode

Exit fullscreen mode

Calculating Your Savings

Let’s say you have 10 TB of data in S3 Standard that hasn’t been accessed in 6 months:

Before lifecycle policies:

After implementing lifecycle policies:

Best Practices

  1. Start Conservative: Begin with longer transition periods and adjust based on your access patterns
  2. Monitor Access Patterns: Use S3 Storage Lens or CloudWatch to understand how your data is accessed
  3. Test First: Apply policies to a test bucket before production
  4. Consider Retrieval Costs: Glacier has retrieval fees, so ensure data truly isn’t accessed frequently
  5. Use Intelligent-Tiering for Unknown Patterns: When uncertain, let AWS optimize for you
  6. Clean Up Old Versions: Don’t forget to lifecycle noncurrent versions in versioned buckets
  7. Document Your Strategy: Comment your Terraform code to explain the reasoning behind transition timings

Monitoring and Validation

After applying lifecycle policies, monitor their effectiveness:

# Check lifecycle rules aws s3api get-bucket-lifecycle-configuration --bucket your-bucket-name # View storage metrics aws s3api list-objects-v2 --bucket your-bucket-name --query 'Contents[*].[Key,StorageClass,LastModified]' --output table

Enter fullscreen mode

Exit fullscreen mode

You can also use AWS Cost Explorer to track storage cost reductions over time.

Common Pitfalls to Avoid

  1. Too Aggressive Transitions: Moving data to Glacier too quickly when it’s still being accessed incurs retrieval costs
  2. Forgetting Minimum Storage Durations: Standard-IA and Glacier have minimum storage duration charges (30 days and 90 days respectively)
  3. Ignoring Small Objects: Objects smaller than 128 KB are charged for 128 KB in IA and Glacier classes
  4. Not Cleaning Up Incomplete Uploads: These can accumulate and cost money
  5. Overlooking Noncurrent Versions: In versioned buckets, old versions can consume significant storage

Conclusion

Implementing S3 lifecycle policies with Terraform is one of the easiest ways to reduce your AWS bill without sacrificing data availability. By automatically transitioning infrequently accessed data to cheaper storage tiers, you can save thousands of dollars annually while maintaining compliance and data retention requirements.

Start with the examples in this article, adjust the transition periods to match your access patterns, and watch your storage costs drop. Your CFO will thank you!

Additional Resources

Have you implemented S3 lifecycle policies in your infrastructure? What strategies have worked best for you? Share your experiences in the comments below!

For further actions, you may consider blocking this person and/or reporting abuse

Залишити відповідь

Ваша e-mail адреса не оприлюднюватиметься. Обов’язкові поля позначені *

Читайте також