Amazon S3 Intelligent Tiering: How it Helps to Optimize Storage Costs
AWS (Amazon Web Services) launched S3 Intelligent-Tiering in 2018, which added the capability to take advantage of S3 without needing to have a deep understanding of your data access patterns. This solution should reduce the amount of manual work you need to do to archive objects with unpredictable access patterns which are not accessed for months at a time. In 2021 AWS announced news about this storage class which you will find at the end of this article, but first, let’s look at what S3 intelligent tiering is and why you should use it.
What is Amazon S3 Intelligent-Tiering?
Let's first look at the more general things around Amazon S3 and its storage classes so that you can better understand why S3 Intelligent-Tiering is useful.
Amazon Simple Storage Service (Amazon S3) is an object storage service provided by AWS. It can be used as a data store for a range of use cases, such as websites, mobile applications, backup and restore, archiving or keeping long-term archives for compliance purposes.
S3 provides different levels of reliability and availability through its storage classes. Each storage class has a different cost structure and guarantees about how quickly you can access your data. Some of them even have minimal storage duration.
- S3 Standard : Designed for frequently accessed data.
- S3 Intelligent-Tiering : Data with changing or unknown access patterns.
- S3 Standard-IA : Designed for long-lived, infrequently accessed data.
- S3 One Zone-IA : Designed for long-lived, infrequently accessed, non-critical data.
- S3 Reduced redundancy : Frequently accessed, non-critical data.
- S3 Glacier Instant Retrieval : Designed for archive data accessed once a quarter with instant retrieval in milliseconds.
- S3 Glacier Flexible Retrieval (formerly Glacier) : Designed for infrequently accessed, archived critical data.
- S3 Glacier Deep Archive : Long-term data archiving with retrieval times within 12 hours.
S3 Intelligent-Tiering
S3 Intelligent-Tiering is a storage class designed to optimize storage costs by automatically moving data to the most cost-effective access tier without affecting performance or operational overhead. It is available in all AWS Regions.
AWS created S3 Intelligent-Tiering to solve the problem of using the correct storage class and optimizing costs when the access pattern is irregular. When access patterns change, S3 Intelligent-Tiering automatically saves costs by moving data between access tiers. This is the perfect storage class when you want to optimize the storage cost of data with unknown or unpredictable access patterns.
For a small monthly object monitoring and automation fee, S3 Intelligent-Tiering monitors the access patterns and moves the objects automatically from one tier to another.
Intelligent-Tiering will automatically choose between these tiers:
- Frequent Access and Infrequent Access tier: The first tier is optimized for frequent access, and the other one is optimized for infrequent access. These tiers provide customers with low latency and high throughput performance. After 30 days in the Frequently Accessed tier, objects that have not been used will be moved automatically to the Infrequently Accessed tier. The frequent access tier is priced as S3 standard, and the Infrequent Access tier is priced as S3 Standard-Infrequent Access.
- Archive Instant Access tier (new): After 90 days in the frequently accessed tier, objects that have not been used will be moved automatically to the Archive Instant Access tier.
And you can also enable these two tiers for archive purposes and to further lower cost:
- Archive Access tier: This tier has the same performance and price as the S3 Glacier Flexible Retrieval storage class. It is used for files that are not accessed for more than 90 days, but you can create a custom policy. - Deep Archive Access tier: This tier has the same performance and price as the S3 Glacier Deep Archive storage class. It is used for files that are not accessed for more than 180 days, but you can create a custom policy.
How does S3 Intelligent-Tiering work?
After the 2021 update, it can be divided into two main solutions:
1. Automatic Access tiers + Deep Archive Access tier
Objects uploaded or transferred to S3 Intelligent-Tiering are automatically stored in the Frequently Accessed tier.
S3 Intelligent-Tiering works by monitoring access and usage patterns and then moving objects that have not been accessed for 30 consecutive days to the Infrequent Access tier.
After 60 days in the Infrequently Accessed tier, objects that have not been used will be moved automatically to the Archive Instant Access tier. And if you activate the Deep Archive Access tier, these objects will be automatically moved after another 90 days (a total of 180 days).
Objects in the Deep Archive Access tier, will be retrieved within 12 hours.
If an object in any of the access tiers is accessed, it is automatically moved back to the Frequent Access tier.
2. Archive + Deep Archive Access tier
If you activate the Archive Access tier, S3 Intelligent-Tiering will stop using the Archive Instant Access tier and automatically move objects that have not been accessed for 90 days to the Archive Access tier. And if you activate the Deep Archive Access tier, these objects will be automatically moved after another 90 days (a total of 180 days).
When an object in any Archive Access Tier is restored, the object will move to the frequent access tier within a few hours and then be ready for retrieval. Objects in the Archive Access tier will be retrieved within 3–5 hours; if they are in the Deep Archive Access tier, they will be retrieved within 12 hours.
S3 Intelligent-Tiering benefits and why you should use it
Normally, AWS customers can leverage lifecycle policies that are helpful in some ways, but they’re a basic instrument. Lifecycle policies always apply to all objects (files) in a given S3 prefix (folder), but the access patterns for objects can vary. You might access some files very frequently while some are rarely accessed and used. This can lead to unnecessary storage bills.
By allowing AWS to determine the appropriate tiers for each object and transform them as needed, customers can not only save money but can also save their time, because you don’t have to decide at the start which files are best for which storage class.
- Frequent and Infrequent Access tiers have the same low latency and high throughput performance as S3 Standard and save up to 40% on storage costs.
- S3 Intelligent-Tiering is designed for durability of 99.999999999% of objects across multiple Availability Zones with 99.9% availability over a given year.
- No operational overhead, no retrieval charges, and no minimum storage duration.
When S3 Intelligent-Tiering is not the right choice
If S3 Intelligent-Tiering is right for your use case, it can help you save some money on your AWS bill. However, it is not suitable for all use cases. The following is a situation where it may not be the right choice. (Though there were more unsuitable situations in the past, after the 2021 update, this is the only remaining situation where S3 Intelligent-Tiering does not make sense.)
- Predictable access and usage patterns: If your objects or whole folders have very predictable access patterns, you can handle them through object lifecycle rules. You just choose after what time your files should be transferred to another storage class. By doing this, you will avoid a monitoring fee.
S3 Intelligent-Tiering pricing
You pay for monthly storage, request and data transfer. When using Intelligent-Tiering you pay a small monthly per-object fee ($0.0025 per 1000 objects) for monitoring and automation.
There is no retrieval fee in S3 Intelligent-Tiering and no fee for moving data between tiers.
- Objects in the Frequent Access tier are billed at the same rate as S3 Standard.
- Objects stored in the Infrequent Access tier are billed at the same rate as S3 Standard Infrequent Access.
- And objects in the Archive Instant Access Tier are billed at the same rate as S3 Glacier Instant Retrieval.
- Objects stored in the Archive Access tier are billed at the same rate as S3 Glacier Flexible Retrieval (Formerly S3 Glacier) and objects stored in the Deep Archive Access tier are billed at the same rate as S3 Glacier Deep Archive.
You can find standard rates for every S3 storage class on the official AWS S3 pricing page.
S3 Intelligent-Tiering cost comparison
With a simple example, we can determine if the usage of S3 Intelligent-Tiering is really useful.
For this comparison, assume that you have some objects under a particular prefix (saved in one folder in the S3 bucket). Some are accessed frequently and some are not and you are not able to simply determine this.
Let’s use the following example which takes into account one year of usage:
- Objects are placed in the region: Europe (Frankfurt) eu-central-1.
- We only use the Automatic Access tiers (Frequent, Infrequent, and Archive InstantAccess tier).
- 2,500,000 objects of 0.5MB each are placed into the S3 bucket.
- Each month, half of the objects are modified twice by a PUT request.
- Each month, half of the objects are read 15 times by a GET request.
Table with S3 Standard costs:
Table with S3 Intelligent-Tiering costs:
So, it should be possible to save approximately $117 (18%) in this simplified example scenario when using S3 Intelligent-Tiering.
How to enable S3 Intelligent-Tiering
There are multiple ways of enabling S3 Intelligent-Tiering for data in the S3 bucket or for newly uploaded data.
Enabling Intelligent-Tiering for folders and files
To set and update object storage classes, you can use the Amazon S3 console, AWS SDKs, or the AWS Command Line Interface (AWS CLI). For more information, see the official AWS documentation: setting the storage class of an object.
Enabling S3 Intelligent-Tiering with lifecycle rules
You can add rules to an S3 Lifecycle configuration to tell Amazon S3 to transition objects from your current storage class to S3 Intelligent-Tiering. For more information, see the official AWS documentation: setting lifecycle configuration on a bucket.
More advanced ways of putting data into Intelligent-Tiering
You can configure S3 Intelligent-Tiering as your default storage class for newly created data, or you can choose to activate one or both of the archive access tiers using the API with PutBucketIntelligentTieringConfiguration, the CLI, or the Amazon S3 console. See more: using S3 Intelligent-Tiering.
Having problems with implementation? Let our certified cloud experts guide you.
S3 Intelligent-Tiering news
AWS announced a few features and changes to their S3 Intelligent-Tiering during the AWS Storage Day 2021 and AWS re-Invent 2021.
- AWS is removing per-object monitoring and automation charges for objects smaller than 128 KB. When AWS released Intelligent-Tiering, objects smaller than 128 KB would never be transitioned to other access tiers and were billed at the usual rate. With these changes, S3 Intelligent-Tiering is the ideal storage class for data independent of object size.
- You will not be charged for objects deleted, transitioned, or overwritten within the first 30 days in S3 Intelligent-Tiering. Previously S3 Intelligent-Tiering was not usable for objects that stayed there for less than 30 days, because this function had a minimum storage duration of 30 days.
- The Amazon S3 Intelligent-Tiering automatically includes a new Archive Instant Access tier (similar to S3 Glacier Instant Retrieval) with further cost savings for data that needs millisecond retrieval.
AWS' extensive service and pricing options provide the flexibility to effectively manage costs, learn more in our blog post: Best Practices for Reducing AWS Bill
An AWS Solutions Architect with over 5 years of experience in designing, assessing, and optimizing AWS cloud architectures. At Stormit, he supports customers across the full cloud lifecycle — from pre-sales consulting and solution design to AWS funding programs such as AWS Activate, Proof of Concept (PoC), and the Migration Acceleration Program (MAP).
