5 Functions of EBS Volumes You're Not Using

Amazon Elastic Block Storage

Since the introduction of Amazon Elastic Block Storage (EBS) in 2008, AWS has added new volume types and functionality, features many cloud users are not using to their advantage.

We have chosen 5 that are worth exploring:

1. RAID
2. Re-warming
3. EBS size while in use without downtime
4. CloudWatch Events
5. Sharing Snapshots across AWS accounts

Before we discuss EBS further, it is important to note a few definitions that will better help in understanding storage system performance.

Throughput: The number of bytes per second transferred to or from a disk.
- It reports read and write throughput separately in one-minute intervals
- It is measured in units of megabytes per second (MB/s)
- Typical values for throughput range from zero to the I/O channel’s maximum bandwidth
IOPS: An abbreviation for the total Input/output Operations Per Second.
- It reports read and write IOPS separately on one minute intervals.
- Total IOPS is the sum of the read and write IOPS
- IOPS values range from zero to tens of thousands of operations per second
Latency: The elapsed time between the submission of an I/O request and its completion.
- It reports as the average latency for a given time interval
- Read and write latency are measured separately in one-minute intervals using units of seconds
- Typical values for latency are in the millisecond (ms): for example, Amazon EBS reports 3 ms as 0.003 seconds

With these definitions we can now discuss and get to know the five lesser-known functions of EBS volumes: RAID, pre-warming, increasing EBS size while in use, CloudWatch Events, and Snapshot sharing across AWS accounts.

1. RAID

Amazon EBS volume data is replicated across multiple servers in different availability zones to prevent the loss of data from the failure of any single component.

This replication makes Amazon EBS volumes more reliable, which means customers who follow the guidance found on the Amazon EBS and Amazon EC2 product detail pages typically achieve good performance out of the box. However, there are certain scenarios where it is necessary to achieve a higher network throughput with much better IOPS.

One way to accomplish this is by configuring a software level RAID array. RAID is supported by almost all operating systems and is used to boost the IOPS and network throughput of a volume in Amazon EBS.

Before configuring RAID, it is important to know how large your RAID array should be and how many IOPS are required.

In the most basic terms, configuring a RAID array works by using two volumes as one, using the combined IOPS and network throughput of both volumes.

Things to Remember When Configuring RAID on Amazon EBS:

Typically, RAIDS types such as 0, 1, 5 and 6 can be configured on AWS.
As per AWS, the ideal RAID configuration is RAID 0 and 1: the reason for this is because in RAID 0 users can stripe multiple volumes together to gain better on-instance redundancy (I/O performance).
In the same way, RAID 1 is used to achieve better availability and fault tolerant behaviour as two volumes are mirrored, with one acting as backup copy for the other in case of a failure.
Different types of instance resources such as General Purpose, Provisioned IOPS, Throughput Optimized, Cold HDD volumes can be combined together in a RAID 0 configuration. This uses the accumulated available bandwidth of all the volumes to achieve higher throughput and IOPS.

A diagrammatic representation of the architecture consisting of RAID 0 arrays should look this this:

Why only RAID 0? Why not other RAIDs on EBS?

AWS also supports RAID 1 on Amazon EBS volumes, which creates a greater fault tolerance capability, and makes it ideal for use cases where data durability is part of the requirement. However, it does not provide write performance improvement because the data is written to multiple volumes simultaneously.

Amazon EBS does not recommend RAID 5 and RAID 6 because the parity write operations of these RAID modes consume some of the IOPS available to your volumes.

Depending on the configuration of RAID array, RAID 5 and 6 modes provide 20-30% fewer usable IOPS than a RAID 0 configuration at an increased cost. Using identical volume sizes and speeds, a two-volume RAID 0 array can outperform a four-volume RAID 6 array that costs twice as much.

It is also recommended to avoid booting from a RAID volume as the grub is typically installed on only one device and if any of the RAID array mirror device fails, you won't be able to boot the OS.

Click here for a step-by-step guide for configuring RAID on EBS volume.

2. Pre-Warming

Pre-warming is a term for pre-initializing, which is a function that helps achieve high throughput when a new volume is accessed.

Pre-warming an Amazon EBS volume is a preliminary action taken before accessing the block storage in order to prevent the significant increase in the latency of an I/O operation that occurs the first time each block is accessed. Such high latency causes degradation in application performance.

The following use case will help explain how pre-warming an EBS volume helps to improve performance.

Customer ABC is running a performance intensive workload which integrates with Amazon Elastic MapReduce (EMR) to process large chunks of data quickly. Customer

ABC had a sudden requirement for more storage but does not want to lose out application performance. They are also aware that attaching a new volume with provisioned IOPS (io1) would create about a 50% latency which would in turn result in performance degradation.

In this case, the priority of Customer ABC is to avoid performance degradation after the storage size is extended. There are several actions Customer ABC can take ensure that there won’t be any degradation in performance while running with extended storage:

Customer ABC can avoid this performance hit by performing a read operation using utilities like dd or fio on all the blocks on volume before using it.
They can ensure that important data is preserved on the volume itself
For a new volume created from a snapshot, Customer ABC can follow the same process of performing read operations, making sure all the existing data is preserved for all types of EBS volumes.

Note: While initializing io1 volumes that were restored from snapshots, the performance of the volume may drop below 50% of its expected level, which causes the volume to display a warning state in the I/O Performance status check. This is expected, and the warning state on io1 volumes can be ignored while they are initializing.

3. Increasing EBS Volume Size While in Use

5 Things You Don't Know About EBS Volumes Amazon EBS is elastic which means you can modify the size, IOPS, and the types of volume on the go without difficulty.

To modify the instance size, it is required to stop the instance, take snapshot of the volume, create a new volume, and attach it.

Recently, Amazon announced that you can now modify EBS volume size and type while they are in an in-use state, significantly decreasing the effort required to extend or modify attributes of a volume.

Executing this modification of volume size comes at no additional charge, although there is a charge for higher storage. Increasing EBS volume size also helps in production use cases where there is an application with a database on EBS.

In the past, DB size would increase, but the administrator would not be able to modify the size as that requires some down time which is not possible in a 24 x 7 live system. Now with this feature, it’s easier to modify the size of a volume. It is important to note that this feature is not available for previous generation instance types.

In general, the following steps are involved in modifying a volume:

Issue the modification command.
Monitor the progress of the modification.
If the size of the volume was modified, extend the volume's file system to take advantage of the increased storage capacity.

For more visit: Modifying the Size, IOPS, or Type of an EBS Volume.

4. CloudWatch Events for AWS EBS

AWS automatically provides data such as instance metrics and volume status checks via Amazon CloudWatch. These are state notifications and can be used to monitor Amazon EBS volumes.

Typically, monitoring data is available on Amazon CloudWatch in five-minute periods at no additional charge. CloudWatch also provides a monitoring option with for PIOPS (Provisioned IOPS) instance types such as io1; data for such instance types are available on Amazon CloudWatch at every one minute, accessible via CloudWatch API or EC2 instances.

Amazon EBS has several state notification events for volumes:

Ok: Normal volume performance
Warning: Degraded volume performance
Impaired: Stalled volume performance, severely impacted
Insufficient data: No information is available on the volume

A user can always configure CloudWatch alarms to trigger an SNS notification based on a state change. In addition to this, CloudWatch Events allows a user to configure Amazon EBS to emit notifications whenever a snapshot or encryption state has a status change.

Users can also use CloudWatch Events to establish rules that trigger programmatic actions in response to changes in the snapshot or encryption key states.

In a use case scenario, Customer XYZ wants to ensure whenever a snapshot is created, the snapshot is shared with another account or copied to another region for disaster-recovery purposes.

A possible solution is for Customer XYZ to leverage Amazon CloudWatch to establish rules that trigger programmatic actions in response to a change in snapshot or encryption.

CloudWatch’s CopySnapshot feature, the event is sent to Customer XYZ’s AWS account when an action to copy a snapshot completes. If either event succeeded then it should trigger a Amazon Lambda function which will automatically copy the snapshot from one specified region to another.

Refer following documentation to see a step-by-step guide for implementing triggers on CloudWatch Events feature: Create Triggers on CloudWatch Events

5. Sharing Snapshots Across AWS Accounts

With Amazon EBS users can share snapshots publically and privately. Sharing snapshots publically means that all the data in the snapshot is shared with all AWS users, hence it is advisable that proper checks be performed before changing permissions of the snapshot.

AWS allows you to share two types snapshots privately and publicly: Unencrypted volume snapshots and Encrypted volume snapshots using custom CMK.

Encrypted snapshots help achieve maximum security between sharer and receiver. Snapshots are encrypted with a custom customer managed key (CMK). To share the snapshot with others, the same custom CMK which was used to encrypt volume is needed.

There are several things to consider before sharing snapshots. Below are some of snapshot sharing’s limitations:

You can share both encrypted and unencrypted volume snapshots between different AWS accounts and regions.
Encrypted snapshots can only be shared if a custom key has been created and shared with another account. For more visit: How to Copy Encrypted Volume to Another Account
User-defined tags are not copied from the source snapshot to the new snapshot.
You can have up to five snapshot copy requests in-progress to a single destination per account. The first snapshot copied to another region is always a full copy and every copy created thereafter is incremental.

A step by step-by-step guide for snapshot creation can be found here:
How to Create an EBS Snapshot

Conclusion

The five features of EBS volumes described above are not widely used, though some AWS users may be aware of them. All of these functions can help achieve optimum performance customized to suit every specific need a customer has.

But before implementing these tasks it is important to perform sandbox testing before moving on to an actual environment. It is advisable to go through proper documentation and benchmarking material provided by AWS and sandbox testing results before actually undertaking any of these tasks.