If you are familiar with Amazon web services, you might know that AWS provides various storage options for your EC2 instances. Quoting from the AWS docs, “Amazon EC2 provides you with flexible, cost effective, and easy-to-use data storage options for your instances. Each option has a unique combination of performance and durability. These storage options are used independently or in combination to suit our requirements.” One of the storage options available is Elastic File System. But before going through EFS, we would begin with brief introduction to all available storage options for EC2 instances. AWS provides various types for storage mechanism, each having unique features of its own. Most widely used ones are:
- Elastic Block Storage (EBS)
- Simple Storage Service (S3)
- Elastic File System (EFS)
Elastic Block Storage
It is a low latency, highly available, disk volume based block storage method. EBS stores data as blocks, have fast IOPS and is relatively faster and cheaper than EFS. Block level storage is usually deployed in SAN or storage area network environment. Block storage treats storage volume as an independent disk drive and it can be controlled by external server operating system. We can attach it to only one EC2 instance within the same region and same availability zone. Although it can be migrated from one region to another but it cannot be accessed across the regions.
Simple Storage Service
Simple storage service is an object storage method, that can store and retrieve any amount of data, at any time, from anywhere on the web. It store data as objects in buckets. Each object holds some data (files) and metadata, bundled up as an object. Unlike files and blocks, it stores objects in flat architecture and each object is assigned with an unique object ID. They can be retrieved with the help of their object ID. It is highly available, redundant and cheapest than all three of them. Also, it is accessible across regions and good for storing static contents like images, videos, logs etc.
Elastic File System
Now coming to our topic, Elastic File System is simple, scalable and elastic file storage system for our EC2 instances. It is analogous to NAS (network attached storage), which is a file level data storage connected to a network and providing data to a heterogeneous group of clients across the network. EFS is NFS-v4 and SSD based and a fully managed file system. Also, it provides standard file system semantics and works with all standard operating system APIs.
It is elastic in manner, as the storage size increases or decreases elastically with data size in our storage. It provides low and consistent latency regardless of file system size. Although, IOPS and throughput varies with data size in EFS. It is highly scalable too as it can grow to petabyte scale.
One key feature of EFS is that it can be mounted to thousands of the EC2 instances irrespective of its size, location and availability. It is highly available and accessible across various availability zones. It provides data consistency to all attached EC2 instances.
EFS Availability and its Architecture
Currently, Amazon provides this product for few regions only i.e. US east (N. Virgina and Ohio), US west (Oregon), EU (Ireland), Asia Pacific (Sydney). We can create a file system in any of the above region and mount it in various availability zones. For this, we create mount targets at each availability zone where your EC2 instances are present. These mount targets acts as an end point of our elastic file system. Each mount point associates with a DNS name and an IP address. Each of the instances access the file system by these mount targets, communicating with them via IP address and DNS name. These mount targets hold the content of EFS in their respective availability zone, having links with each other. Any changes in the file system from one region, reflects to all the other active regions. It ensures that all instances have access to updated file system, maintaining the reputation of high data availability and consistency.
This feature of EFS (being shareable to multiple instances), make it one of the good choices for storage options (other than S3) when we go for load balancing and auto-scaling as newly generated instances can have access over common file system.
Securing the Elastic File System
As a matter of fact, there are a lot of security measures we can take for securing our EFS. we can create an access control list by managing security groups, allowing the inbound traffic only from desired instances or more precisely from specific security groups. We can also maintain user policies and control user’s access over file system, like creating IAM users giving strict permissions by attaching needful policies. Apart from traffic and user restriction, we can also grant user specific permissions to files and directories in our file system, hence making it accessible only for certain users or group.
So far we have learned that concept of EFS can be very productive for scenario having hundreds of instances running and high availability, data consistency, scalability etc are key requirements.
In our next blog, we will create an EFS and mount it on our EC2 instances.
2 comments