Today we’ll talk about EFS, some use cases, usability and some challenges I faced while using this service.
So, what is EFS ? EFS stands for Elastic File System which is a managed service provided by AWS. It is similar to an NFS service that you are probably already familiar with (Network File System). EFS enables you to create and configure a shared file system without any provisioning, deploying, patching, or maintenance. EFS will automatically scale up or down as files are added or removed, and can burst to higher throughput if needed. By default, EFS utilizes high availability and durability (99.999999999 percent (That’s 11 9s!)). You can connect EFS to numerous services such as EC2, ECS, EKS, Fargate, Lambda and local servers. We can talk all day about storage types and classes, backups and monitoring but I want to get to the good stuff : -)
There are a couple of ways to connect your EFS, network wise:
The first and by far the easiest method is to mount it onto an EC2 when they both reside in the same VPC.
Mount command with NFS: mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,
retrans=2,noresvport fs-12a34b56c78d.efs.us-east-2.amazonaws.com:/ /mnt/efs_folder
You can also add it to /etc/fstab as a persistent mount point that will not change after restart. While testing NFS on EFS, I was not able to properly use ‘nfs4_acl’ package, and encountered many issues with it.
If you want to allow other services that reside in different VPCs to connect to your EFS, you can use a peering connection or transit gateway between 2 or more VPCs, or you can even go the distance and use ALB, NLB or a Service Endpoint (Private link). The latter is a specialized use case that even AWS Awesome support was unsure if it would work… We got it to work ;-) When should you use peering versus transit gateway, you ask? Good question grasshopper, I am glad you asked. Peering is a single network connection which enables you to route traffic between two different VPCs (see more here). Transit gateway is basically a “router in the clouds”. It will easily connect between your on-prem networks and those you have on AWS. If there comes a time when you have too many VPCs, peering or VPN connections to handle, then TGW is the better choice to use. It provides a single gateway for your network needs (see more here).
Transit Gateway connection diagram:
Service Endpoint connection diagram:
The mount command for the endpoint is as follows: mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,
retrans=2,noresvport vpce-1a2345678b1234c1-ab12abcd.vpce-svc-112a3456b78910cd.us-east-2.vpce.amazonaws.com:/ /mnt/efs_folder
If you wish to use the /etc/fstab file, add this line: vpce-1a2345678b1234c1-ab12abcd.vpce-svc-112a3456b78910cd.us-east-2.vpce.amazonaws.com:/ /mnt/efs_folder nfs4 nfsvers=4.1,rsize=1048576,
wsize=1048576,hard,timeo=600,retrans=2,noresvport 0 0
As you can see, we are not using the EFS ID or EFS DNS name when mounting, but the VPC Endpoint!
After we had great fun building the infrastructure, we are now ready to configure the EFS. Let’s get cracking!
In the initial page, you will create the file system.
Insert the desired name and the location of your VPC:
Once created, you will be able to view and manage your file system size, monitoring, tags, EFS policy access points and network.
Click Edit to perform some basic configuration, such as adjusting your backups, storage tier (A or IA), and throughput mode.
Note that encryption is enabled by default.
Creating the EFS is not enough, though.
You need to create access points - which can be considered as folders.
There are many ways you can set up EFS. Use it as a one giant folder under root or set up multiple folders for each user \ group \application storage. You can go even further and divide each access point to be used by different local OS users, each one with its own set of permissions.
If you want to allow root access to the entire EFS, select / (root location).
This is the place to note that you need to differentiate between the two root concepts. Local OS folder \ user - for both EFS and OS, so make this distinction.
Pillars of Security:
There are a couple of ways in which you can configure the permission level for the EFS: 1. Use an IAM role with specific permissions, and copy that policy to the EFS policy as a second layer of protection. 2. Use the IAM role as a dummy role, and have EFS handle all permission levels.
Take a minute to think about those two options: which one is the best? Which one would you use?
Well, in my mind – neither of them.
The first one will not really double the protection because IAM roles hold greater power than the EFS policy, and it will only cause you to work twice as hard when updating policies. The second is just plain silly, since the mount uses IAM as a real security layer - it is wasteful not to use it. Furthermore, it only opens the possibility for errors during IAM role clean up. Nobody will understand why there are dummy roles and as a result, the dummy roles may be deleted and you can guess what would happen immediately after that.
The best method in my mind is to treat each service (IAM, EFS, EC2 and VPC) as a separate, independent security pillar. Use the IAM role as the user pillar; a way to manage users \assumed roles and provide them with specific access (like read, write, mount, etc.). Use EFS as the storage pillar; EFS policy will protect everything related to EFS access (we’ll see later on how this is handled). Use EC2 as the local OS pillar; define what to use in your mount entry - should you use IAM? TLS? Should you grant specific access to users or have open access to anyone on the access point?
Use VPC as the network pillar; should you use a simple network connection or a private link? EFS policy: As you can see in the example below, in the left pane, we’ve prevented root access, enforced read-only access, prevented anonymous access, and enforced in-transit encryption. This, combined with the EFS encryption at rest, seals the deal - data is encrypted, and writing\reading will not happen unless it is encrypted, so it’s a win for team encryption).
IAM Role: In order to use IAM role as an enforcer of who can actually mount or write\read the access point, you need to be able to add an “instance-profile" to your EC2 instance.
Mind you - only one instance-profile can be added per instance. Having said that, you should remember that if an SSM connection is needed, then the role should have SSM connectivity as well; otherwise SSM will not work without these policies "AmazonSSMManagedInstanceCore", "AmazonS3ReadOnlyAccess".
IAM roles override EFS policy - if “ClientRootAccess” is granted on the IAM level but denied on the EFS level, then root would still be able to write to the EFS.
Root squashing: Understanding the concept of root squashing is important. By not mentioning ClientRootAccess, you are effectively blocking root (local OS user) from being able to write to the EFS. On the other hand, don't forget that only root can mount the EFS (see more here).
Mount, Read & Write, for a specific EFS access point: If you want to allow root access to the EFS (that is, allow access to all files and folders), use the "ClientRootAccess" permission. For read-only permission, use “ClientMount”, which seems redundant since a non-root user without write capabilities can only read, but since there are no read permissions,for all intent and purposes the “ClientMount” is read-only.
IAM role example:
In the EC2 console, add the desired IAM role. This will allow the instance to assume the role, which will grant the instance connection to EFS.
Initial EC2 Local setup: If your environment requires the usage of a proxy, you need to set it up in order to download the amazon-efs-utils package (otherwise you can’t use EFS as a file system, only NFS).
To do this setup, first, export your http_proxy & https_proxy variables into your environment variables (use the ‘env’ command to review). For example: export http_proxy=http://proxy.url:port export https_proxy=http://proxy.url:port