This is the second in a three-part series on Amazon S3 Security In-Depth. In Part I of this series, we discussed the different mechanisms you can use to allow access to your Amazon S3 buckets and objects. In Part II, we will take a deeper look at managing access to your S3 resources using AWS Identity and Access Management (IAM).
When thinking about IAM, there are two broad categories to consider: identities and permissions.
Identities refer to the various mechanisms that AWS provides to identify who is requesting a particular AWS action, for authenticating that person or entity, and for organizing similar entities into groups. This topic includes IAM users and groups, credentials, and roles. We will cover more about best practices for managing IAM identities in Part III of this series.
Permissions refer to what a particular identity is allowed to do in the AWS account. Permissions are managed by writing identity-based policies, which are collections of statements. Writing identity-based policies within IAM is the main focus of this Part II of our series. Identity-based policies are used mostly within AWS Identity and Access Management (IAM), so we will refer to them as IAM policies throughout this post. However, the same syntax in IAM policies is also used for S3 bucket policies that were discussed in the previous part in this series.
In this post, we’ll cover:
- The basics of IAM policies and statements, including a breakdown of the important elements;
- An example walkthrough of an IAM policy with multiple statements;
- Best practices for writing and managing IAM statements.
Table of Contents
AWS IAM Policies and Statements
IAM is an AWS service for managing both authentication and authorization in determining who can access which resources in your AWS account. At the core of IAM’s authorization system is an IAM policy.
Let’s take a look at the example below of an IAM policy being created in the AWS console.
The entire document from lines 1-15 is the IAM policy. An IAM policy is a JSON document with an optional “Version” key plus a “Statement” key. The value of the “Statement” key is an array of IAM statements.
There are two IAM statements in this example -- one is from lines 4-8, and the other is from lines 9-13. There is no limit on the number of IAM statements in an IAM policy, though there are limits on the number of characters in an IAM policy. The number of characters varies from 2,048 characters to 10,240 characters depending on the type of policy.
There are three critical elements to an IAM statement. They are:
- Action
- Resource
- Effect
We’ll take a look at each of these separately.
IAM Statement Elements: Action
Action describes the API actions that the statement affects. Action is in the form of “<resource>:<verb>”, where the resource is a particular AWS service (e.g., s3, ec2, or dynamodb) and the verb is a particular action to take within that service (e.g. GetObject or CreateBucket). There are 51 different actions within S3 alone. When writing IAM statements for S3 access, some of the popular actions you will use are:
- s3:GetObject: For reading an object on S3;
- s3:PutObject: For writing an object to S3;
- s3:ListBucket: For reading the objects in an S3 bucket;
- s3:ListAllMyBuckets: For listing all of the S3 buckets in your AWS account.
When writing actions, you can use an asterisk to act as a wildcard on your action. You can use the wildcard at the beginning, middle, or end of your action, or as the entire action if you want to write a statement about all actions. For example, each of the following is a valid action:
- s3:Get*: This would affect all S3 actions that start with “Get”, such as “GetObject”, “GetBucketPolicy”, and “GetBucketLocation”.s3:*: This would affect all actions within the S3 service.*: This would affect all API actions across AWS, including all actions in S3 but also all actions in EC2, SQS, DynamoDB, etc.
You should be careful when using wildcards in your Actions to avoid unintentionally granting broad access. Check out the “Be Careful with Wildcards” piece in the Best Practices section below.
IAM Statement Elements: Resource
The second critical part of an IAM statement is the Resource. The resource element describes the AWS resources to which the statement applies.
Every AWS resource is given an Amazon Resource Name, or ARN. This ARN uniquely identifies a single AWS resource across all AWS accounts. An ARN is structured into 6 parts, each separated by a colon. As you move from left to right, each part gets more specific about the particular resource it identifies.
The general structure is:
arn:partition:service:region:account-id:resource
Let’s look at these pieces individually:
- arn: The first part is always “arn”, regardless of the resource type.
- partition: The second part describes the partition of AWS you’re using. This is usually “aws”, but it may be different if you’re using special regions of AWS, such as GovCloud (“aws-us-gov”) or the China region (“aws-cn”).
- service: The third part is the service you’re using within AWS, such as “s3”, “ec2”, or “sqs”.
- region: The fourth part is the AWS region you’re using if applicable. As of September 2018, AWS has 18 regions around the world, such as “us-east-1” for the Northern Virginia region in the United States, or “eu-central-1” for the Frankfurt, Germany region. For S3 resources, the region is not included in the ARN.
- account id: The fifth part is the account ID of the AWS account that owns the resource. For S3 resources, the account id is not included in the ARN.
- resource: The sixth and final part of the ARN is the resource name. This could be the name of an S3 bucket, an EC2 instance id, or an SQS queue name. Depending on the AWS service, the resource name may include a path to further identify a resource.
Below are examples of ARNs for an S3 bucket, an EC2 instance, and an SQS queue, respectively.
- arn:aws:s3:::my_bucket
- arn:aws:ec2:us-east-1:123456789012:instance/i-1234567890abcdef0
- arn:aws:sqs:us-east-1:123456789012:queue1
Notice how the S3 bucket does not have a value for the region or account id. This is because S3 bucket names are globally unique and thus do not need region or account identifiers.
One important note about S3 ARNs -- there’s a key distinction between bucket resources and object resources. If you want to specify an S3 object, you must include a “/” in the resource name to indicate the bucket and the object. For example, if you have a bucket named “my_bucket” with a single object named “logs.txt”, the resource for the bucket would be “arn:aws:s3:::my_bucket”, while the resource for the object would be “arn:aws:s3:::my_bucket/logs.txt”.
When writing an IAM statement, you must specify one or more resources to which the statement will apply. This allows you to narrowly tailor your IAM statements -- one statement could provide broader permissions to a wide range of resources, while another statement could grant narrow permissions to one or two protected resources.
IAM Statement Elements: Effect
The final element required element of an IAM statement is the effect. Before looking into the effect, let’s review the authorization flow in IAM.
First, all API requests are denied by default. If you make a request that is not authenticated, it will be denied. An API request may be allowed if there is a specific statement that allows the API request for the requesting user. Finally, a request will be denied if there is a specific statement that denies the API request for the requesting user, even in the face of a statement that purports to allow the request.
With this authorization flow in mind, let’s return to the “Effect” portion of an IAM statement. The value for the “Effect” property can be “Allow” or “Deny”. A value of “Allow” will give the attached identity the ability to make the API request on the requested resources in the statement. A value of “Deny” will strictly deny the API request on the resources in the statement.
In the next section, we’ll put all of the elements together to show you how to write multiple IAM statements in an IAM policy.
Putting it all together: Writing IAM policies
Now that we know the basics of IAM statements, let’s write our first IAM policy.
Imagine we have the following scenario. We own an S3 bucket called “MSP360-examples”. We want to allow one of our users to list all of the objects in the bucket. We also want the user to generally be able to read and write freely to the bucket. However, there is a prefix of “secrets/” in the bucket that contains sensitive information. We do not want the user to be able to read or write in the “secrets/” prefix.
You can use the AWS console, the AWS CLI, or numerous infrastructure-as-code tools to manage your IAM permissions. I’m going to use the Access Manager built into the MSP360 Explorer as it has an easy wizard for generating IAM statements.
To manage IAM within MSP360 Explorer, click the “Access Manager” button in the toolbar.
As we break down our needs, we can think about three statements for our policy:
- A statement to allow listing the objects in a bucket;
- A statement to allow read & write access in the bucket;
- A statement to deny read and write access to objects that start with “secrets/*”.
We can see these three statements in the following policy in MSP360’s Access Manager:
The text version of the IAM statement is as follows:
{
"Statement": [
{
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::cloudberry-examples",
"Condition": {}
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::cloudberry-examples/*",
"Condition": {}
},
{
"Effect": "Deny",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::cloudberry-examples/secrets/*",
"Condition": {}
}
]
}
In the first statement, we allow access to the “s3:ListBucket” action on our “MSP360-examples” bucket resource. In the second statement, we allow access for the “s3:GetObject” and “s3:PutObject” actions on all objects in the “MSP360-examples” bucket. Notice how we indicated that it applied to the objects in the bucket by including a “/” separator at the end of the resource name, then using an asterisk to indicate it’s a wildcard.
Finally, the last statement uses the “Deny” effect to explicitly deny “s3:GetObject” and “s3:PutObject” actions on all objects in the “secrets/” prefix in our bucket. Note that this explicit
“Deny” overrides the permissions granted in the previous statement.
In this example, you can see how you can write multiple statements in a single IAM policy. You also can see how an explicit Deny of a particular action can override a separate statement that Allows the same action.
S3 IAM Permission Best Practices
Now that we know the basics of IAM policies and statements, let’s cover a few of the best practices.
Before we dive into specific recommendations, the first and most important practice is to follow the Principle of Least Privilege. This is a general best practice across information security in which a particular user should be given only the privileges which are required to perform its given function. When applied to S3 policies, it means don’t give a user read and write permissions when only read permissions are needed and don’t give access to an entire bucket if only a specific prefix is needed.
With this general principle in mind, here are a few additional tips.
#1 Be Careful with Wildcards
You can use an asterisk -- “*” -- in the Action and Resource properties in your IAM statements. These asterisks may seem like an easy shortcut to quickly write your IAM statements, but they can easily create security holes.
Wildcard characters, such as * and ? can be used to provide the given action to all resources or items
I would avoid using wildcards in the Action property at all. It may be tempting to use “s3:*” to provide full access to S3, but, rarely, a user will need all of those permissions. There are only 51 total actions within S3, and only a few of them are relevant to most users. It’s worth the extra time to be explicit about the actions you want to allow.
In the Resources property, the story is more nuanced. If you are granting permissions on multiple objects in an S3 bucket, you will likely need to use a wildcard to specify a particular prefix that a user is allowed to access. It will be infeasible to list each of the specific objects that a user can access and would require frequent updates as new objects are added or deleted. For this reason, it is acceptable to use wildcards for specifying prefixes in S3 bucket resources. You should avoid using wildcards at any other part of the ARN, such as the bucket name.
For example, using “arn:aws:s3:::cloudberry-examples/uploads/*” to provide access to all objects under the “uploads/” prefix is acceptable. Using “arn:aws:s3:::*” to provide access to all S3 buckets in your account should be avoided.
#2 Split Bucket Statements from Object Statements
Each S3 action applies to either bucket resources or object resources.
For example, the “s3:ListBucket” action applies to buckets only and allows a user to list all of the objects in the bucket. The “s3:GetObject” action applies to objects only and allows a user to read the content of an object and its associated metadata. AWS’s list of S3 Actions does a nice job of showing whether an action applies to buckets or objects.
When writing IAM policies, keep your bucket statements separate from your object statements. It will keep your statements narrowly tailored and will be easier to reason about the effects of making changes.
#3 Use AWS Managed Policies
AWS provides AWS Managed Policies which are policies created by AWS to fill a specific use case. Often you will need to provide some S3 access as part of using other AWS services, such as Amazon Redshift for data warehousing, Amazon SageMaker for machine learning, or AWS Lambda for event-driven compute. Using AWS managed policies with these services can help you get started quickly while still using least-privilege permissions.
Conclusion
Managing S3 permissions correctly is vital, but it can be difficult to absorb all the different terminology when writing IAM policies. In this post, we explored the permissions side of IAM -- writing policies and statements and understanding the key elements of permission statements. After showing an example policy, we then discussed the best practices for writing S3 IAM statements.
In the next part of this series, we will look at the identity aspect of IAM - authentication, users, groups, and roles.
- Backup to Amazon S3, Amazon S3 Glacier and more
- Image-based and file-level backup
- Encryption and compression
