If you would like to read the other parts in this article series please go to:
- Secure services and resources with AWS Identity and Access Management (Part 1)
- Secure services and resources with AWS Identity and Access Management (Part 2)
- Secure services and resources with AWS Identity and Access Management (Part 3)
- Secure services and resources with AWS Identity and Access Management (Part 4)
In this multi-part article, we’re discussing how to use AWS Identity and Access Management (IAM) to create and manage users and groups and assign permissions to them so as to granularly control how they can access your AWS resources. In Part 1, we talked about IAM basics: how it works, users and groups, authentication and credentials. We then introduced the concept of IAM policies.In Part 2, we delved more deeply into the subject of IAM policies, permissions, and roles, as well as how to integrate IAM into a federated environment. In Part 3, we continued the discussion of policies with some specifics about how you work with managed policies using the AWS Management Console and the AWS command line interface. Last time, in Part 4, talked about some tips on logging IAM events with AWS CloudTrail.
Today in our final installment, Part 5, we’re going to address how to troubleshoot IAM problems.
What could possibly go wrong?
Anyone who has worked in IT and especially with security knows that the sheer complexity of modern computing and the interactions between systems means that inevitably, sometimes things don’t work quite as smoothly and seamlessly as they were designed to. IAM is no different; there are common configuration errors, conflicts and other scenarios that can lead to problems, so it’s important to have a basic understanding of how to track down what might be happening when you or your users encounter an IAM-related road block.
Security of any kind is always about a balance between accessibility and protection. Users, who need access to resources in order to get their jobs done and who don’t like having to jump through a bunch of hoops and spend extra time getting to those resources, tend to favor accessibility over security. Administrators, who are responsible for seeing that hackers don’t get in and sensitive data doesn’t leak out, tend to favor security over accessibility. This puts the two groups in something of an adversarial relationship at times, and we forget that our end goals are the same: to keep the business process flowing as quickly but at the same time as safely as possible.
For maximum convenience and productivity, we would open up access so that users would always be assured of being able to connect to resources – but that wouldn’t be secure. For maximum security, we would lock everything down as tightly as possible, but that would hinder and frustrate users and possibly even cause loss of revenue if workers aren’t able to do their work properly. Thus the quest for balance. But even when we take a balanced approach, sometimes security may get in the way and those who are supposed to have access may find themselves unable to connect to the AWS services and access the resources they need.
Troubleshooting “Access Denied” Messages
One of the most frequent, and for users one of the most frustrating, issues that you may need to troubleshoot is the “access denied” message. There are few things that are more annoying, when you’re trying to make a request to a service that you need to do a job, than to be told “no.”
Permissions and Policies
If a user gets an “access denied” message when making a request to an AWS service, the first thing to do is check and make sure that user account has permission to perform the attempted action on the specific resource. Sometimes there will be conditions set in the policies for the user’s individual account, a group that the user is part of, or a role that the user is assigned. The user will have to meet those conditions in order to access the resource. For example, there might be a condition set that allows the user to access the resource only when authenticated using MFA (Multi-Factor Authentication).
Remember that the default for IAM security is “deny all.” That means that any actions you don’t explicitly allow for that user, group or role will be denied. You also have to explicitly grant access to all resources you want that user to be able to take action on. Any resources not explicitly allowed will be denied.
Also remember, from our discussion about IAM policies, that policies can be attached to a user or group, or they can be attached to the resource. So be sure to check the policies on all if a user is being denied access and you can’t figure out why. If a user is trying to access a resource that has resource-based policies such as S3 buckets, you need to make sure that the user is listed as a principal in that resource policy and that access is explicitly granted.
Manually Signed Requests
Another reason that access might be denied would be if the user is manually signing the request for services and hasn’t done it properly. Requests to AWS are required to be signed in order to verify the requester’s identity and prevent tampering with data while it’s in transit.
A manually signed request would be a request that is made without using the AWS SDKs or the AWS CLI, which will sign the requests for you as part of constructing and sending requests to AWS. You just need to configure them with an access key.
Signature Version 2 or Signature Version 4 can be used to sign requests that are created programmatically but Version 4 is recommended if the particular service supports it. The code for signing a request must be in the proper syntax and must contain the necessary signing information. You can find the process for signing requests using Signature Version 4 here:
http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
Making Requests with Temporary Credentials
Sometimes users may get the dread “access denied” message when they use temporary security credentials for accessing AWS. Many of the AWS services support the use of temporary credentials. However, some AWS services do not, so your first troubleshooting step would be to make sure the service is one that accepts temporary credentials.
Temporary credentials are generated by the actions of the AWS STS (Security Token Service). STS is a web service by which you can get temporary credentials for IAM users that have limited privileges. One of the big differences between these temporary credentials and regular long-term creds is that they’re, well, temporary. That means they expire after a specified period of time, and can no longer be used. If a user attempts to make a request to an AWS service with expired credentials, access would be denied.
Launching an EC2 instance with a role
Sometimes when you are attempting to launch an instance in the EC2 service with a role, you might get an “access denied” error. The first thing to do in troubleshooting this situation is to try launching an instance without an instance profile. That way you can determine whether or not the problem is related to the IAM roles for your EC2 instances.
You need to make sure that the IAM user that is making the request has the correct permissions and that you’re using an instance profile that’s valid. If an instance profile is empty (has no role) you may get an “access denied” message. You may need to create a role, using the AWS Management Console, command line interface or the IAM API. Roles allow applications to securely make requests from EC2 instances without the need for you to manage the credentials being used by the apps.
After you create the role, you need to define the accounts or services that can assume the role and which actions and resources the application can use when it assumes the role. Then you designate the role when you launch the app and the role will use a set of temporary credentials. The instance profile is a container for the IAM role. You assign permissions for roles by creating JSON policies like the ones you create for users, as we discussed in a previous installment of this series.
Root user and S3 bucket
Another “access denied” scenario that sometimes rears its ugly head occurs when you’re signed in as root user, but you can’t access an S3 bucket. This can be confusing because the root account is supposed to allow full access to all resources in the account. Amazon actually recommends that you not use the root user account, but instead store its credentials in a secure location and create a regular IAM user account to which you assign administrator permissions for most administrative tasks.
However, if you are using the root account for some reason and find yourself unable to access an S3 bucket, this is because an IAM user with full access has assigned a bucket policy and didn’t explicitly make the root user a principal. You’ll recall that if permissions aren’t explicitly granted, access will be denied.
That doesn’t mean the root user can’t access the bucket, though – it just means you have to use a little workaround: you can, as root user, change the bucket policy yourself, and give the root user explicit access to the bucket. Problem solved!
Summary
This wraps up our five part series on AWS Identity and Access Management – what it is, how it works and how to use it and troubleshoot problems that might arise with it. IAM is a valuable tool for securing access to AWS resources and ensuring that users have the access they need to do their jobs, and no more. Some of the individual AWS services have additional security controls outside of IAM and we will be looking at those in a future article.
If you would like to read the other parts in this article series please go to:
- Secure services and resources with AWS Identity and Access Management (Part 1)
- Secure services and resources with AWS Identity and Access Management (Part 2)
- Secure services and resources with AWS Identity and Access Management (Part 3)
- Secure services and resources with AWS Identity and Access Management (Part 4)
