In the previous blog post of this series, we talked about how to simulate a breach in an AWS environment. We discussed, as part of the continuous security process, how to identify and simulate cloud breach scenarios in your AWS account to check if your security controls are able to prevent and/or detect some specific threats.
We simulated a credential theft scenario through the AWS Metadata Service, and how to use those credentials to access and download information from the S3 service.
AWS Credential Harvesting scenario outcome not prevented
In this post, we would like to share different security mechanisms that can be considered as additional layers to protect the credentials stored in the Metadata Service.
Are my credentials secured?
When a role is assigned to an EC2 instance, the credentials assigned to this role are stored into the instance Metadata Service. As we mentioned, the credentials are accessible by making a request to the HTTP endpoint http://169.254.169.254. These credentials are session-based, so they expire after a specific amount of time (between 1-6 hours), reducing the impact of exposure when a compromise happens.
However, although the impact of exposure is lower, there are certain classes of vulnerabilities that might allow an attacker to access these credentials, such as Server Side Request Forgery (SSRF) or XML External Entity (XXE) injection. In this case, a simple misconfiguration in the instance or any application in it could allow an attacker to steal the credentials and use them while they are still valid.
Managing access to these credentials could be implemented using the two methods described below.
Protecting Your Credentials Using a Metadata Proxy
Server Side Request Forgery[1] (SSRF) is a vulnerability that allows an attacker to trick the application into making an arbitrary HTTP request on its behalf. However, the attacker cannot control the HTTP headers of the affected request. Thus, because official AWS tools or SDKs often use well-known custom headers in their requests, it is possible to define a policy that only allows requests to the Metadata Service that include a known header, denying any other request.
The valid requests to the Metadata service could have one of these User-Agent headers configured:
- Aws-sdk-
- Botocore/
- Boto3/
- Aws-cli/
- aws-chalice/
So, it is possible to configure an application that filters those requests having the correct User-Agent, only allowing access to the Metadata Service through this application, working as a proxy.
Metadata Proxy schema
Using this proxy, any request not including a valid User-Agent header will be discarded, protecting your credentials against any vulnerability where the attacker could not have control over the HTTP headers sent in the request.
Remember that in the first part of this blog series, we had an EC2 instance with a misconfigured proxy that allowed the exploitation of SSRF vulnerabilities to access the metadata service.
Metadata credentials obtained using a SSRF vulnerability
In this example, we are going to use the AWS Metadata Proxy implemented by Netflix[2] to block any unintended request. The proxy listens in the 9090/tcp port and forwards requests to the Metadata service, only allowing those having one of the “User-Agent” headers listed above.
We have created a user in an EC2 instance called ‘proxy_user’. This user is going to execute a proxy within the instance. Moreover, we have modified iptables to only allow requests to the Metadata service through the proxy. More specifically, they only allow the user ‘proxy_user’ to directly access the 169.254.169.254 endpoint:
Iptables configuration to restrict access to Metadata service
Now, if an attacker tries to exploit a SSRF vulnerability as we showcased previously, the request returns a ‘401 – Unauthorized’ error:
Unauthorized access to Metadata Service prevented
This mechanism helps to prevent vulnerabilities such as SSRF in common user-interfacing web applications. When these web applications trigger an HTTP request, either as part of their normal workflow or manipulated by an attacker, they do not add the specific headers. Consequently, if a SSRF vulnerability is exploited, the headers that are whitelisted by the AWS Metadata Proxy are not added to the requests. Therefore not being able to access whatever is behind the proxy, in our case the AWS Metadata Service.
However, take into account that if an attacker is able to manipulate an application that sets those headers as part of their regular workflow when triggering HTTP requests to external entities, this security control would be rendered useless.
Protecting Your Credentials Using IMDSv2
In November 2019 AWS released IMDSv2 [3], an update to their Metadata Service containing an additional defense against this kind of threats, improving its security.
With IMDSv2, every request is now protected by session authentication. If IMDSv2 is configured, an additional HTTP PUT request is required to get a security token needed to make a request to the Metadata Service. So, all the requests to get information from the metadata service must include this token, mitigating SSRF, or other similar vulnerabilities.
To enable IMDSv2, the following aws-cli [4] command can be executed to update the Metadata options of the target instance.
Once security tokens are enabled in the instance, we can see how it is not possible to get the credentials from the Metadata Service without having the authentication token needed. Instead, a “401 Unauthorized error” is returned.
Protecting credentials using IMDSv2
Detecting an Instance Credential Exfiltration
AWS GuardDuty is a threat detection service that continuously monitors for malicious activity using some AWS logging subsystems as data sources.
If GuardDuty is enabled, a credential theft might be detected, having an alert like the one shown below:
GuardDuty alert when credentials are used outside EC2 service
This alert is triggered by GuardDuty whenever credentials related to an IAM instance profile are being used from an external IP address. However, GuardDuty refers to an IP address as external if it is outside of the whole EC2 service, not just outside of your EC2 instance. Given that, we have to consider that this detection can be bypassed if the stolen credentials are used from an EC2 instance in another AWS account.
Continuous Validation using AIQ platform
Let’s see what results we get if we execute an Assessment in AttackIQ Platform that contains a scenario to emulate the behavior described above, this time using any of the security mechanisms proposed.
We can see that the outcome of the scenario execution is shown as prevented because the Metadata service is not reachable.
AWS Credential Harvesting scenario outcome prevented
By executing the assessments continuously, it is possible to identify if any of the instances launched in our AWS account are misconfigured, using IMDSv1 (instead of IMDSv2) or exposing the Metadata Service.
AWS Credential Harvesting scenario result summary
Summary
At this point, the credential harvesting threat is solved, but the attacker could still have access to the information found in S3 buckets if credentials were already stolen.
In view of this, we need to include an additional security layer to prevent and detect any unauthorized access to data stored in S3. But let us see this in detail in the next post, stay tuned!
References
- [1] Server Side Request Forgery:
https://owasp.org/www-community/attacks/Server_Side_Request_Forgery - [2] AWS Metadata Proxy:
https://github.com/Netflix-Skunkworks/aws-metadata-proxy - [3] Configuring Instance Metadata Service:
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html - [4] Modify Instance Metadata options AWS CLi reference:
https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-metadata-options.html - [5] Add defense in depth against open firewalls:
https://aws.amazon.com/blogs/security/defense-in-depth-open-firewalls-reverse-proxies-ssrf-vulnerabilities-ec2-instance-metadata-service/
