Actively Monitoring your AWS S3 Bucket Security
In a previous blog, I discussed Preventive Measures on S3 Security. In this blog, I will discuss the Monitoring Measures.
Visibility into your AWS account activity is a key aspect of security and operational best practices. To instrument S3 buckets and monitor for suspicious activity, an in-depth monitoring-based approach can go a long way in enhancing your organization’s data access and security efforts.
1. Amazon S3 CloudWatch Metrics
S3 provides three types of metrics for your bucket:
-
Storage Metrics (Free)
-
Request Metrics (Paid)
-
Data Transfer Metrics (Paid)
Request metrics ‐ Monitor Amazon S3 requests to quickly identify and act on operational issues. The metrics are available at 1-minute intervals after some latency to process. For an additional charge, request metrics can be used to track your S3 bucket usage in a more granular way.
You can record the actions taken by users, roles, or AWS services on Amazon S3 resources by maintaining log records for auditing and compliance purposes. To do this, you can use Amazon S3 server access logging, AWS CloudTrail logs, or a combination of both. It is recommended that you use AWS CloudTrail for logging bucket and object-level actions for your Amazon S3 resources.
2. CloudTrail
Amazon S3 is integrated with AWS CloudTrail, which records actions taken by a user, role, or an AWS service in Amazon S3. CloudTrail captures a subset of API calls for Amazon S3 as events, including calls from the Amazon S3 console and code calls to the Amazon S3 APIs. If you create a trail, you can enable continuous CloudTrail events to an Amazon S3 bucket, including events for Amazon S3. If you don’t configure a trail, you can still view the most recent events in the CloudTrail console in Event history. Using the information collected by CloudTrail, you can determine the request that was made to Amazon S3, the IP address from which the request was made, who, when it was made, and other additional details.
3. S3 Access Logging
Monitoring S3 buckets is an essential first step towards ensuring better data security in your organization. Bucket access logging empowers your security teams to identify attempts of malicious activity within your environment.
To track requests for access to your bucket, you can enable server access logging. Each access log record provides details about a single access request, such as the requester, bucket name, request time, request action, response status, and an error code, if relevant.
There is no extra charge for enabling server access logging on an Amazon S3 bucket, and you are not charged when the logs are PUT to your bucket either.
S3 bucket access logging captures all requests made to a bucket, such as PUT, GET, and DELETE actions. Bucket access logging is a recommended security best practice that can help teams uphold compliance standards or identify unauthorized access to your data. In particular, S3 access logs will be one of the first sources required in any data breach investigation as they track data access patterns over your buckets.
-
Source Bucket: The S3 bucket to monitor
-
Target Bucket: The S3 bucket that will receive S3 access logs from source buckets
-
Access Logs: Information on requests made to your buckets
It’s important to note that target buckets must reside in the same region and account as the source buckets.
S3 access logs can be queried with AWS Athena, a service to query data on S3 with SQL.
Following are the recommendations -
-
Enable S3 Server Access Logging for all buckets: This feature is provided for free, and the only cost associated with it is the storage cost of the logs, which is low. The logs provide high-value context that can be used during an investigation, especially if unauthorized data access is of concern.
-
Enable CloudTrail Data Events on sensitive buckets: Due to the cost of enabling Data Events, we advise you only to enable it when required. It could include buckets with sensitive PII or financial data.
4. AWS Config
AWS Config enables continuous monitoring of your AWS resources, making it simple to assess, audit, and record resource configurations and changes. AWS Config does this through the use of rules that define the desired configuration state of your AWS resources. AWS Config provides several AWS managed rules that address a wide range of security concerns. You can also create custom rules to codify your compliance requirements using AWS Lambda functions.
AWS Config monitors our Amazon Simple Storage Service (S3) bucket ACLs and policies for violations that allow public read or public write access. If AWS Config finds a policy violation, we’ll have it trigger an Amazon CloudWatch Event rule which, triggers an AWS Lambda function that either corrects the S3 bucket ACL or notifies you via Amazon Simple Notification Service (Amazon SNS) that the policy is in violation and allows public read or public write access.
We’ll show you how to do this in five main steps:
-
Enable AWS Config to monitor Amazon S3 bucket ACLs and policies for compliance violations.
-
Create an IAM Role and Policy that grants Lambda function permissions to read S3 bucket policies and send alerts through SNS.
-
Create and configure a CloudWatch Events rule that triggers the Lambda function when AWS Config detects an S3 bucket ACL or policy violation.
-
Create a Lambda function that uses the IAM role to review S3 bucket ACLs and policies, correct the ACLs, and notify your team of out-of-compliance policies.
-
Verify the monitoring solution.
AWS Config Rules incur costs in each region that they are active.
5. Amazon Macie
Amazon Macie is a fully managed data security and privacy service that uses machine learning and pattern matching to discover and protect your sensitive data in AWS.
As organizations manage growing volumes of data, identifying and protecting their sensitive data at scale can become increasingly complex, expensive, and time-consuming. Amazon Macie automates the discovery of sensitive data at scale and lowers the cost of protecting your data.
Macie automatically provides an inventory of Amazon S3 buckets, including a list of unencrypted buckets, publicly accessible buckets, and buckets shared with AWS accounts outside those defined in AWS Organizations. After that, Macie applies machine learning and pattern matching techniques to the buckets you select, to identify and alert you about sensitive data, such as personally identifiable information (PII).
Macie’s alerts, or findings, can be searched and filtered in the AWS Management Console and sent to Amazon CloudWatch Events for easy integration with the existing workflow or event management systems. Or, they can be used in combined with AWS services, such as AWS Step Functions, to take automated remediation actions.
The service also allows you to define your custom sensitive data types to discover and protect the sensitive data that may be unique to your business or use case.
Macie provides native multi-account support so you can view your data security posture across your entire S3 environment from a single Macie administrator account.
Macie also gives you the flexibility to identify sensitive data residing in other data stores by temporarily moving it to S3. For example, you can initiate Amazon Relational Database Service (RDS) or Amazon Aurora snapshots to export data in these services to Amazon S3, where it can be evaluated for sensitive data using Macie. This allows you to utilize Macie to help you maintain data privacy and security.
Amazon Macie is a regional service.
Use Amazon Macie to automatically scan your S3 buckets for security concerns, including public settings. This is a paid service, so be aware of the pricing. Once Amazon Macie is set up, it runs automatically. You need to react to the alerts.
Pricing — With Amazon Macie, you are charged based on the number of Amazon S3 buckets evaluated for bucket-level security and access controls and the quantity of data processed for sensitive data discovery.
Macie only charges for the bytes processed in supported object types that it inspects. As part of Macie sensitive data discovery jobs, you will incur the standard Amazon S3 charges for GET and LIST requests.
6. Amazon GuardDuty
Amazon GuardDuty is a threat detection service that continuously monitors for malicious activities and unauthorized behaviors to protect your AWS accounts, workloads, and data stored in Amazon S3.
With GuardDuty, you now have an intelligent and cost-effective option for continuous threat detection in AWS. GuardDuty analyzes tens of billions of events across multiple AWS data sources, such as AWS CloudTrail event logs, Amazon VPC Flow Logs, and DNS logs. With a few clicks in the AWS Management Console, GuardDuty can be enabled with no software or hardware to deploy or maintain.
To use GuardDuty, you must first enable it. When GuardDuty is enabled, a GuardDuty detector is created in that region. GuardDuty can be enabled in a region through the console or using the createDetector API.
By integrating with Amazon CloudWatch Events, GuardDuty alerts are actionable, easy to aggregate across multiple accounts, and straightforward to push into existing event management and workflow systems.
Amazon GuardDuty continuously monitors and analyzes AWS CloudTrail S3 data events (e.g. GetObject, ListObjects, DeleteObject) to detect suspicious activity indicating a bucket compromise, such as suspicious data access patterns that identify credential misuse, unusual S3 API activity from a remote host, unauthorized S3 access from known malicious IP addresses, and API calls to retrieve data in S3 buckets from users that have no prior history of accessing the bucket or invoked from an unusual location across all of your Amazon S3 buckets.
When GuardDuty detects suspicious or unexpected behavior in your AWS environment, it generates a finding. A finding is a notification that contains details about a potential security issue that GuardDuty discovers. The finding details include information about what and when it happened, which AWS resources were involved in the suspicious activity, and other information.
GuardDuty produces detailed security findings to the console and Amazon EventBridge, making alerts actionable and easy to integrate into existing event management and workflow systems. It also triggers automated remediation actions using AWS Lambda. You can optionally deliver findings to an S3 bucket to aggregate results from multiple regions and integrate them with third-party security analysis tools.
ThreatPurpose: ResourceTypeAffected/ThreatFamilyName.ThreatFamilyVariant!Artifact
Impact:S3/ObjectDelete.Unusual
An IAM entity invoked an API used to delete data in an S3 bucket.
GuardDuty gives you security at scale, no matter your size.
7. AWS Trusted Advisor
AWS Trusted Advisor is an online tool that provides real-time guidance to help you provide your resources following AWS best practices. Trusted Advisor checks help optimize your AWS infrastructure, increase security and performance, reduce your overall costs, and monitor service limits.
AWS Trusted Advisor provides best practices (or checks) in four categories: cost optimization, security, fault tolerance, and performance improvement. The status of the check on the dashboard page is shown through color-coding:
Red: action recommended
Yellow: investigation recommended
Green: no problem detected
AWS Trusted Advisor now helps all customers secure their data better by providing the S3 Bucket Permissions check for free!
Previously available only to Business and Enterprise support customers, this check identifies S3 buckets that are publicly accessible due to ACLs or policies that allow read/write access to any user.
Amazon S3 Bucket Permissions
Checks buckets in Amazon Simple Storage Service (Amazon S3) that have open access permissions. Bucket permissions that grant List access to everyone can result in higher than expected charges if unintended users list objects in the bucket at a high frequency. Bucket permissions that give Upload/Delete access create potential security vulnerabilities by allowing anyone to add, modify, or remove items in a bucket. This check examines explicit bucket permissions, but it does not read associated bucket policies that might override the bucket permissions.
Amazon S3 Bucket Logging
Checks the configuration of Amazon Simple Storage Service (Amazon S3) buckets that have server access logging enabled. When logging is initially enabled, the configuration is automatically validated; however, future modifications can result in logging failures. This check examines explicit Amazon S3 bucket permissions, but it does not read associated bucket policies that might override the bucket permissions.
For all your cloud based services needs, reach out to us at Clairvoyant.