Kanji
・ Cloud engineer / freelance ・ Born in 1993 ・ Born in Ehime Prefecture / Lives in Shibuya-ku, Tokyo ・ AWS history 5 years Profile details
Table of Contents
Amazon S3 (Simple Storage Service) is a scalable and highly durable object storage service . It makes it easy to store, retrieve, and manage data, and is used for a wide range of purposes such as backup, archiving, and data lakes.
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. Customers of all sizes and industries can use Amazon S3 to store and protect any amount of data for a range of use cases, such as data lakes, websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. Amazon S3 provides management features so that you can optimize, organize, and configure access to your data to meet your specific business, organizational, and compliance requirements. What is Amazon S3 - AWS Documentation
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. Customers of all sizes and industries can use Amazon S3 to store and protect any amount of data for a range of use cases, such as data lakes, websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. Amazon S3 provides management features so that you can optimize, organize, and configure access to your data to meet your specific business, organizational, and compliance requirements.
When building an environment on AWS, consider what types of data will be generated and how they will be stored. For example, the following types of data may be involved:
If you build a system using only EC2, you may first consider storing data on EBS volumes. However, for long-term data storage or high availability, storing data in S3 can improve cost efficiency and fault tolerance. Note that access speed is slower compared to data stored within the OS, so frequently accessed data should be stored on EBS, while data requiring long-term storage such as archives or backups should be stored in S3.
For “log data,” the following AWS services can use S3 buckets as output destinations. It is recommended to store CloudTrail logs in S3 buckets for long-term retention, making later audits and analysis easier.
Classify data according to its importance and confidentiality. For example, the following classifications can be considered. Based on the type of data classified here, consider appropriate access controls and encryption.
If necessary, further subdivide the data classification. For example, logs included in general data can be further classified as “audit logs,” “business logs,” and “system logs.” “Public data” can also be classified according to the scope of publication, such as “internal use,” “partner use,” and “public.”
Set appropriate access controls according to the data classification. Access control for S3 buckets can be implemented in the following ways:
Consider the following policies for each control method:
Use IAM policies to control access permissions to S3 buckets. IAM policies grant access permissions to users or roles for S3 buckets.
Controlling access by bucket name in IAM policies can become complicated as the number of buckets increases, so combine with KMS encryption for control. Here is an example of encrypting data classified in “1. Data Classification”:
By encrypting, even if you grant read-only policies such as AWS managed “ReadOnlyAccess,” users without access to the KMS key cannot view the data contents. Also, by controlling access to the KMS key with IAM policies, you can allow only specific users or roles to decrypt the data.
Use VPC endpoint policies to restrict access to specific S3 buckets. Like IAM policies, controlling by bucket name can become complicated, so consider restricting access to S3 buckets for specific AWS accounts or organizations using VPC endpoint policies. This helps prevent unauthorized data transfer and improves security.
For example, by setting a VPC endpoint policy like below, you can allow access to S3 buckets only for a specific organization. Access to AWS-managed S3 buckets for SSM Agent or AWS Systems Manager Patch Manager is also allowed additionally.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowOnlyOrgAccess", "Effect": "Allow", "Principal": "*", "Action": "s3:*", "Resource": "arn:aws:s3:::*", "Condition": { "StringEquals": { "aws:PrincipalOrgID": [ "o-xxxxxxx", ], "aws:ResourceOrgID": [ "o-xxxxxxx" ] } } }, { "Sid": "AllowAWSManagedBuckets", "Effect": "Allow", "Principal": "*", "Action": ["s3:GetObject"], "Resource": [ "arn:aws:s3:::al2023-repos-ap-northeast-1-de612dc2/*", "arn:aws:s3:::aws-windows-downloads-ap-northeast-1/*", "arn:aws:s3:::amazon-ssm-ap-northeast-1/*", "arn:aws:s3:::aws-ssm-ap-northeast-1/*", "arn:aws:s3:::patch-baseline-snapshot-ap-northeast-1/*", "arn:aws:s3:::aws-patch-manager-ap-northeast-1-552881074/*", "arn:aws:s3:::aws-patchmanager-macos-ap-northeast-1-552881074/*" ] } ] }
Use bucket policies to control access to S3 buckets. By default, unless you define Allow rules, only the AWS account that created the bucket has access. By defining Allow rules, you can grant access to users, roles, or AWS services outside the account that created the bucket. Grant only the minimum necessary permissions to reduce security risks.
Security Hub defines controls for bucket policies, and the following rules are recommended:
[S3.5] S3 general purpose buckets should require requests to use SSL. [S3.6] S3 general purpose bucket policies should restrict access to other AWS accounts.
The first rule recommends setting a bucket policy to deny requests that do not use SSL when accessing the S3 bucket. You can set a bucket policy like below to deny requests that do not use SSL:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "RequireSSL", "Effect": "Deny", "Principal": "*", "Action": "s3:*", "Resource": "arn:aws:s3:::example-bucket/*", "Condition": { "Bool": { "aws:SecureTransport": "false" } } } ] }
You can use bucket ACLs (Access Control Lists) to control access to S3 buckets. However, as described below, bucket ACLs are not recommended, and it is best to use IAM policies or bucket policies for access control.
Most modern use cases for Amazon S3 no longer require the use of ACLs. Therefore, unless you have an unusual situation where you need to control access to individual objects, it is recommended to disable ACLs. Disabling ACLs makes it easier to control access to all objects in the bucket using policies, regardless of who uploaded the objects.
Controlling object ownership and disabling bucket ACLs - Amazon Simple Storage Service
Since there are few opportunities to use them, explanations about bucket ACLs are omitted.
Use KMS (Key Management Service) to encrypt data in S3 buckets. Data encrypted with a KMS key is controlled by the KMS key policy. As mentioned in “1. IAM Policy,” you can restrict access to data in the bucket by controlling access to the KMS key with IAM policies. Here is a recap of setting KMS keys according to data classification:
SSE-S3 (S3 standard encryption) is server-side encryption provided by S3 that automatically encrypts data. It is free to use, but you cannot control access with IAM policies or KMS key policies. If access control is not required, you can easily encrypt data using SSE-S3.