So, after turning on "Block public access (bucket settings)" feature in an S3 bucket (like shown in the screenshot below).
Django storages started failing with 403 Forbidden
error when trying to upload a file to that bucket:
>>> s3_storage = S3BotoStorage()
>>> s3_storage.save('/test.txt', BytesIO(b'hello world'))
...
S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error>{gfm-js-extract-pre-1}<Message>Access Denied</Message><RequestId>1F57D22859459D90</RequestId><HostId>+0tfaa09UEOWVFba6lfdS2gPpHZOabih6+CfKPrqOJgaLvzAvmf+TviQ1p2+wMJhxqk+j1b3jds=</HostId></Error>
I tried to give the user I'm using permission to perform any S3 action on the given bucket by updating its policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::996345623464:user/foobar_production"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::foobar"
}
]
}
But, even after granting AmazonS3FullAccess
permission via IAM, it still fails with the same error.
However, what I noticed is, other operations like listing objects in a directory works just fine and this proves that the user I'm using has actually access to the bucket:
>>> s3_storage.listdir('/')
(['a'], ['b'])
After some trial and error, it turned out that this error was related to the default ACL settings AWS_DEFAULT_ACL
where -at the time of writing- is set to public-read
by default which clashes with "Block public access" setting that prevents objects from being publically accessible and that's why we're getting the 403 Forbidden
error.
So, to resolve this, we should avoid giving access to the public when uploading a file and there are few ways to do it:
- set
AWS_DEFAULT_ACL
to anything exceptpublic-read
andpublic-read-write
likeprivate
(you can read more about available ACLs in the official AWS documentation):
BE CAREFUL this setting affects all S3 storage classes in your app
AWS_DEFAULT_ACL = 'private'
- specify
default_acl
parameter when initiatingS3BotoStorage
:
s3_storage = S3BotoStorage(default_acl='private')
- create a new storage class that inherits
S3BotoStorage
and definesdefault_acl
:
class MyAwesomeS3Storage(S3BotoStorage):
default_acl = 'private'
s3_storage = MyAwesomeS3Storage()