Django storages failed to upload files to AWS S3 after blocking public access in the bucket settings

So, after turning on "Block public access (bucket settings)" feature in an S3 bucket (like shown in the screenshot below).

screenshot of

Django storages started failing with 403 Forbidden error when trying to upload a file to that bucket:

>>> s3_storage = S3BotoStorage()
>>> s3_storage.save('/test.txt', BytesIO(b'hello world'))
...

S3ResponseError: S3ResponseError: 403 Forbidden

<?xml version="1.0" encoding="UTF-8"?>  
<Error>{gfm-js-extract-pre-1}<Message>Access Denied</Message><RequestId>1F57D22859459D90</RequestId><HostId>+0tfaa09UEOWVFba6lfdS2gPpHZOabih6+CfKPrqOJgaLvzAvmf+TviQ1p2+wMJhxqk+j1b3jds=</HostId></Error>

I tried to give the user I'm using permission to perform any S3 action on the given bucket by updating its policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::996345623464:user/foobar_production"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::foobar"
        }
    ]
}

But, even after granting AmazonS3FullAccess permission via IAM, it still fails with the same error.

However, what I noticed is, other operations like listing objects in a directory works just fine and this proves that the user I'm using has actually access to the bucket:

>>> s3_storage.listdir('/')
(['a'], ['b'])

After some trial and error, it turned out that this error was related to the default ACL settings AWS_DEFAULT_ACL where -at the time of writing- is set to public-read by default which clashes with "Block public access" setting that prevents objects from being publically accessible and that's why we're getting the 403 Forbidden error.

So, to resolve this, we should avoid giving access to the public when uploading a file and there are few ways to do it:

set AWS_DEFAULT_ACL to anything except public-read and public-read-write like private (you can read more about available ACLs in the official AWS documentation):
BE CAREFUL this setting affects all S3 storage classes in your app

AWS_DEFAULT_ACL = 'private'

specify default_acl parameter when initiating S3BotoStorage:

s3_storage = S3BotoStorage(default_acl='private')

create a new storage class that inherits S3BotoStorage and defines default_acl:

class MyAwesomeS3Storage(S3BotoStorage):  
  default_acl = 'private'

s3_storage = MyAwesomeS3Storage()

Menu