Anonymous data access?

[meggart]

Currently the user is required to provide valid credentials in order to generate and aws_config. However, if a user only wants to access public buckets, this should not be necessary. For example in the python package s3fs https://github.com/dask/s3fs/blob/master/s3fs/core.py#L107 there is an option to connect to an S3 bucket without explicit authentication by the user. I think it is using boto3 for the connection, which explicitly allows to set aws_access_key_id=None, aws_secret_access_key=None https://github.com/boto/boto3/blob/02bdc5fa3fb6818f548a73d5e42dad84a946b52b/boto3/session.py#L48

Would anonymous access to public AWS S3 buckets be possible with this package as well or is there some fundamental design decision that makes this impossible?

[iamed2]

I tried to read the boto/botocore code and couldn't find how this is implemented. I think it would take some dedicated work to figure out the right way to do it. In the meantime, you could make HTTP requests to the S3 interface without any auth and that should work.

[meggart]

Ok, thanks for taking time to look into this. I experienced the same frustration when looking at the boto code, so we will probably try using HTTP directly for now.

[martin-gutwin]

Found a very simple fix for this, but I'm not sure if it satisfies the requirements for a pull request:
#79

[iamed2]

Now works for do_request as of #79, haven't checked whether services (e.g. the s3 function) will work. I thought they wouldn't initially but now I'm second-guessing myself.

[martin-gutwin]

Yes, it will if you create an AWSConfig object like the following and use it as follows:

aws = aws_config(creds=nothing)
s3_get(aws, bucket, key)

Of course you can still specify a region, if needed. This works because in AWSCore the request body and the AWSConfig are simply merged before passing them to a call of do_request. Only problem I see is for a future merge of #63 it probably won't work anymore, because of the struct using creds::AWSCredentials as a field. A simple fix might be creds::Union{AWSCredentials, Nothing}, but I'm not sure if that's appropriate.

[iamed2]

Nice, we should probably add a test for that for AWSS3.jl

Probably fine to have creds::Union{AWSCredentials, Nothing} in #63. It's stalled for now though.

[martin-gutwin]

I agree, but first there is this JuliaCloud/AWSS3.jl#40 issue I stumbled upon. I didn't really look into it deeply though.