I cannot fetch temporary AWS credentials and then upload files

RESOLVED FIXED

Status

RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: armenzg, Unassigned)

Tracking

Details

Attachments

(1 attachment)

(Reporter)

Description

2 years ago
Created attachment 8762179 [details]
store_on_s3.py

I'm using this client:
https://tools.taskcluster.net/auth/clients/#mozilla-ldap%252farmenzg@mozilla.com%252fupload_to_S3

With this scope:
>  auth:aws-s3:read-write:tc-gp-public-31d/ateam/pulse-action-dev/*

I'm calling the script attached.

If I call without validate, it will go a little further:
> bucket = conn.get_bucket(bucket, validate=True)

Either there or when we try to upload the file I get a 403.
Easily done:
  https://gist.github.com/jonasfj/c129216da7ea73c1fa96ff80210ac507

boto has a lot of obscurities that makes it hard to use.
Lack of region support and other things. I bet our mistake here is using boto, when there is boto3:
https://boto3.readthedocs.io/en/latest/reference/services/s3.html

Newer libraries like aws-sdk for node is much better.
I wouldn't be surprised if that goes for boto3 too :)
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → WORKSFORME
(Reporter)

Comment 2

2 years ago
I should have tested this during our time in London.
I modified your script to point to 'ateam/pulse-actions-dev', however, I still get the cryptic 403 message:

(TC_scheduling) armenzg@armenzg-thinkpad:~/repos/TC_developer_scheduling_experiments$ python test.py 
Traceback (most recent call last):
  File "test.py", line 15, in <module>
    k.set_contents_from_string("Hello World")
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/boto/s3/key.py", line 1426, in set_contents_from_string
    encrypt_key=encrypt_key)
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/boto/s3/key.py", line 1293, in set_contents_from_file
    chunked_transfer=chunked_transfer, size=size)
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/boto/s3/key.py", line 750, in send_file
    chunked_transfer=chunked_transfer, size=size)
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/boto/s3/key.py", line 951, in _send_file_internal
    query_args=query_args
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/boto/s3/connection.py", line 668, in make_request
    retry_handler=retry_handler
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/boto/connection.py", line 1071, in make_request
    retry_handler=retry_handler)
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/boto/connection.py", line 940, in _mexe
    request.body, request.headers)
  File "/home/armenzg/venv/TC_scheduling/local/lib/python2.7/site-packages/boto/s3/key.py", line 884, in sender
    response.status, response.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>325994C384DF6EE9</RequestId><HostId>UfYxSDUM/dx86vtKTaXvibFVf+UIs4X3lJLiMuqmdmLZ5L8dv0xUnIOgyI20xskHC+oPFdOJ9VE=</HostId></Error>
Status: RESOLVED → REOPENED
Flags: needinfo?(jopsen)
Resolution: WORKSFORME → ---
(Reporter)

Comment 3

2 years ago
Redirecting to dustin while jonasfj is away.
Flags: needinfo?(jopsen) → needinfo?(dustin)
(Reporter)

Comment 4

2 years ago
Here's the gist: https://gist.github.com/armenzg/91bb3f786eecd51f96021e505e206ee2

My apologies!
(Reporter)

Comment 5

2 years ago
There was an error in my script with jonasfj pointed to me.
Status: REOPENED → RESOLVED
Last Resolved: 2 years ago2 years ago
Flags: needinfo?(dustin)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.