1177141 - Make JSON files available from the Hello dashboard accessible without persona authentication

Status: RESOLVED FIXED

(Cloud Services Graveyard :: Metrics: Pipeline, defect, P1)

Description

[Romain Testard [:RT]]

The JSON files available on the Hello dashboard (https://metrics.services.mozilla.com/loop-server-dashboard/) are used for reporting on Google spreadsheets although since they require persona authentication the JSON data cannot be collected automatically from the spreadsheet, making the reporting on separate spreasheet very manual.
This bug is to remove the persona authentication on JSON file access.
A way forward may be to push the JSON files to an S3 drive (we may re-use our existing S3 drive used for TokBox data).

Comment 1

[Romain Testard [:RT]]

Katie, could you lease point us to someone who could help implement this?

Comment 2

[Katie Parlante]

Romain: its now in our triage queue.

Comment 3

[Wesley Dawson [:whd]]

https://github.com/mozilla-services/puppet-config/pull/1413

The files are updated daily and available to people with credentials for s3://net-mozaws-prod-metrics-data/loop-server-dashboard, which I can provide.

Comment 4

[Romain Testard [:RT]]

Thanks Wesley. The intention here is to access the files from a Google spreadsheet, is-it possible with the authentication solution in place?

Comment 5

[Wesley Dawson [:whd]]

http://engetc.com/projects/amazon-s3-api-binding-for-google-apps-script/ seems to be a possible solution for accessing the files in S3 from a Google spreadsheet. If you need unauthenticated HTTP(S) access, you can use the credentials I provide to sync the files somewhere and serve them from there.

Comment 6

[Katie Parlante]

Echoing what whd said, you should be able to write scripts that pull the files from S3. Some resources:

- Command line tool: https://github.com/aws/aws-cli 
- Documentation for using S3 from aws-cli: http://docs.aws.amazon.com/cli/latest/reference/s3/index.html
- Boto is the python sdk: https://github.com/boto/boto3

You guys will have to figure out the details, but accessing data via s3 is an industry standard and shouldn't present odd barriers the way persona auth does.

Comment 7

[Romain Testard [:RT]]

Thanks Wesley and Katie!
Jorge can you please confirm it provides you what you need for the spreadsheet integration?

Comment 8

[Thomas Huelbert]

marking as resolved, please re opening in the even this is not meeting your needs.