784493 - change talos data format expected for --authfile

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[Jeff Hammel]

Talos currently looks for module level variables oauthKey and
oauthSecret:

http://hg.mozilla.org/build/talos/file/c986567c686a/talos/output.py#l373

The new format will use a top-level variable called 'datazilla':

datazilla = {'talos': {'oauthKey': 'foo', 'oauthSecret': 'bar'},
'jetperf': {'oauthKey': ...}, ...}

Talos needs to be changed to accomodate this.

Comment 1

[Jeff Hammel]

Attachment: fix

I'm not putting this up for review yet since it will be bitrotted by bug 784440 . I will probably either combine the patches or land the former first depending on time, etc

Comment 2

[Jeff Hammel]

Attachment: unbitrot

:bhearsum, do you mind reviewing this since :jmaher is out?  If not I can find someone else. Existing output from talos -n -d --develop -a ts --cycles 1 --datazilla-url https://datazilla.mozilla.org/test --datazilla-url https://datazilla.mozilla.org/foo --datazilla-url file://${PWD}/ts.json --authfile ~/mozilla/talos/passwords.py -e `which firefox`

<snip/>
NOISE: Outputting talos results => {'datazilla_urls': ['https://datazilla.mozilla.org/test', 'https://datazilla.mozilla.org/foo', 'file:///home/jhammel/mozilla/src/talos/src/talos/ts.json']}
NOISE: Outputting datazilla results to https://datazilla.mozilla.org/test
NOISE: datazilla: https//datazilla.mozilla.org/test; oauth=True
NOISE: Outputting datazilla results to https://datazilla.mozilla.org/foo
NOISE: No oauth credentials found for project 'foo' in '/home/jhammel/mozilla/talos/passwords.py'
NOISE: datazilla: https//datazilla.mozilla.org/foo; oauth=False
Error posting to https://datazilla.mozilla.org/foo: 403 FORBIDDEN
NOISE: Outputting datazilla results to file:///home/jhammel/mozilla/src/talos/src/talos/ts.json

Comment 3

[Jeff Hammel]

(Oh, and of course the foo project doesn't actually exist and the 403 Forbidden is quite expected)

Comment 4

[[e-mail] (:bhearsum)]

Comment on attachment 654029 [details] [diff] [review]
unbitrot

Review of attachment 654029 [details] [diff] [review]:
-----------------------------------------------------------------

> :bhearsum, do you mind reviewing this since :jmaher is out? 

This is what I get for reminding everyone that I hacked on Talos back in the day, eh?

::: talos/output.py
@@ +467,5 @@
> +                utils.noisy("No oauth credentials found for project '%s' in '%s'" % (project, self.authfile))
> +        utils.noisy("datazilla: %s//%s/%s; oauth=%s" % (scheme, server, project, bool(oauth_key and oauth_secret)))
> +
> +        # submit the request
> +        req = DatazillaRequest.create(scheme, server, project, oauth_key, oauth_secret, results)

Will DatazillaRequest.create() handle key/secret being None? If not, you may want to raise an error rather than making this call. Looks okay to me otherwise.

Comment 5

[Jeff Hammel]

(In reply to Ben Hearsum [:bhearsum] from comment #4)
> Comment on attachment 654029 [details] [diff] [review]
> unbitrot
> 
> Review of attachment 654029 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> > :bhearsum, do you mind reviewing this since :jmaher is out? 
> 
> This is what I get for reminding everyone that I hacked on Talos back in the
> day, eh?

:P  Thanks for the prompt review!

> ::: talos/output.py
> @@ +467,5 @@
> > +                utils.noisy("No oauth credentials found for project '%s' in '%s'" % (project, self.authfile))
> > +        utils.noisy("datazilla: %s//%s/%s; oauth=%s" % (scheme, server, project, bool(oauth_key and oauth_secret)))
> > +
> > +        # submit the request
> > +        req = DatazillaRequest.create(scheme, server, project, oauth_key, oauth_secret, results)
> 
> Will DatazillaRequest.create() handle key/secret being None? If not, you may
> want to raise an error rather than making this call. Looks okay to me
> otherwise.

Yes, I've specifically modified datazilla_client to support this use case (as deployed yesterday), since for the current case we don't have --authfile support in production. [Note also that the current logic still uses None/None here, its just a bit different of a check due to the data structure change]

Comment 6

[Jeff Hammel]

pushed to try: https://tbpl.mozilla.org/?tree=Try&rev=cca1a28f493b ; This is green except for a few pending jobs, so I'm going to go ahead and push to talos' repo to unblock other things.

Comment 7

[Jeff Hammel]

pushed: http://hg.mozilla.org/build/talos/rev/432ec68f5b74