794165 - Talos should not attempt to send results when run with the --develop flag

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[Nathan Froyd [:froydnj]]

12:45 < froydnj> jmaher: so stuff gets sent even with --develop?
12:46 < jmaher> froydnj: it will timeout unless you are on the proper network
12:46 < froydnj> lovely
12:46 < jmaher> that is a good idea though; if we are using --develop it should only use file urls
12:47 < jmaher> froydnj: what type of behavior are you seeing?
12:48 < froydnj> jmaher: what do you mean?
12:48 < jmaher> froydnj: I thought you were doing something and it was failing
12:49 < froydnj> jmaher: oh.  no.
12:49 < froydnj> jmaher: but it's a bit disconcerting when documentation for --develop says it won't send anything, yet the output from running talos 
                 contains messages about sending things to datazilla
12:50 < froydnj> if those timeouts are making it run longer, then boo to that too ;)
12:50 < jhammel> yes, we always send to datazilla, which will actually hopefully be fixed soon(ish)
12:50 < jmaher> froydnj: I think that is a bug that recently happened since we can actually access datazilla from the public now
12:50 < jhammel> though that is --datazilla-url

I *think* the outcome of this conversation is that results are sent.  They shouldn't be (particularly if the server is not even accessible to the outside world).

Comment 1

[Jeff Hammel]

So is the intention to check if the results urls are file:// for --develop runs and err out if they are not?

Comment 2

[Jeff Hammel]

I'm tempted to WONTFIX this. IMHO, we shouldn't send to datazilla by default.  IIRC, we are blocking on mozharness for this, although we *could* modify the existing buildbot configs to do this. In this case we will only send if --datazilla-url is specified . In any case, I'd rather fix this in the sensible way: namely, respecting what is specified on the command line vs working around it.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

if we use --develop, I would like to set these defaults:
--results_url local.out
--datazilla-url local.json

Comment 4

[Ankit Goyal]

Attachment: Patch

If --develop is used and --results_url and --datazilla-url are not passed explicitly, they will default to local.out and local.json respectively.

Comment 5

[Ankit Goyal]

Attachment: Patch

Created another one.

In this even if --requets_url and --datazilla-url are passed, they will default if --develop is set

This was not clear to me, hence I created 2 patches.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8541391 [details] [diff] [review]
Patch

the other patch has a simpler solution for developers.

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8541409 [details] [diff] [review]
Patch

One thing to consider is printing a message at the end of the test run that indicates where there data can be found.


Sometime shortly after this:
http://hg.mozilla.org/build/talos/file/tip/talos/run_tests.py#l280

add:
if develop:
    print "Thanks for running Talos locally results are in %s and %s" % (results_out, results_json)

^ I just used psuedocode there, it should be more realistic.

I marked this as r+, well done.  Please take a little bit of extra time and add this one line.  I will see when you upload it and commit it to the repository.

Comment 8

[Ankit Goyal]

Attachment: Updated

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8541555 [details] [diff] [review]
Updated

Review of attachment 8541555 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks!

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

https://hg.mozilla.org/build/talos/rev/43e43d31d9e6

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

Ankit, thanks again for helping out here.  I have been always thinking I should solve this bug, but never got around to doing it- this will save many people loads of time.

Do let me know if you have further interest in contributing.  We can find another bug in Talos or another project.

Comment 12

[Ankit Goyal]

It's weird to say, but would you change `assigned to` me. I am keeping a count. ;-)

Comment 13

[Joel Maher ( :jmaher ) (UTC -8)]

apologies, I forget this sometimes.