756818 - Disable trobocheck3 everywhere except Try, until it gets fixed

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Phil Ringnalda (:philor)]

Even by the pathetically low standards of Android tests, trobocheck3 just doesn't run. It fails somewhere around 6 or 8 times before you get a green, except for standout runs of bad luck like https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=ab3f805f3210&jobname=remote-trobocheck3 where we're at 21 failures and counting.

We don't have the capacity for that kind of failure, so I filed bug 756817 about fixing it to run at least just as poorly as other Android tests, and in the meantime we should turn it off everywhere except Try.

Comment 1

[Phil Ringnalda (:philor)]

Attachment: fix?

I *think* this is how you run something try-only. Either that or it's how you completely break configs, one or the other.

Comment 2

[Armen [:armenzg]]

Attachment: enable trobocheck3 only for try

That was a very good stab at it but our configs are funny and won't make things make sense. You have to add "_tests" at the end of it.

jmaher, what do you think? Should we do this? or you have something that will make trobocheck3 more reliable?

Here is how I tested the patch in case you wanted to know:
  508  ./test-masters.sh 
  509  ./setup-master.py -l
  510  ./setup-master.py master bm19-tests1-tegra
  511  cd master
  512  buildbot start .
* loaded http://localhost:8201/builders to see
  513  hg diff
  514  vi config.py
  515  buildbot reconfig .
* loaded http://localhost:8201/builders to confirm

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 627214 [details] [diff] [review]
enable trobocheck3 only for try

we could find the magic thing to make robocheck work, but that could take 1 day or a month, lets make this try only for now and maybe if all goes week next week we will enable this everywhere.

Comment 4

[Armen [:armenzg]]

Comment on attachment 627214 [details] [diff] [review]
enable trobocheck3 only for try

Okie dokie.

Thanks philor!

Comment 5

[Armen [:armenzg]]

http://hg.mozilla.org/build/buildbot-configs/rev/98832a595d3b

Comment 6

[Armen [:armenzg]]

Attachment: [tools] add torobocheck3

Comment 7

[Armen [:armenzg]]

Comment on attachment 627239 [details] [diff] [review]
[tools] add torobocheck3

checked-in, deployed and visible on trychooser page.

https://wiki.mozilla.org/ReleaseEngineering/How_To/Update_the_Try_Syntax