Closed
Bug 1054308
Opened 7 years ago
Closed 6 years ago
Investigate switching Thunderbird comm-central MozMill tests to mozharness
Categories
(Thunderbird :: Testing Infrastructure, defect)
Thunderbird
Testing Infrastructure
Tracking
(Not tracked)
RESOLVED
FIXED
Thunderbird 39.0
People
(Reporter: standard8, Assigned: jcranmer)
References
Details
Attachments
(4 files, 3 obsolete files)
10.95 KB,
patch
|
jlund
:
review+
|
Details | Diff | Splinter Review |
3.85 KB,
patch
|
ted
:
review+
|
Details | Diff | Splinter Review |
3.37 KB,
patch
|
Callek
:
review+
|
Details | Diff | Splinter Review |
4.07 KB,
patch
|
Callek
:
review+
|
Details | Diff | Splinter Review |
Things to do here: - Switch in-tree mozmill to use mozharness - or all the moztools(?) stuff surrounding it. - Work out how to do that in packaged format & switch tbpl
Will the tests need to be changed in any way (the testing code itself) ?
Assignee | ||
Comment 2•7 years ago
|
||
This is as far as I've gotten: * Add a mozmill suite to mozharness * Unpack extra directories * Install mozmill virtualenv +--> This is where things go wrong. Creating a custom virtualenv is problematic in its own right, since it screws up our ability to run the correct virtualenv later. I can try hacking around that, but I'm guessing that the eventual reviewer won't like that. The other option is to try ditching the install mozmill virtualenv script altogether and manually run it in the already-existing virtualenv on test machines. Here, though, we run into the problem that mozmill wants a pinned mozrunner==2.5.13 and the virtualenv already has mozrunner 6.2.
Assignee | ||
Comment 3•7 years ago
|
||
Conclusion: mozmill and jsbridge do not work on the newer version of mozrunner already in the virtualenv.
Assignee | ||
Comment 4•6 years ago
|
||
Not sure who needs to/can review this mozilla-central patch. This is needed because the desktop_unittests script fails if the testsuite name isn't in the in-tree config.
Assignee | ||
Comment 5•6 years ago
|
||
Comment on attachment 8560820 [details] [diff] [review] [mozilla-central] Add the in-tree config for mozmill tests Er, oops, guess I was too premature on this patch. :-(
Attachment #8560820 -
Attachment is obsolete: true
Attachment #8560820 -
Flags: review?(ted)
Assignee | ||
Updated•6 years ago
|
tracking-thunderbird38:
--- → ?
Comment 6•6 years ago
|
||
What is the motivation for tracking-thunderbird38? Why is this important?
Assignee | ||
Comment 7•6 years ago
|
||
(In reply to Kent James (:rkent) from comment #6) > What is the motivation for tracking-thunderbird38? Why is this important? This reduces the maintenance burden of the buildbot infrastructure for testing. The most important piece for TB38 tracking is the mozilla-central change (to add mozmill to the list of acceptable testsuites); the buildbot changes could be done later and backported as necessary.
Comment 8•6 years ago
|
||
OK let's try to push this. You probably have a reasonable chance of getting it aurora I suppose if we miss the cutoff.
Assignee | ||
Comment 9•6 years ago
|
||
Don't know if there's a better reviewer, but here goes. I've mostly copy-pasted the mozharness stuff from xpcshell.
Attachment #8564611 -
Flags: review?(jlund)
Assignee | ||
Comment 10•6 years ago
|
||
This should be the correct change to make to mozilla-central to add the in-tree config stuff.
Attachment #8564612 -
Flags: review?(ted)
Assignee | ||
Comment 11•6 years ago
|
||
I don't have the full buildbot accumen to build/test that patch, but once the above two land and go into production, it should be a matter of copying <http://mxr.mozilla.org/build/source/buildbot-configs/mozilla-tests/thunderbird_config.py#129> with an s/xpcshell/mozmill/g, and then adding in the code to make it ride the 38 train.
Flags: needinfo?(bugspam.Callek)
Comment 12•6 years ago
|
||
Comment on attachment 8564611 [details] [diff] [review] [mozharness] Add mozmill to the list of suites [checkin: see comment 15] Review of attachment 8564611 [details] [diff] [review]: ----------------------------------------------------------------- looks good so far. Have you tried this out against anything? I'm fine landing and then testing if we roll it out slowly. i.e. on a project branch. ::: scripts/desktop_unittest.py @@ +509,5 @@ > + abs_app_extensions_dir, > + overwrite='overwrite_if_exists') > + modules = ['jsbridge', 'mozmill'] > + for module in modules: > + self.install_module(module=os.path.join(dirs['abs_mozmill_dir'], where is this getting installed? It appears off glance that we are adding them to a current virtualenv and it sounded like you suggested that has conflicting modules: https://bugzilla.mozilla.org/show_bug.cgi?id=1054308#c3 ?
Assignee | ||
Comment 13•6 years ago
|
||
(In reply to Jordan Lund (:jlund) from comment #12) > looks good so far. Have you tried this out against anything? I'm fine > landing and then testing if we roll it out slowly. i.e. on a project branch. I've tested the mozharness in a local semi-reproduction of Linux buildbots in a docker container and on an OS X builder. > where is this getting installed? It appears off glance that we are adding > them to a current virtualenv and it sounded like you suggested that has > conflicting modules: https://bugzilla.mozilla.org/show_bug.cgi?id=1054308#c3 > ? I fixed the conflicting module issue in bug 1121107.
Comment 14•6 years ago
|
||
Comment on attachment 8564611 [details] [diff] [review] [mozharness] Add mozmill to the list of suites [checkin: see comment 15] Review of attachment 8564611 [details] [diff] [review]: ----------------------------------------------------------------- ship it
Attachment #8564611 -
Flags: review?(jlund) → review+
Assignee | ||
Comment 15•6 years ago
|
||
Comment on attachment 8564611 [details] [diff] [review] [mozharness] Add mozmill to the list of suites [checkin: see comment 15] Checked into the mozharness repository: http://hg.mozilla.org/build/mozharness/
Attachment #8564611 -
Attachment description: [mozharness] Add mozmill to the list of suites → [mozharness] Add mozmill to the list of suites [checkin: see comment 15]
Assignee | ||
Comment 16•6 years ago
|
||
Sorry, that last comment should say http://hg.mozilla.org/build/mozharness/rev/08cd66cbfacd
Comment 17•6 years ago
|
||
Comment on attachment 8564612 [details] [diff] [review] [mozilla-central] Add the in-tree config for mozmill tests [checkin: see comment 18] Review of attachment 8564612 [details] [diff] [review]: ----------------------------------------------------------------- Sorry for the delay!
Attachment #8564612 -
Flags: review?(ted) → review+
Assignee | ||
Comment 18•6 years ago
|
||
Pushed the m-c change: https://hg.mozilla.org/integration/mozilla-inbound/rev/1f73582b2c39
Keywords: leave-open
Assignee | ||
Updated•6 years ago
|
Attachment #8564612 -
Attachment description: [mozilla-central] Add the in-tree config for mozmill tests → [mozilla-central] Add the in-tree config for mozmill tests [checkin: see comment 18]
Assignee | ||
Comment 20•6 years ago
|
||
I've not tested this patch, and I have no idea how.
Flags: needinfo?(bugspam.Callek)
Attachment #8570984 -
Flags: review?(bugspam.Callek)
Updated•6 years ago
|
Attachment #8570984 -
Flags: review?(bugspam.Callek) → review+
Comment 21•6 years ago
|
||
Comment on attachment 8570984 [details] [diff] [review] [buildbot] Switch TB 39+ to mozharness Review of attachment 8570984 [details] [diff] [review]: ----------------------------------------------------------------- Per request over IRC, landed: https://hg.mozilla.org/build/buildbot-configs/rev/8342d69fe6b7
Comment 22•6 years ago
|
||
In production: https://hg.mozilla.org/build/buildbot-configs/rev/8342d69fe6b7
Comment 23•6 years ago
|
||
(In reply to Chris Cooper [:coop] from comment #22) > In production: https://hg.mozilla.org/build/buildbot-configs/rev/8342d69fe6b7 Err I had to backout due to an issue with buildbot-configs testing (duplicate buildernames), it seems I forgot to comment here... I'll have to try and help joshua on this one I suspect.
Assignee | ||
Comment 24•6 years ago
|
||
Assignee | ||
Updated•6 years ago
|
Attachment #8570984 -
Attachment is obsolete: true
Assignee | ||
Updated•6 years ago
|
Attachment #8580167 -
Attachment description: Port Thunderbird mozmill tests to mozharness, → [buildbot] Port Thunderbird mozmill tests to mozharness,
Attachment #8580167 -
Flags: review?(bugspam.Callek)
Comment 25•6 years ago
|
||
Comment on attachment 8580167 [details] [diff] [review] [buildbot] Port Thunderbird mozmill tests to mozharness, r+ pending a passing result at: https://travis-ci.org/Callek/build-buildbot-configs/builds/55140520
Attachment #8580167 -
Flags: review?(bugspam.Callek) → review+
Assignee | ||
Comment 26•6 years ago
|
||
(In reply to Joshua Cranmer [:jcranmer] from comment #24) > Created attachment 8580167 [details] [diff] [review] > [buildbot] Port Thunderbird mozmill tests to mozharness, Pushed as https://hg.mozilla.org/build/buildbot-configs/rev/5c63a6b51c55
Comment 27•6 years ago
|
||
In production: https://hg.mozilla.org/build/buildbot-configs/rev/5c63a6b51c55
Comment 28•6 years ago
|
||
Since March 23, Mozmill runs in comm-central are failing with: self.buildbot_config isn't set after running read_buildbot_config! Running post_fatal callback... Exiting -1 Is that because we need the second half of this bug landed?
Comment 29•6 years ago
|
||
16:15:26 INFO - Running main action method: read_buildbot_config 16:15:26 INFO - buildbot_json_path is not set. Skipping... 16:15:26 FATAL - self.buildbot_config isn't set after running read_buildbot_config! 16:15:26 FATAL - Running post_fatal callback... so it looks like we need to add: 'buildbot_json_path': os.environ.get('PROPERTIES_FILE'), to the mozharness config for mozmill
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 61•6 years ago
|
||
jlund, can you look at Callek's comment 29 and see what we can do to get mozmill up again on comm-central? It has been completely failing for days.
Flags: needinfo?(jlund)
Comment 62•6 years ago
|
||
apologies for not getting to this yet. I am on point for merge & uplift release work today so I won't be able to look into this until my EOD.
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Description
•