Verify that trychooser's tryload is looking at the right names for pending jobs and adding them up into the correct buckets

RESOLVED FIXED

Status

Release Engineering
General
RESOLVED FIXED
3 years ago
2 months ago

People

(Reporter: philor, Assigned: nthomas)

Tracking

({trychooser})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

3 years ago
This will require either being able to see the source that generates http://builddata.pub.build.mozilla.org/reports/pending/pending_compile_try.txt and http://builddata.pub.build.mozilla.org/reports/pending/pending_test_try.txt and http://builddata.pub.build.mozilla.org/reports/pending/pending_test_nontry.txt, or perhaps (if, as I suspect, you would need to see both the source and a database dump) will require looking at those three URLs during the middle of a weekday when everything is pending long enough to catch it in there.

One (minor one) I did see tonight is that winxp/win7 test load should be "load_try_test["winxp-ix"] + load_nontry_test["winxp-ix"] + load_nontry_test["winxp"]" because despite using the same winxp-ix slaves, tests on the addon-sdk tree are listed as winxp/win7 rather than winxp-ix/win7-ix.

win64 is probably wrong: I'm pretty sure there's one pool of win64 slaves, and both win32 and win64 should be the sum of win32+win64, and since we stopped doing win64 tests it should be N/A.

Depending on how you look at it, linux/emulator tests should include fedora* (debug b-c and emulator reftests) and ubuntu*-hw (talos).
(Reporter)

Comment 1

3 years ago
So far, the only thing else I've seen from loading them at random times is that something is getting miscounted as try builds on linux-hp, since it will claim 85 pending at times when there are only a dozen. Maybe fedora* and ubuntu*-hw? I've never seen them mentioned as pending tests.
(Assignee)

Comment 2

3 years ago
The problem is that I have an old, hardcoded, mapping of builder names to platforms. It quickly gets out of date.

Much better would be to redo it using
  http://builddata.pub.build.mozilla.org/reports/allthethings.json.gz  (also not .gz but 10MB!)
which catlee set that up recently. It checks for changes in buildbotcustom, buildbot-config, and tools repos every 15 minutes and dumps information out of actual master instances, so it's stays up to date. We can then reliably map from buildername to a slavepool, which is similar to a platform. With some human friendly naming for pools, the pending/running can get put into the text files trychooser consumes, and also be plotted using flot/d3/some client-side plotting (so that it scales automatically as pools come and go).
(Assignee)

Comment 3

3 years ago
Created attachment 8447797 [details] [diff] [review]
[tools] Update trychooser for new platform names

I've fixed up my pending list -> slave pool mapping, and simplified it quite a bit. eg all the flavours of linux build on 'linux64'; similarly we only have win64 compile slaves. There are also includes a few consistency changes (s/mac10.6-rev4/mac10.6/), s/win8/win8-ix/), and drops tegra (a little early, but we're nearly there). Adds support for linux test running on both hw and vm.

This would get deployed at around the same time as I update http://builddata.pub.build.mozilla.org/reports/pending/ with the matching changes.
Assignee: nobody → nthomas
Status: NEW → ASSIGNED
Attachment #8447797 - Flags: review?(philringnalda)
(Reporter)

Comment 4

3 years ago
Comment on attachment 8447797 [details] [diff] [review]
[tools] Update trychooser for new platform names

Sometime in the future, I look forward to someone coming up with a Grand Unified Theory Of When To Sum And When To Max (I vaguely think summing linux tests is probably wrong, but don't much care). Meantime, this'll do fine.
Attachment #8447797 - Flags: review?(philringnalda) → review+
(Assignee)

Comment 5

3 years ago
Comment on attachment 8447797 [details] [diff] [review]
[tools] Update trychooser for new platform names

https://hg.mozilla.org/build/tools/rev/f55676fd01cf
Attachment #8447797 - Flags: checked-in+
(Assignee)

Comment 6

3 years ago
http://builddata.pub.build.mozilla.org/reports/pending/pending.html & running.html, so we have accurate conversion to slave pool. Trychooser is updated so that we use that.
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED

Updated

3 years ago
Depends on: 1052187
(Assignee)

Comment 7

3 years ago
This was already deployed. In other news, backend which assigns jobs to pools now follows allthethings.json, so it stays up to date without intervention (new platforms would need consideration in try chooser).
No longer depends on: 1052187
Component: Tools → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.