823135 - Show build load in trychooser webpage

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Benoit Girard (:BenWa)]

If we add CORS headers to the load information provided here:
http://build.mozilla.org/builds/pending/try.html

We could have http://trychooser.pub.build.mozilla.org/ do a cross-domain request of the load information and display which server have the highest load so users could elect to do load balancing.

Comment 1

[Benoit Girard (:BenWa)]

I lied, we don't need CORS header for an un authenticated cross domain GET. We just need to patch trychooser

Comment 2

[Benoit Girard (:BenWa)]

Attachment: patch

This only shows build load and the mapping isn't perfect yet but it shows a decent overview.

Comment 3

[Benoit Girard (:BenWa)]

Attachment: patch

Forgot hg add

Comment 4

[Boris Zbarsky [:bzbarsky]]

> I lied, we don't need CORS header for an un authenticated cross domain GET.

Er... I thought you do.  At least "Access-Control-Allow-Origin: *"?

Comment 5

[Ed Morley [:emorley]]

I really like this! :-)

Perhaps we could have:
* linux (pending_build_count - pending_test_count)
* linux64 (pending_build_count - pending_test_count)
etc?

pending_build_count would come from:
http://builddata.pub.build.mozilla.org/reports/pending/pending_compile_try.txt

pending_test_count from:
http://builddata.pub.build.mozilla.org/reports/pending/pending_test_try.txt
+
http://builddata.pub.build.mozilla.org/reports/pending/pending_test_nontry.txt

Comment 6

[Benoit Girard (:BenWa)]

(In reply to Ed Morley (Away 20th Dec-2nd Jan) [UTC+0; email:edmorley@moco] from comment #5)
> I really like this! :-)
> 
> Perhaps we could have:
> * linux (pending_build_count - pending_test_count)
> * linux64 (pending_build_count - pending_test_count)
> etc?

Let's start with that. We should later add a proper table layout:
Platform                      Build Queue   Test Queue
Linux                           50            1000
...

Comment 7

[Benoit Girard (:BenWa)]

(In reply to Boris Zbarsky (:bz) from comment #4)
> > I lied, we don't need CORS header for an un authenticated cross domain GET.
> 
> Er... I thought you do.  At least "Access-Control-Allow-Origin: *"?

I assumed so but it's not the case. It also works in other browsers. You can see the XHR request in the patch. I checked the headers we get back and 'Access-Control-Allow-Origin' isn't in there.

Comment 8

[Ed Morley [:emorley]]

Here's my initial pass at mappings, please can people sanity check/fill in the gaps:
https://etherpad.mozilla.org/VmTLRubLS9

Comment 9

[Boris Zbarsky [:bzbarsky]]

> I checked the headers we get back and 'Access-Control-Allow-Origin' isn't in there.

I don't know how you checked, but:

~% wget -S -O /dev/null http://build.mozilla.org/builds/pending/pending_test_try.txt |& grep Access-Control-Allow-Origin
  Access-Control-Allow-Origin: *

so the site is in fact already sending the right CORS headers.  Which is why it works.

Comment 10

[Benoit Girard (:BenWa)]

I checked using our devtools console:
http://dl.dropbox.com/u/10523664/Screenshots/83.png

I'll check with devtools why that wouldn't show up.

Comment 11

[Boris Zbarsky [:bzbarsky]]

That's the 304.  Then we read the actual 200 from our cache, in your case, and it has whatever headers we cached it with.

Try clearing your cache and seeing what the devtools console has to say for the first load after that.

Comment 12

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to Ed Morley (Away 20th Dec-2nd Jan) [UTC+0; email:edmorley@moco] from comment #8)
> Here's my initial pass at mappings, please can people sanity check/fill in
> the gaps:
> https://etherpad.mozilla.org/VmTLRubLS9

There's danger in using the information in the .txt files because it's based on a mapping of job name to slave pool which has not kept up with the new tests, linux+android builds moving to mock, win32 builds happening on the win64 pool etc. An extra mapping from pool back to platform adds more fun. 

In principle we can get buildbot to spit out that mapping and then plug that into my scripts, but you might want to look at using regex's on data like
  https://secure.pub.build.mozilla.org/buildapi/pending?format=json
in the meantime.

Comment 13

[Benoit Girard (:BenWa)]

Attachment: patch

Demo:
http://people.mozilla.com/~bgirard/trychooser/

Comment 14

[[e-mail] (:bhearsum)]

(In reply to Nick Thomas [:nthomas] (away Dec 21 - Jan 3) from comment #12)
> (In reply to Ed Morley (Away 20th Dec-2nd Jan) [UTC+0; email:edmorley@moco]
> from comment #8)
> > Here's my initial pass at mappings, please can people sanity check/fill in
> > the gaps:
> > https://etherpad.mozilla.org/VmTLRubLS9
> 
> There's danger in using the information in the .txt files because it's based
> on a mapping of job name to slave pool which has not kept up with the new
> tests, linux+android builds moving to mock, win32 builds happening on the
> win64 pool etc. An extra mapping from pool back to platform adds more fun. 
> 
> In principle we can get buildbot to spit out that mapping and then plug that
> into my scripts, but you might want to look at using regex's on data like
>   https://secure.pub.build.mozilla.org/buildapi/pending?format=json
> in the meantime.

I went over the data with Benoit with the other day and it seems like it's in good enough shape for now. Since Mock has made its way down the trains, you can assume that the linux + linux-hp + linux64 pending platforms do virtually all jobs for the "linux" and "linux64" try chooser platforms (esr10 being the only exception that I can think of). This will become less accurate the next time there's a change to slaves that rides the train. I _think_ that as long as try chooser maintains a correct mapping to mozilla-central, that it's accurate enough for answering the question "roughly, which platform is loaded the most", since inbound+try is the majority of our load.

Comment 15

[Ed Morley [:emorley]]

Thank you for confirming Ben! :-)

Comment 16

[[e-mail] (:bhearsum)]

Comment on attachment 694561 [details] [diff] [review]
patch

Review of attachment 694561 [details] [diff] [review]:
-----------------------------------------------------------------

A couple of glitches when I tested this locally:
* Text overlap in the column headers
* NaN in one the rows

https://people.mozilla.com/~bhearsum/sattap/62cb7f35.png

::: trychooser/index.html
@@ +43,5 @@
> +            <li>
> +                <label>Platform</label>
> +                <span id="test_header" class="test_queue">Test</span>
> +                <span id="build_header" class="build_queue">Build</span>
> +            </li>

As you mentioned in an earlier comment, this would probably be better as a table. Though if you can get this to display fine I'm not picky.

::: trychooser/tryload.js
@@ +7,5 @@
> +  ["linux ", "linux64"],
> +  ["mac", "macosx64"],
> +  ["win", "win32"],
> +  ["linux ", "android"],
> +];

Is this used? I'm not seeing any references to "platformMap" except here.

Comment 17

[Benoit Girard (:BenWa)]

Attachment: patch v2

I just increased the width by 15 pixel this should be sufficient for any font at that point size. I forgot to init win7/winxp to 0 which will fix the NaN problem.

Comment 18

[[e-mail] (:bhearsum)]

Comment on attachment 694880 [details] [diff] [review]
patch v2

Review of attachment 694880 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good to me.

Comment 19

[[e-mail] (:bhearsum)]

Feel free to land this anytime. I'll deploy it next week.

Comment 20

[[e-mail] (:bhearsum)]

Comment on attachment 694880 [details] [diff] [review]
patch v2

I landed & deployed this.