750285 - Stop doing android-xul builds on try by default

Status: RESOLVED DUPLICATE of bug 774424

(Release Engineering :: General, defect)

Description

[Justin Wood (:Callek)]

So since we are beginning to do android-native beta's (off aurora) and dev is focused on native we should stop doing android-xul builds on try by default.

A rough ballpark figure threw out there was stated as ~25% tegra load drop if we turn these off, and was stated in IRC that blassey was ok/happy with this change.

Comment 1

[Phil Ringnalda (:philor)]

You would wind up getting more than that, because the options that the Tree Management Cabal would be ok/happy with that include not running it on try would be:

* call it tier 2, run it hidden on inbound and central and the release branches so that people interested in it could tell when it broke, but not require backouts over breaking it

* call it tier 3, and just stop running it at all anywhere other than try when someone goes out of their way to ask for it there

Comment 2

[Benjamin Smedberg]

Who is interested in it? Is this a configuration we want to support at all, or can we just remove the code?

I am personally worried that we're going to end up breaking content processes inbetween the time that fennec-xul stops being tested and b2g starts using content processes (at least running mochitests in that configuration). But I don't think that really affects this decision, unless the B2G folk want these to remain active specifically to keep those tests running.

Comment 3

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

If it's not overly taxing infra, I would like to keep the xul-fennec tests going for the sake of OOP mochitest.  We run cross-process reftests on linux-x11 so that's less critical.

We're spinning up content processes for b2g but it's not quite ready yet.

Comment 4

[Armen [:armenzg]]

Notice that this is for try only. We would not loose coverage on other trees.
Notice that this change would make developers to have to be explicit on try.

With this comment, would you still have any objections to make this change happen?

Comment 5

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

OOP mochitest is a Tier I configuration because of b2g.  Turning off these tests on try by default is bad for developer ergonomics.

A % load reduction doesn't help me understand the tradeoff.  But, for example, an argument like,

  Turning off fennec-xul builds+tests on try (by default) would reduce average native-fennec turnaround time by 2 hours.

would be pretty compelling right now.  Can someone explain in those kinds of terms what benefit we get from not running these jobs by default?

Comment 6

[Armen [:armenzg]]

Which of the whole set of suites would OOP be? [1] We can leave that on by default.

I can't specifically get in those terms but I know that our wait times are pretty bad.

From dev-tree.amanagement [2]
tegra: 6144
  0:     3227    52.52%
 15:      932    15.17%
 30:      489     7.96%
 45:      354     5.76%
 60:      217     3.53%
 75:      149     2.43%
 90+:      776    12.63% 

This means that almost 50% of our jobs don't start immediately.
We can also look at it that more than 30% of our jobs don't start withing 30 minutes.

It is also known that reducing the load helps our infra to have less infra failures (purples) which indirectly means reducing the number of jobs that get re-triggered.
We have other initiatives to improve some of these problems but it will take several weeks before getting substantially better.

[1] https://tbpl.mozilla.org/?jobname=Android%20XUL%20Tegra%20250%20mozilla-central%20opt%20test&rev=6e34995a746e
[2] https://groups.google.com/forum/?fromgroups#!topic/mozilla.dev.tree-management/fXZ1YWZFlTY

Comment 7

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

By far the most useful of these tests are the mochitests.  Do you have any data on how much load we would save by turning off all the reftests+crashtests+jsreftests by default?  (I.e. how much of the load they make up.)

Comment 8

[Armen [:armenzg]]

I don't have good data sources for that.

Based on:
https://tbpl.mozilla.org/?tree=Try&jobname=Android%20XUL%20Tegra%20250%20try%20opt&rev=a695632fc51a&noignore=1
That would 9 chunks (C,R & JS times 3).
Which would mean approximately 270 minutes per push.

This might be around 40% of the whole "Android XUL" jobs which would bring my grand total of 25% to be around 10% of savings.

This now looks minuscule.

https://tbpl.mozilla.org/?tree=Try&jobname=Android&rev=a695632fc51a&noignore=1

Comment 9

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

That compromise makes me happy.

Comment 10

[Phil Ringnalda (:philor)]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #4)
> Notice that this change would make developers to have to be explicit on try.

Pretty moot question, since you currently have no mechanism to exclude anything from trychooser's idea of all.

Comment 11

[Aki Sasaki (not active)]

I turned off android-xul everywhere in bug 774424, so we are indeed no longer building them by default on try.