772580 - nightly-only builds aren't buildable on try

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[John Ford [:jhford] CET/CEST Berlin Time]

After working out what try chooser magic is needed to trigger desktop b2g builds, it was determined that desktop b2g builds aren't available on try.

Please enable them on try.

I'm going to try to set up a local windows environment for bug 771095, but using try would be much easier.

Comment 1

[[e-mail] (:bhearsum)]

This affects b2g desktop, linux qt/rpm, and maybe other types of things too. l10n also isn't buildable on try, but for more complex reasons.

Comment 2

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Is this a DUP of bug#767501 ?

Comment 3

[Chris AtLee [:catlee]]

No, this is a more general problem than with just B2G.

Some types of builds are only done nightly. These builds are not possible to run on try at the moment. B2G desktop builds are currently this type of build.

Comment 4

[Phil Ringnalda (:philor)]

The other problem with nightly-only and not-on-try is that when I'm acting in accordance with the tree owner's rules, I need to hide them, because visible on mozilla-central and friends means that you are tier-1 and bustage must be backed out immediately, and not running on try means that you are by definition not tier-1. (Obviously, I don't always do what I'm supposed to do, or the Linux RPM nightlies would have been hidden long ago, but they've only been broken by code once, and I quite enjoyed backing that one out.)

Comment 5

[[e-mail] (:bhearsum)]

As a concrete example, Rafael was looking for a way to test bug 783546 and I ended up telling him just to push it to mozilla-central and have me trigger the one build he wanted. This isn't an every day problem, but it sucks really bad every time we hit it.

Comment 6

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: version 1 of patch

I am not sure why the [notsimple] tag was added, but this patch creates the builders and puts them in the schedulers.  I have tested by running checkconfig and by starting up a master and seeing the missing builders in the builder list.

Comment 7

[[e-mail] (:bhearsum)]

Comment on attachment 660177 [details] [diff] [review]
version 1 of patch

How does this interact with try syntax? Does this always trigger nightly-only builders? If not, under what conditions will they be triggered? We're in a load crunch right now, so anything that adds unnecessary load is highly undesirable.

Comment 8

[John Ford [:jhford] CET/CEST Berlin Time]

I don't know, but my guess is that like the other b2g builds, it won't run unless you use -p all

Comment 9

[John Ford [:jhford] CET/CEST Berlin Time]

hmm, it also has thunderbird enabled.  I don't know if that's desired but there is code to deal with turning off whole platforms:

    enabled_platforms = []
    for platform in sorted(config['platforms'].keys()):
        pf = config['platforms'][platform]
        if pf['stage_product'] in config['enabled_products']:
            enabled_platforms.append(platform)

    # generate a list of builders, nightly builders (names must be different)
    # for easy access
    for platform in enabled_platforms:
        ...

Comment 10

[[e-mail] (:bhearsum)]

I don't think this is an urgent enough problem to warrant hacking in a bad solution. This is an issue that requires some rethinking and redesign to fix properly.

Comment 11

[John Ford [:jhford] CET/CEST Berlin Time]

i think it's important to have Desktop B2G builds on try because:
- desktop B2G builds are not tier 1 supported specifically because of this issue
- desktop B2G builds are nearly impossible to work on because of this issue
- desktop B2G builds are very important for Firefox OS to be exposed to web developers

Comment 12

[Phil Ringnalda (:philor)]

Sorry if I misled you into thinking that this was the only reason they aren't tier 1. It's not my call, it's bsmedberg's, but I think a nightly-only tier 1 build would be horrible idea. Since it's going to burn at 3:45am, what that's saying is "whenever this is busted, I want edmorley to have to close every single trunk tree and then start bisecting hundreds of csets on try, while every desktop Firefox, mobile, and b2g developer sits idle, unable to push anywhere." Having it be tier 2 means some b2g developer needs to open https://tbpl.mozilla.org/?noignore=1&jobname=b2g in the morning, and if he sees bustage use his knowledge of the product to guess what's likely to have caused it and bisect locally, without stopping everyone else from pushing while he does.

Comment 13

[Chris AtLee [:catlee]]

John, I've taken a quick look at your patch, and can verify that it adds only the desktop builds to try builders/schedulers. As far as I can tell this doesn't impact thunderbird, nor are any other nightly only builds added.

So my only concern left here is the impact to try syntax and load on try. Under what cases are these builds triggered? If this requires '-p all' to get B2G desktop builds on try, I think that will be a blocking issue because of the additional load placed on try.

I think the correct solution would include your patch, plus supporting b2g in try syntax (bug 767501). If you're up for it, we could use the help on that bug as well.

Comment 14

[[e-mail] (:bhearsum)]

Removing review request until Chris' questions get answered.

Comment 15

[Chris AtLee [:catlee]]

Mostly fixed now with TC.