786984 - Thunderbird's MozMill builders don't obey trychooser options

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Mark Banner (:standard8)]

The MozMill tests for Firefox were originally set up with a type of "mozmill-all", however Thunderbird's tests are just "mozmill".

Thunderbird and Firefox do have slightly different routines for mozmill tests. I'm not sure if we want to merge those together or extend the options on trychooser (which could potentially confuse folks).

Comment 1

[Phil Ringnalda (:philor)]

Probably what would least confuse folks would be to put a "mozmill" in a Thunderbird-specific bit of trychooser, and remove the "mozmill-all" which hasn't actually done anything since February 2011 when it was commented out of configs.

Comment 2

[Mark Banner (:standard8)]

That would seem to be a simple replacement just on the trychooser html.

What I don't quite understand is that even putting in "-u none" into the patch arguments, the mozmill tests still get run on try-comm-central.

Comment 3

[Mark Banner (:standard8)]

Actually, now I look at it, "-u none" doesn't affect xpcshell or mozmill tests for try-comm-central...

... and having said that, I now see the code at fault. I'll see if I can put a patch up later, or at least explain the issue.

Comment 4

[Mark Banner (:standard8)]

Attachment: buildbotcustom patch

This shows the basic issue for the unit test builder not respecting the trychooser arguments.

In misc.py the dep builders use enable_try in the branch config to work out whether or not to use the try chooser.

However the unit test builders don't have that and just rely on the branch being 'try' rather than 'try-comm-central'.

I've not tested these patches, although I believe they will work.

Comment 5

[Mark Banner (:standard8)]

Attachment: buildbot-configs patch

Add enable_try to the relevant branches for the unittest suites. I couldn't find a b2g_config.py (like there is for dep builders), so I'm assuming there isn't one atm.

Comment 6

[Mark Banner (:standard8)]

Attachment: Tools patch - patch trychooser for mozmill-all -> mozmill

As per philor's suggestion, this changes mozmill-all to mozmill on the trychooser selection page as the Firefox builders aren't using mozmill-all atm.

Comment 7

[Mark Banner (:standard8)]

Comment on attachment 656923 [details] [diff] [review]
buildbot-configs patch

http://hg.mozilla.org/build/buildbot-configs/rev/ef98a7702a61

Comment 8

[Mark Banner (:standard8)]

Comment on attachment 656921 [details] [diff] [review]
buildbotcustom patch

Checked in: http://hg.mozilla.org/build/buildbotcustom/rev/15cf147265b4

Comment 9

[Mark Banner (:standard8)]

Comment on attachment 656924 [details] [diff] [review]
Tools patch - patch trychooser for mozmill-all -> mozmill

Checked in: http://hg.mozilla.org/build/tools/rev/a52bc9b58615

Comment 10

[[e-mail] (:bhearsum)]

This made it to production today.

Comment 11

[Mark Banner (:standard8)]

At the moment, this appears to have stopped tests running on try-comm-central. try is ok though, so I'm now pondering the issue.

Comment 12

[Mark Banner (:standard8)]

Attachment: Fix try-comm-central's broken unit tests

Ok, the problem here is that trychooser has a nice subtle hard-coded 'try' for the branch name when generating the unit test builder names. The platform builders don't use this, which explains why we've not noticed issues on the platform selection side.

I've changed the trychooser to accept a buildbotBranch value (slightly different to the "branch" for the unittest scheduler, as I understand that is something more like platform-try-comm-central-debug etc) and pass that through to the chooser to use when generating the builder pretty names. Hopefully that will work.

I don't think we need to worry about the "TB " prefix as that is included in the platform name right from in the main configs.

I've added a test for the try-comm-central branch, although I can't actually run that on my local machine, but I think it should pass.

Comment 13

[Rail Aliiev [:rail]]

Comment on attachment 657816 [details] [diff] [review]
Fix try-comm-central's broken unit tests

lgtm

Comment 14

[Mark Banner (:standard8)]

Comment on attachment 657816 [details] [diff] [review]
Fix try-comm-central's broken unit tests

http://hg.mozilla.org/build/buildbotcustom/rev/e6d93b6a8359

Comment 15

[Armen [:armenzg]]

Updated try syntax chooser.

Instructions on how to do so are in here:
https://wiki.mozilla.org/ReleaseEngineering/How_To/Update_the_Try_Syntax

Comment 16

[Armen [:armenzg]]

Live on production masters.

Comment 17

[Mark Banner (:standard8)]

Just verified it is now working fine on ThunderbirdTry and choosing xpcshell and/or mozmill works correctly, so we can save builder time properly :-)