588546 - Unittests for all but win32 are not being triggered, packageTests step not run either

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[Lukas Blakk [:lsblakk] use ?needinfo]

bug 585098 was not enough to fix this. Right now http://hg.mozilla.org/build/buildbot-configs/file/dea6275083fa/mozilla2/release_master.py#l260 looks to http://hg.mozilla.org/build/buildbot-configs/file/dea6275083fa/mozilla2/release-firefox-mozilla-2.0.py#l43 for a list of what unittests to run.  We need to be able to generate test packages for all builders and use that unittestPlatforms flag to only set which platforms have their unittests run on the production-master instead of (or as well as) being run on the test-master minis.

Comment 1

[Armen [:armenzg]]

Attachment: package

Hi Ben,
For m-c and 2.0 releases we are running unit tests on the minis rather than on the builders and therefore we disabled them on the builders side to make releases consistent.
Unfortunately disabling these platforms on release_config.py sets packageTests to False which prevents the release builds from packaging the tests and does not send the sendchanges.

For dependent builds we have an extra parameter called packageTest which is used to enable packaging tests regardless if unit tests are run on the master or in another one.
http://mxr.mozilla.org/build/source/buildbotcustom/misc.py#612
612         # Allow for test packages on platforms that can't be tested
613         # on the same master.
614         packageTests = pf.get('packageTests', packageTests)

The patch I am asking for review does not need this variable and it simply makes *all* platforms to packageTests and to do the sendchanges. I can't think of any scenario as of now that we would not want to packageTest and do the sendchanges.

Let me know if this patch is good enough or if you would want me to add an extra parameter in release-firefox-mozilla-{central|2.0}.py.

Comment 2

[[e-mail] (:bhearsum)]

Comment on attachment 469066 [details] [diff] [review]
package

I think we're better off changing the condition on line 348 to 'if pf['enable_opt_unittests'] and then setting unittestPlatforms = enUSPlatforms in the m-c/2.0 configs. That way, all platforms we run unittest are listed as such in the release config regardless of where they run.

Deciding whether or not to package tests and where/how to do the sendchange can still be controlled by if platform in unittestPlatforms. For 1.9.2 this ends up triggering tests on the build machines, for m-c/2.0, on the test machines.

Comment 3

[Armen [:armenzg]]

Attachment: listen to enable_opt_unittests from config.py instead of the unittestPlatforms list

I have checked that the right builders load up when changing the different release configs (release-firefox-mozilla-{1.9.1,1.9.2,mozilla-central,mozilla-2.0}.py).

How does this look?

Comment 4

[[e-mail] (:bhearsum)]

Comment on attachment 469241 [details] [diff] [review]
listen to enable_opt_unittests from config.py instead of the unittestPlatforms list

Looks good to me, but can you update mozilla2-staging, too?

Comment 5

[Armen [:armenzg]]

Comment on attachment 469241 [details] [diff] [review]
listen to enable_opt_unittests from config.py instead of the unittestPlatforms list

production: http://hg.mozilla.org/build/buildbot-configs/rev/3421e6d08f6e
staging: http://hg.mozilla.org/build/buildbot-configs/rev/7bd7b20145ed

Comment 6

[Armen [:armenzg]]

Bustage fix: http://hg.mozilla.org/build/buildbot-configs/rev/5aeb32b3f87e

Not sure how it even worked to begin with.