1198190 - Support cross-debug as a build variant, add 64_cross_debug mac variant

Status: RESOLVED FIXED

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[(not currently active) Ted Mielczarek]

For my cross-mac builds in bug 921040 I need to be able to select a separate sub-config. The existing mac sub-configs do at least one thing I don't want, they run `make check`. This patch makes "mac-cross" a valid build platform to enable that, and also copies the 64_debug mac sub-config to a mac-cross sub-config, removing the check-test step as well.

A try push with this change is here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=6177892c36cd

(The build is failing due to bug 1198179.)

Comment 1

[(not currently active) Ted Mielczarek]

Attachment: MozReview Request: bug 1198190 - Support cross-debug as a build variant, copy 64_debug mac sub-config to 64_cross_debug, disable check tests on mac-cross. r?jlund

bug 1198190 - Support mac-cross as a build platform, copy 64_debug mac sub-config to mac-cross, disable check tests on mac-cross. r?jlund

Comment 2

[Jordan Lund (:jlund)]

Comment on attachment 8652255 [details]
MozReview Request: bug 1198190 - Support cross-debug as a build variant, copy 64_debug mac sub-config to 64_cross_debug, disable check tests on mac-cross. r?jlund

https://reviewboard.mozilla.org/r/17105/#review15273

::: testing/mozharness/mozharness/mozilla/building/buildbase.py:360
(Diff revision 1)
>          if not cls.platform:

if cls.platform is not known, that's because --platform wasn't passed[1] so we are asking mozharness to guess the sub config (magic time!). So, you could simply pass --platform 'mac-cross' before --custom-build-variant-cfg.

actually, if we do stick with this patch, we should patch [1]'s help anyway to include the ability to pass 'mac-cross'



[1] http://mxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/building/buildbase.py#461

::: testing/mozharness/mozharness/mozilla/building/buildbase.py:368
(Diff revision 1)
> +                    if 'cross' in cfg_file_name:

as another option, we can extend the dynamic list of sub configs themselves. in order to do that, you would do something like:
```
diff --git a/testing/mozharness/mozharness/mozilla/building/buildbase.py b/testing/mozharness/mozharness/mozilla/building/buildbase.py
--- a/testing/mozharness/mozharness/mozilla/building/buildbase.py
+++ b/testing/mozharness/mozharness/mozilla/building/buildbase.py
@@ -305,18 +305,19 @@ class BuildOptionParser(object):
     # --custom-build-variant-cfg asan
     # and the script will pull up the appropriate path for the config
     # against the current platform and bits.
     # *It will warn and fail if there is not a config for the current
     # platform/bits
     build_variants = {
         'asan': 'builds/releng_sub_%s_configs/%s_asan.py',
         'tsan': 'builds/releng_sub_%s_configs/%s_tsan.py',
         'b2g-debug': 'b2g/releng_sub_%s_configs/%s_debug.py',
+        'cross-debug': 'builds/releng_sub_%s_configs/%s_cross_debug.py',
         'debug': 'builds/releng_sub_%s_configs/%s_debug.py',
         'asan-and-debug': 'builds/releng_sub_%s_configs/%s_asan_and_debug.py',
         'stat-and-debug': 'builds/releng_sub_%s_configs/%s_stat_and_debug.py',
         'mulet': 'builds/releng_sub_%s_configs/%s_mulet.py',
         'code-coverage': 'builds/releng_sub_%s_configs/%s_code_coverage.py',
         'graphene': 'builds/releng_sub_%s_configs/%s_graphene.py',
         'horizon': 'builds/releng_sub_%s_configs/%s_horizon.py',
         'source': 'builds/releng_sub_%s_configs/%s_source.py',
         'api-9': 'builds/releng_sub_%s_configs/%s_api_9.py',
```

then from tc, you call with `--custom-build-variant-cfg debug-cross` instead of `--custom-build-variant-cfg debug`

you would also have to:
```
mv testing/mozharness/configs/builds/releng_sub_mac-cross_configs/64_debug.py testing/mozharness/configs/builds/releng_sub_mac_configs/64_cross_debug.py
```

this has the benefit of not pollutiing the platform list but instead extending the subconfigs list (BuildOptionParser.build_variants). The latter seems to be the current idiom for extending buildbase.py. Though there is the side effect of having to name all your subconfigs with 'cross' as they will live in the rest of the mac sub configs (releng_sub_mac_configs).

I'd lean towards my above approach but feel free to push back if you don't agree

Comment 3

[Jordan Lund (:jlund)]

https://reviewboard.mozilla.org/r/17105/#review15273

> as another option, we can extend the dynamic list of sub configs themselves. in order to do that, you would do something like:
> ```
> diff --git a/testing/mozharness/mozharness/mozilla/building/buildbase.py b/testing/mozharness/mozharness/mozilla/building/buildbase.py
> --- a/testing/mozharness/mozharness/mozilla/building/buildbase.py
> +++ b/testing/mozharness/mozharness/mozilla/building/buildbase.py
> @@ -305,18 +305,19 @@ class BuildOptionParser(object):
>      # --custom-build-variant-cfg asan
>      # and the script will pull up the appropriate path for the config
>      # against the current platform and bits.
>      # *It will warn and fail if there is not a config for the current
>      # platform/bits
>      build_variants = {
>          'asan': 'builds/releng_sub_%s_configs/%s_asan.py',
>          'tsan': 'builds/releng_sub_%s_configs/%s_tsan.py',
>          'b2g-debug': 'b2g/releng_sub_%s_configs/%s_debug.py',
> +        'cross-debug': 'builds/releng_sub_%s_configs/%s_cross_debug.py',
>          'debug': 'builds/releng_sub_%s_configs/%s_debug.py',
>          'asan-and-debug': 'builds/releng_sub_%s_configs/%s_asan_and_debug.py',
>          'stat-and-debug': 'builds/releng_sub_%s_configs/%s_stat_and_debug.py',
>          'mulet': 'builds/releng_sub_%s_configs/%s_mulet.py',
>          'code-coverage': 'builds/releng_sub_%s_configs/%s_code_coverage.py',
>          'graphene': 'builds/releng_sub_%s_configs/%s_graphene.py',
>          'horizon': 'builds/releng_sub_%s_configs/%s_horizon.py',
>          'source': 'builds/releng_sub_%s_configs/%s_source.py',
>          'api-9': 'builds/releng_sub_%s_configs/%s_api_9.py',
> ```
> 
> then from tc, you call with `--custom-build-variant-cfg debug-cross` instead of `--custom-build-variant-cfg debug`
> 
> you would also have to:
> ```
> mv testing/mozharness/configs/builds/releng_sub_mac-cross_configs/64_debug.py testing/mozharness/configs/builds/releng_sub_mac_configs/64_cross_debug.py
> ```
> 
> this has the benefit of not pollutiing the platform list but instead extending the subconfigs list (BuildOptionParser.build_variants). The latter seems to be the current idiom for extending buildbase.py. Though there is the side effect of having to name all your subconfigs with 'cross' as they will live in the rest of the mac sub configs (releng_sub_mac_configs).
> 
> I'd lean towards my above approach but feel free to push back if you don't agree

I made a mistake, given my diff, you should do the following from tc call args:

s/--custom-build-variant-cfg debug-cross/--custom-build-variant-cfg cross-debug/

Comment 4

[(not currently active) Ted Mielczarek]

Thanks, either of those solutions sound OK. This patch was just "whatever I could do to make it work".

Comment 5

[(not currently active) Ted Mielczarek]

https://reviewboard.mozilla.org/r/17105/#review15273

> I made a mistake, given my diff, you should do the following from tc call args:
> 
> s/--custom-build-variant-cfg debug-cross/--custom-build-variant-cfg cross-debug/

Thanks for the suggestion!

Comment 6

[(not currently active) Ted Mielczarek]

Comment on attachment 8652255 [details]
MozReview Request: bug 1198190 - Support cross-debug as a build variant, copy 64_debug mac sub-config to 64_cross_debug, disable check tests on mac-cross. r?jlund

bug 1198190 - Support cross-debug as a build variant, copy 64_debug mac sub-config to 64_cross_debug, disable check tests on mac-cross. r?jlund

Comment 7

[(not currently active) Ted Mielczarek]

also needed a patch for bug 1202811 (the desktop-build image changed to CentOS 6 since I last tried this):
https://treeherder.mozilla.org/#/jobs?repo=try&revision=bb8411975181

Comment 8

[(not currently active) Ted Mielczarek]

That build is still busted, but that's because of other fallout from the CentOS 6 switch. The mozharness bits seem fine.

Comment 9

[Jordan Lund (:jlund)]

Comment on attachment 8652255 [details]
MozReview Request: bug 1198190 - Support cross-debug as a build variant, copy 64_debug mac sub-config to 64_cross_debug, disable check tests on mac-cross. r?jlund

https://reviewboard.mozilla.org/r/17105/#review16711

looks great!

Comment 10

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/eaa94ea819ee

Comment 11

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/eaa94ea819ee