787008 - get rid of lion_branches overriding

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

Attachment: kill lion_branches (with fire!)

In config.py right now, we have PLATFORMS set to the old, pre-r5 defaults. Later on, we override those defaults for all branches except esr10. This is silly, we should set the defaults properly and override esr10. In the future, we should be setting PLATFORMS up correctly from the start rather than doing this overriding.

The attached patch does this. I also have a metadiff of before/after config.py printouts to attach.

Comment 1

[[e-mail] (:bhearsum)]

Attachment: metadiff

Comment 2

[[e-mail] (:bhearsum)]

Comment on attachment 656823 [details] [diff] [review]
metadiff

>--- /home/bhearsum/tmp/before.txt	2012-08-30 07:40:28.311515665 -0400
>+++ /home/bhearsum/tmp/after.txt	2012-08-30 07:54:14.455486445 -0400
>@@ -1035,7 +1035,6 @@
>                                     'SYMBOL_SERVER_SSH_KEY': '/Users/cltbld/.ssh/ffxbld_dsa',
>                                     'SYMBOL_SERVER_USER': 'ffxbld',
>                                     'TINDERBOX_OUTPUT': '1'},
>-                            'l10n_slaves_key': 'macosx64-lion',
>                             'mozconfig': 'macosx64/accessibility/nightly',
>                             'nightly_signing_servers': 'mac-dep-signing',
>                             'packageTests': True,

I'm 99% sure we don't need this l10n_slaves_key stuff anymore. Rail, I know you understand it better than me, what do you think? I can do a dump_master.py run if it helps...

>@@ -6419,11 +6411,15 @@
>                                 'builds_before_reboot': 1,
>                                 'dep_signing_servers': 'mac-dep-signing',
>                                 'download_symbols': True,
>+                                'enable_ccache': False,

We're now being explicit about no ccache for esr10.

>                                 'enable_checktests': True,
>                                 'enable_shared_checkouts': True,
>                                 'enable_shark': True,
>                                 'enable_unittests': False,
>-                                'env': {'HG_SHARE_BASE_DIR': '/builds/hg-shared',
>+                                'env': {'CCACHE_COMPRESS': '1',
>+                                        'CCACHE_DIR': '/builds/ccache',
>+                                        'CCACHE_UMASK': '002',
>+                                        'HG_SHARE_BASE_DIR': '/builds/hg-shared',
>                                         'LC_ALL': 'C',
>                                         'MOZ_CRASHREPORTER_NO_REPORT': '1',
>                                         'MOZ_OBJDIR': 'obj-firefox',

But still inheriting the env vars, which will be unused.

>@@ -34136,7 +34100,10 @@
>                                                       'MOZ_CRASHREPORTER_NO_REPORT': '1',
>                                                       'MOZ_OBJDIR': 'obj-firefox',
>                                                       'XPCOM_DEBUG_BREAK': 'stack-and-abort'},
>-                                    'macosx64-lion': {'CHOWN_REVERT': '~/bin/chown_revert',
>+                                    'macosx64-lion': {'CCACHE_COMPRESS': '1',
>+                                                      'CCACHE_DIR': '/builds/ccache',
>+                                                      'CCACHE_UMASK': '002',
>+                                                      'CHOWN_REVERT': '~/bin/chown_revert',
>                                                       'CHOWN_ROOT': '~/bin/chown_root',
>                                                       'HG_SHARE_BASE_DIR': '/builds/hg-shared',
>                                                       'LC_ALL': 'C',
>@@ -34150,7 +34117,10 @@
>                                                       'SYMBOL_SERVER_SSH_KEY': '/Users/cltbld/.ssh/ffxbld_dsa',
>                                                       'SYMBOL_SERVER_USER': 'ffxbld',
>                                                       'TINDERBOX_OUTPUT': '1'},
>-                                    'macosx64-lion-debug': {'HG_SHARE_BASE_DIR': '/builds/hg-shared',
>+                                    'macosx64-lion-debug': {'CCACHE_COMPRESS': '1',
>+                                                            'CCACHE_DIR': '/builds/ccache',
>+                                                            'CCACHE_UMASK': '002',
>+                                                            'HG_SHARE_BASE_DIR': '/builds/hg-shared',
>                                                             'LC_ALL': 'C',
>                                                             'MOZ_CRASHREPORTER_NO_REPORT': '1',
>                                                             'MOZ_OBJDIR': 'obj-firefox',
>@@ -34239,7 +34209,10 @@
>                                                             'MOZ_CRASHREPORTER_NO_REPORT': '1',
>                                                             'MOZ_OBJDIR': 'obj-firefox',
>                                                             'XPCOM_DEBUG_BREAK': 'stack-and-abort'},
>-                                          'macosx64-lion': {'CHOWN_REVERT': '~/bin/chown_revert',
>+                                          'macosx64-lion': {'CCACHE_COMPRESS': '1',
>+                                                            'CCACHE_DIR': '/builds/ccache',
>+                                                            'CCACHE_UMASK': '002',
>+                                                            'CHOWN_REVERT': '~/bin/chown_revert',
>                                                             'CHOWN_ROOT': '~/bin/chown_root',
>                                                             'HG_SHARE_BASE_DIR': '/builds/hg-shared',
>                                                             'LC_ALL': 'C',
>@@ -34253,7 +34226,10 @@
>                                                             'SYMBOL_SERVER_SSH_KEY': '/Users/cltbld/.ssh/ffxbld_dsa',
>                                                             'SYMBOL_SERVER_USER': 'ffxbld',
>                                                             'TINDERBOX_OUTPUT': '1'},
>-                                          'macosx64-lion-debug': {'HG_SHARE_BASE_DIR': '/builds/hg-shared',
>+                                          'macosx64-lion-debug': {'CCACHE_COMPRESS': '1',
>+                                                                  'CCACHE_DIR': '/builds/ccache',
>+                                                                  'CCACHE_UMASK': '002',
>+                                                                  'HG_SHARE_BASE_DIR': '/builds/hg-shared',
>                                                                   'LC_ALL': 'C',
>                                                                   'MOZ_CRASHREPORTER_NO_REPORT': '1',
>                                                                   'MOZ_OBJDIR': 'obj-firefox',

These blocks are enabling ccache on spidermonkey for m-i/ionmonkey, which we didn't have before because of the previous implementation.

Comment 3

[Rail Aliiev [:rail]]

Comment on attachment 656822 [details] [diff] [review]
kill lion_branches (with fire!)

Review of attachment 656822 [details] [diff] [review]:
-----------------------------------------------------------------

* You forgot about thunderbird_config.py
* I have a patch for l10n_slaves_keys

Incoming!

::: mozilla/config.py
@@ -1401,5 @@
> -            else:
> -                slave_list = SLAVES['macosx64-lion']
> -            if BRANCHES[b]['platforms'].has_key(p):
> -                BRANCHES[b]['platforms'][p]['slaves'] = slave_list
> -                BRANCHES[b]['platforms'][p]['l10n_slaves_key'] = 'macosx64-lion'

You need to specify l10n_slaves_key above because of http://hg.mozilla.org/build/buildbotcustom/file/3add56b1f225/misc.py#l915

::: mozilla/l10n-changesets_mobile-beta.json
@@ +3,5 @@
>      "revision": "3e911ef81869",
>      "platforms": ["android"]
>    },
>    "cs": {
> +    "revision": "9caef02df392",

Probably something from other patch. :)

Comment 5

[Rail Aliiev [:rail]]

Attachment: Get rid of l10n_slaves_key, use pf['slaves'] instead of

We use pf['l10n_slaves_key'] or platform.split('-')[0] to determine which key to use to get l10n slaves for a platform. We can use pf.get('l10n_slaves', pf['slaves']) instead and set l10n_slaves for platforms which use different set of slaves for repacks. PLATFORM snippet for win32 (the only platform which uses different set of slaves for builds and repacks):

...
'slaves': SLAVES['win64'],
'l10n_slaves': SLAVES['win32'],
...

config patch incoming.

Comment 6

[Rail Aliiev [:rail]]

Attachment: kill lion_branches, l10n_slaves_key (with fire!)

Ben's patch without android changesets + thunderbird conifgs changes + kill l10n_slaves_key

Comment 7

[Rail Aliiev [:rail]]

Attachment: Firefox metadiff

Comment 8

[Rail Aliiev [:rail]]

Attachment: TB metadiff

Comment 9

[[e-mail] (:bhearsum)]

Comment on attachment 656872 [details] [diff] [review]
kill lion_branches, l10n_slaves_key (with fire!)

Will go to production soon.

Comment 10

[[e-mail] (:bhearsum)]

Made it to production!