990573 - run xulrunner post-release task automatically after Firefox post-release

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect)

Description

[Rail Aliiev [:rail]]

brain_usage--
checklist_items--
memory_corruption--
sad_people--
code++

Comment 1

[Rail Aliiev [:rail]]

Attachment: xr-buildbotcustom.diff

The patch looks straight forward, but I'll test it in staging before landing.

Comment 2

[[e-mail] (:bhearsum)]

Comment on attachment 8399989 [details] [diff] [review]
xr-buildbotcustom.diff

Review of attachment 8399989 [details] [diff] [review]:
-----------------------------------------------------------------

Looks reasonable

Comment 3

[Rail Aliiev [:rail]]

Attachment: xr-buildbotcustom-1.diff

The patch wasn't enough and the builder failed:

exceptions.RuntimeError: copy_property 'buildid' is not set in the triggering build

ScriptFactory calls Trigger with hard coded copy_properties=['buildid', 'builduid']. I changed it to be backward compatible (even if we don't use it anywhere) and be able to override them.

Staging says "green"!

Comment 4

[[e-mail] (:bhearsum)]

Comment on attachment 8400009 [details] [diff] [review]
xr-buildbotcustom-1.diff

Review of attachment 8400009 [details] [diff] [review]:
-----------------------------------------------------------------

::: process/factory.py
@@ +6462,5 @@
>          self.use_credentials_file = use_credentials_file
>          if platform and 'win' in platform:
>              self.get_basedir_cmd = ['cd']
> +        if copy_properties is None:
> +            self.copy_properties = ['buildid', 'builduid']

Given the wide ranging use of ScriptFactory these days, this seems like a bad default...is there any chance I can convince you to drop it as a default and update all the of instantiations of it?

Comment 5

[Rail Aliiev [:rail]]

Attachment: xr-buildbotcustom-2.diff

SGTM. An extra pair of eyes should also help.

Comment 6

[[e-mail] (:bhearsum)]

Comment on attachment 8400734 [details] [diff] [review]
xr-buildbotcustom-2.diff

Review of attachment 8400734 [details] [diff] [review]:
-----------------------------------------------------------------

I ran dump_master.py on the before/after. No changes at all to the test masters, and only this to the build side:
--- builds-before.txt	2014-04-02 12:16:12.155692332 -0400
+++ builds-after.txt	2014-04-02 12:19:21.551697753 -0400
@@ -53136,6 +53136,7 @@
     OutputStep {'data': <WithProperties "TinderboxPrint: tools_revlink: https://hg.mozilla.org/build/tools/rev/%(script_repo_revision)s">, 'log': 'output', 'name': 'tinderboxprint_script_revlink'} {'BUILDBOT_CONFIGS': 'https://hg.mozilla.org/build/buildbot-configs', 'BUILDBOTCUSTOM': 'https://hg.mozilla.org/build/buildbotcustom', 'CLOBBERER_URL': 'http://clobberer.pvt.build.mozilla.org/always_clobber.php'}
     MockCommand {'command': ['scripts/scripts/release/stage-tasks.sh',  'mozilla/production_config.py',  'postrelease',  '--extra-excludes=*.zip',  '--extra-excludes=*.zip.asc',  '--ssh-user',  'ffxbld',  '--ssh-key',  'ffxbld_dsa'], 'description': None, 'descriptionDone': None, 'env': {'PROPERTIES_FILE': <WithProperties "%(basedir)s/buildprops.json">}, 'haltOnFailure': True, 'log_eval_func': None, 'logfiles': {}, 'maxTime': None, 'mock': False, 'mock_args': ['--unpriv'], 'mock_login': 'mock_mozilla', 'mock_workdir_mutator': <function <lambda>>, 'mock_workdir_prefix': '%(basedir)s/', 'name': 'run_script', 'target': None, 'timeout': 1200, 'usePTY': 'slave-config', 'warnOnWarnings': True, 'workdir': '.'} {'BUILDBOT_CONFIGS': 'https://hg.mozilla.org/build/buildbot-configs', 'BUILDBOTCUSTOM': 'https://hg.mozilla.org/build/buildbotcustom', 'CLOBBERER_URL': 'http://clobberer.pvt.build.mozilla.org/always_clobber.php'}
     C {'alwaysRun': True, 'command': ['bash', '-c', 'for file in `ls -1`; do cat $file; done'], 'description': None, 'descriptionDone': None, 'extract_fn': <function extractProperties>, 'flunkOnFailure': False, 'log_eval_func': None, 'logfiles': {}, 'name': 'set_script_properties', 'property': None, 'strip': True, 'usePTY': 'slave-config', 'warnOnFailure': False, 'workdir': 'properties'} {'BUILDBOT_CONFIGS': 'https://hg.mozilla.org/build/buildbot-configs', 'BUILDBOTCUSTOM': 'https://hg.mozilla.org/build/buildbotcustom', 'CLOBBERER_URL': 'http://clobberer.pvt.build.mozilla.org/always_clobber.php'}
+    C {'copy_properties': [], 'log_eval_func': None, 'logfiles': {}, 'schedulerNames': ['release-mozilla-beta-xr_postrelease'], 'set_properties': {}, 'updateSourceStamp': True, 'waitForFinish': False} {'BUILDBOT_CONFIGS': 'https://hg.mozilla.org/build/buildbot-configs', 'BUILDBOTCUSTOM': 'https://hg.mozilla.org/build/buildbotcustom', 'CLOBBERER_URL': 'http://clobberer.pvt.build.mozilla.org/always_clobber.php'}
     C {'alwaysRun': True, 'command': ['rm', '-f', 'oauth.txt'], 'description': None, 'descriptionDone': None, 'log_eval_func': None, 'logfiles': {}, 'name': 'rm_oauth_txt', 'usePTY': 'slave-config', 'workdir': '.'} {'BUILDBOT_CONFIGS': 'https://hg.mozilla.org/build/buildbot-configs', 'BUILDBOTCUSTOM': 'https://hg.mozilla.org/build/buildbotcustom', 'CLOBBERER_URL': 'http://clobberer.pvt.build.mozilla.org/always_clobber.php'}
 release-mozilla-beta-push_to_mirrors ScriptFactory
     SetBuildProperty {'property_name': 'master', 'value': <function <lambda>>} {'BUILDBOT_CONFIGS': 'https://hg.mozilla.org/build/buildbot-configs', 'BUILDBOTCUSTOM': 'https://hg.mozilla.org/build/buildbotcustom', 'CLOBBERER_URL': 'http://clobberer.pvt.build.mozilla.org/always_clobber.php'}
@@ -57511,6 +57512,7 @@
     OutputStep {'data': <WithProperties "TinderboxPrint: tools_revlink: https://hg.mozilla.org/build/tools/rev/%(script_repo_revision)s">, 'log': 'output', 'name': 'tinderboxprint_script_revlink'} {'BUILDBOT_CONFIGS': 'https://hg.mozilla.org/build/buildbot-configs', 'BUILDBOTCUSTOM': 'https://hg.mozilla.org/build/buildbotcustom', 'CLOBBERER_URL': 'http://clobberer.pvt.build.mozilla.org/always_clobber.php'}
     MockCommand {'command': ['scripts/scripts/release/stage-tasks.sh',  'mozilla/production_config.py',  'postrelease',  '--extra-excludes=*.zip',  '--extra-excludes=*.zip.asc',  '--ssh-user',  'ffxbld',  '--ssh-key',  'ffxbld_dsa'], 'description': None, 'descriptionDone': None, 'env': {'PROPERTIES_FILE': <WithProperties "%(basedir)s/buildprops.json">}, 'haltOnFailure': True, 'log_eval_func': None, 'logfiles': {}, 'maxTime': None, 'mock': False, 'mock_args': ['--unpriv'], 'mock_login': 'mock_mozilla', 'mock_workdir_mutator': <function <lambda>>, 'mock_workdir_prefix': '%(basedir)s/', 'name': 'run_script', 'target': None, 'timeout': 1200, 'usePTY': 'slave-config', 'warnOnWarnings': True, 'workdir': '.'} {'BUILDBOT_CONFIGS': 'https://hg.mozilla.org/build/buildbot-configs', 'BUILDBOTCUSTOM': 'https://hg.mozilla.org/build/buildbotcustom', 'CLOBBERER_URL': 'http://clobberer.pvt.build.mozilla.org/always_clobber.php'}
     C {'alwaysRun': True, 'command': ['bash', '-c', 'for file in `ls -1`; do cat $file; done'], 'description': None, 'descriptionDone': None, 'extract_fn': <function extractProperties>, 'flunkOnFailure': False, 'log_eval_func': None, 'logfiles': {}, 'name': 'set_script_properties', 'property': None, 'strip': True, 'usePTY': 'slave-config', 'warnOnFailure': False, 'workdir': 'properties'} {'BUILDBOT_CONFIGS': 'https://hg.mozilla.org/build/buildbot-configs', 'BUILDBOTCUSTOM': 'https://hg.mozilla.org/build/buildbotcustom', 'CLOBBERER_URL': 'http://clobberer.pvt.build.mozilla.org/always_clobber.php'}
+    C {'copy_properties': [], 'log_eval_func': None, 'logfiles': {}, 'schedulerNames': ['release-mozilla-release-xr_postrelease'], 'set_properties': {}, 'updateSourceStamp': True, 'waitForFinish': False} {'BUILDBOT_CONFIGS': 'https://hg.mozilla.org/build/buildbot-configs', 'BUILDBOTCUSTOM': 'https://hg.mozilla.org/build/buildbotcustom', 'CLOBBERER_URL': 'http://clobberer.pvt.build.mozilla.org/always_clobber.php'}
     C {'alwaysRun': True, 'command': ['rm', '-f', 'oauth.txt'], 'description': None, 'descriptionDone': None, 'log_eval_func': None, 'logfiles': {}, 'name': 'rm_oauth_txt', 'usePTY': 'slave-config', 'workdir': '.'} {'BUILDBOT_CONFIGS': 'https://hg.mozilla.org/build/buildbot-configs', 'BUILDBOTCUSTOM': 'https://hg.mozilla.org/build/buildbotcustom', 'CLOBBERER_URL': 'http://clobberer.pvt.build.mozilla.org/always_clobber.php'}
 release-mozilla-release-push_to_mirrors ScriptFactory
     SetBuildProperty {'property_name': 'master', 'value': <function <lambda>>} {'BUILDBOT_CONFIGS': 'https://hg.mozilla.org/build/buildbot-configs', 'BUILDBOTCUSTOM': 'https://hg.mozilla.org/build/buildbotcustom', 'CLOBBERER_URL': 'http://clobberer.pvt.build.mozilla.org/always_clobber.php'}
@@ -61773,6 +61775,9 @@
  'name': 'release-mozilla-release-firefox_tag',
  'properties': {'scheduler': 'release-mozilla-release-firefox_tag'},
  'upstream_name': 'release-mozilla-release-firefox_reset_schedulers'}
+<buildbot.schedulers.triggerable.Triggerable> {'builderNames': ['release-mozilla-release-xr_postrelease'],
+ 'name': 'release-mozilla-release-xr_postrelease',
+ 'properties': {'scheduler': 'release-mozilla-release-xr_postrelease'}}
 <buildbot.schedulers.triggerable.Triggerable> {'builderNames': ['release-mozilla-release-linux_repack_1/10',
                   'release-mozilla-release-linux_repack_2/10',
                   'release-mozilla-release-linux_repack_3/10',
@@ -62082,6 +62087,9 @@
  'name': 'release-mozilla-beta-firefox_tag',
  'properties': {'scheduler': 'release-mozilla-beta-firefox_tag'},
  'upstream_name': 'release-mozilla-beta-firefox_reset_schedulers'}
+<buildbot.schedulers.triggerable.Triggerable> {'builderNames': ['release-mozilla-beta-xr_postrelease'],
+ 'name': 'release-mozilla-beta-xr_postrelease',
+ 'properties': {'scheduler': 'release-mozilla-beta-xr_postrelease'}}
 <buildbot.schedulers.triggerable.Triggerable> {'builderNames': ['release-mozilla-beta-linux_repack_1/10',
                   'release-mozilla-beta-linux_repack_2/10',
                   'release-mozilla-beta-linux_repack_3/10',


copy_properties are printed in the dump master output, since they're step args, so I think that's pretty good verification.

::: process/factory.py
@@ +6459,5 @@
>          self.get_basedir_cmd = ['bash', '-c', 'pwd']
>          self.triggered_schedulers = triggered_schedulers
>          self.env = env.copy()
>          self.use_credentials_file = use_credentials_file
> +        self.copy_properties = copy_properties or []

I think you can just set the default in the signature to [], and then do self.copy_properties = copy_properties[:] here. If you want to keep this instead, that's fine too.

Comment 7

[Rail Aliiev [:rail]]

Comment on attachment 8400734 [details] [diff] [review]
xr-buildbotcustom-2.diff

https://hg.mozilla.org/build/buildbotcustom/rev/ad267d4ff918

Comment 8

[Jordan Lund (:jlund)]

https://hg.mozilla.org/build/buildbotcustom/rev/ad267d4ff918 -> in production :)