1190766 - Tracking bug for Sep-21-2015 migration work

Status: RESOLVED FIXED

(Release Engineering :: Release Requests, defect)

Description

[Rail Aliiev [:rail]]

+++ This bug was initially created as a clone of Bug #1178507 +++

Comment 1

[Jordan Lund (:jlund)]

Attachment: 150813_1190766_migration_work_bump_m-r_buildbot-configs.patch

bumpity bump

Comment 2

[Jordan Lund (:jlund)]

Attachment: 150813_1190766_migration_work_bump_devel_branches_buildbot-configs.patch

preemptive r? for next week (21st)

Comment 3

[Jordan Lund (:jlund)]

hrm, I'm hitting ssl handshake errors during trial runs while pulling from hg.m.o. I hope this is a temporary server issue or I have done something to my local machine that is making migration script sad.

18:32:14     INFO - retry: Calling _get_revision with args: (<mozharness.base.vcs.mercurial.MercurialVCS object at 0x10ed80050>, '/Users/jlund/devel/mozilla/merge_duty/build/mozi
lla-release'), kwargs: {}, attempt #3
18:32:14     INFO - Setting /Users/jlund/devel/mozilla/merge_duty/build/mozilla-release to ssh://hg.mozilla.org/releases/mozilla-release revision default.
18:32:14     INFO - Cloning ssh://hg.mozilla.org/releases/mozilla-release to /Users/jlund/devel/mozilla/merge_duty/build/mozilla-release.
18:32:14     INFO - Running command: ['hg', '--config', 'ui.merge=internal:merge', 'clone', 'ssh://hg.mozilla.org/releases/mozilla-release', '/Users/jlund/devel/mozilla/merge_dut
y/build/mozilla-release']
18:32:14     INFO - Copy/paste: hg --config ui.merge=internal:merge clone ssh://hg.mozilla.org/releases/mozilla-release /Users/jlund/devel/mozilla/merge_duty/build/mozilla-releas
e
18:32:16     INFO -  downloading bundle https://hg.cdn.mozilla.net/releases/mozilla-release/9c898cde2175e9e98b916d996ed286a9dff0c853.gzip.hg
18:32:16    ERROR -  abort: error fetching bundle: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)
18:32:16    ERROR -  Automation Error: hg not responding
18:32:16     INFO -  (consider contacting the server operator if this error persists)
18:32:16    ERROR - Return code: 255
18:32:16     INFO - rmtree: /Users/jlund/devel/mozilla/merge_duty/build/mozilla-release
18:32:16     INFO - retry: attempt #3 caught exception: Unable to clone ssh://hg.mozilla.org/releases/mozilla-release to /Users/jlund/devel/mozilla/merge_duty/build/mozilla-relea
se!
18:32:16     INFO - retry: Failed, sleeping 240 seconds before retrying
^CTraceback (most recent call last):
  File "mozharness/scripts/merge_day/gecko_migration.py", line 653, in <module>
    gecko_migration.run_and_exit()
  File "/Users/jlund/devel/mozilla/merge_duty/mozharness/mozharness/base/script.py", line 1717, in run_and_exit
    rc = self.run()
  File "/Users/jlund/devel/mozilla/merge_duty/mozharness/mozharness/base/script.py", line 1693, in run
    self.run_action(action)
  File "/Users/jlund/devel/mozilla/merge_duty/mozharness/mozharness/base/script.py", line 1635, in run_action
    self._possibly_run_method(method_name, error_if_missing=True)
  File "/Users/jlund/devel/mozilla/merge_duty/mozharness/mozharness/base/script.py", line 1576, in _possibly_run_method
    return getattr(self, method_name)()
  File "mozharness/scripts/merge_day/gecko_migration.py", line 531, in pull
    super(GeckoMigration, self).pull(repos=repos)
  File "/Users/jlund/devel/mozilla/merge_duty/mozharness/mozharness/base/vcs/vcsbase.py", line 137, in pull
    parent_dir=parent_dir)
  File "/Users/jlund/devel/mozilla/merge_duty/mozharness/mozharness/base/vcs/vcsbase.py", line 106, in vcs_checkout_repos
    revision_dict[dest]['revision'] = self.vcs_checkout(**kwargs)
  File "/Users/jlund/devel/mozilla/merge_duty/mozharness/mozharness/base/vcs/vcsbase.py", line 84, in vcs_checkout
    args=(vcs_obj, kwargs['dest']),
  File "/Users/jlund/devel/mozilla/merge_duty/mozharness/mozharness/base/script.py", line 852, in retry
    time.sleep(sleeptime)
KeyboardInterrupt
python mozharness/scripts/merge_day/gecko_migration.py -c   11.94s user 9.60s system 8% cpu 4:26.65 total



ni: gps to make it aware in case this is unexpected

Comment 4

[Gregory Szorc [:gps]]

This error is likely bug 1201993. You need Python >= 2.7.9 and Mercurial >= 3.3 for SNI to work. bundleclone was previously screening for Python < 2.7.9 and excluding URLs requiring SNI. It will soon screen for Mercurial < 3.3.

So, the solution is to synchronize the bundleclone extension at revision e93d6e07ba33 or newer with all of automation. This was previously done in bug 1196067. I'll leave this bug open to track that.

FWIW, automation should be configured to prefer S3 URLs when running in EC2. Since the stack references /Users/jlund, I'm guessing you are running this on a local machine which doesn't have the hgrc preferences defined, so using the CDN is expected. This bug will likely not occur when run in automation. We should still upgrade bundleclone everywhere though.

Comment 5

[Rail Aliiev [:rail]]

Attachment: version_files.diff

This should be landed prior to Mon merge to make mozilla-central bumps work.

* moved version_files to configs to allow train rides
* No need to bump mobile/android/confvars.sh for gecko 43 (see http://hg.mozilla.org/mozilla-central/rev/72fde1ca508a)
* version_about.txt is version_display.txt actually
* weave_version has been moved to moz.build

Comment 6

[Jordan Lund (:jlund)]

Comment on attachment 8663107 [details] [diff] [review]
version_files.diff

Review of attachment 8663107 [details] [diff] [review]:
-----------------------------------------------------------------

looks good. I am roughly following along. I'll land this locally and do a test run before tomorrow

::: testing/mozharness/configs/merge_day/aurora_to_beta.py
@@ +3,5 @@
> +    "version_files": [
> +        "browser/config/version.txt",
> +        "browser/config/version_display.txt",
> +        "config/milestone.txt",
> +        "mobile/android/confvars.sh",  # TODO: remove this line for gecko 43

you have 'remove this line for gecko 43' for both this file and beta_to_release.py. 

Though I'm guessing this should be removed one uplift before the beta_to_release.py line equivalent?

Comment 7

[Jordan Lund (:jlund)]

I think this is good:

aurora 2 beta output: http://people.mozilla.org/~jlund/beta_diff.log

central 2 aurora diff output: http://people.mozilla.org/~jlund/aurora_diff.log

rail: sanity check ^

Comment 8

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e61f327085da

Comment 9

[Rail Aliiev [:rail]]

Comment on attachment 8663107 [details] [diff] [review]
version_files.diff

https://hg.mozilla.org/integration/mozilla-inbound/rev/e61f327085da

(In reply to Jordan Lund (:jlund) from comment #6)
> Comment on attachment 8663107 [details] [diff] [review]
> version_files.diff
> 
> Review of attachment 8663107 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> looks good. I am roughly following along. I'll land this locally and do a
> test run before tomorrow
> 
> ::: testing/mozharness/configs/merge_day/aurora_to_beta.py
> @@ +3,5 @@
> > +    "version_files": [
> > +        "browser/config/version.txt",
> > +        "browser/config/version_display.txt",
> > +        "config/milestone.txt",
> > +        "mobile/android/confvars.sh",  # TODO: remove this line for gecko 43
> 
> you have 'remove this line for gecko 43' for both this file and
> beta_to_release.py. 
> 
> Though I'm guessing this should be removed one uplift before the
> beta_to_release.py line equivalent?

I rephrase is as
"remove this line before gecko 43 merge"

Comment 10

[Rail Aliiev [:rail]]

(In reply to Jordan Lund (:jlund) from comment #7)
> I think this is good:
> 
> aurora 2 beta output: http://people.mozilla.org/~jlund/beta_diff.log
> 
> central 2 aurora diff output:
> http://people.mozilla.org/~jlund/aurora_diff.log
> 
> rail: sanity check ^

LGTM

Comment 11

[Rail Aliiev [:rail]]

Comment on attachment 8660520 [details] [diff] [review]
150813_1190766_migration_work_bump_devel_branches_buildbot-configs.patch

https://hg.mozilla.org/build/buildbot-configs/rev/0b0f81a66c8d

Comment 12

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/e61f327085da