976490 - build-tools: Can't push /opt/vcs2vcs/vcs_sync/build/conversion/build-tools to build-tools-github!

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Pete Moore [:pmoore][:pete]]

On 25 Feb 2014, at 10:21, [e-mail] wrote:

Unable to push these repos:
build-tools: Can't push /opt/vcs2vcs/vcs_sync/build/conversion/build-tools to build-tools-github!



Summary is non-zero:

error - Unable to push build-tools.

09:21:05     INFO - Running command: ['/opt/vcs2vcs/vcs_sync/build/venv/bin/hg', '--config', 'web.cacerts=/etc/pki/tls/certs/ca-bundle.crt', '-v', 'gexport'] in /opt/vcs2vcs/vcs_sync/build/conversion/build-tools
09:21:05     INFO - Copy/paste: /opt/vcs2vcs/vcs_sync/build/venv/bin/hg --config web.cacerts=/etc/pki/tls/certs/ca-bundle.crt -v gexport
09:21:05     INFO - Calling ['/opt/vcs2vcs/vcs_sync/build/venv/bin/hg', '--config', 'web.cacerts=/etc/pki/tls/certs/ca-bundle.crt', '-v', 'gexport'] with output_timeout 900
09:21:05     INFO -  exporting hg objects to git
09:21:05     INFO -  converting revision 85fece495d379673e47ae8b481031bf53689ee31
09:21:05    ERROR -  abort: data/lib/python/mozilla_buildtools/test/sample-patcher-config.cfg.i@59fcd33f2135: no match found!
09:21:05    ERROR -  Automation Error: hg not responding
09:21:06    ERROR - Return code: 255
09:21:06     INFO - rmtree: /opt/vcs2vcs/vcs_sync/build/upload/gecko-mapfile
09:21:06     INFO - retry: Calling <built-in function remove> with args: ('/opt/vcs2vcs/vcs_sync/build/upload/gecko-mapfile',), kwargs: {}, attempt #1
09:21:06     INFO - Copying /opt/vcs2vcs/vcs_sync/build/conversion/build-tools/.hg/git-mapfile to /opt/vcs2vcs/vcs_sync/build/upload/gecko-mapfile
09:21:06     INFO - Getting output from command: ['/opt/vcs2vcs/vcs_sync/build/venv/bin/python', '-c', 'from distutils.sysconfig import get_python_lib; print(get_python_lib())']
09:21:06     INFO - Copy/paste: /opt/vcs2vcs/vcs_sync/build/venv/bin/python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"
--
09:21:06     INFO - Running command: ['git', 'push', '-f', '[e-mail]:mozilla/build-tools.git', '+refs/heads/master:refs/heads/master', '+refs/tags/FIREFOX_28_0b6_RELEASE:refs/tags/FIREFOX_28_0b6_RELEASE', '+refs/tags/FIREFOX_28_0b6_BUILD1:refs/tags/FIREFOX_28_0b6_BUILD1', '+refs/tags/FENNEC_28_0b6_RELEASE:refs/tags/FENNEC_28_0b6_RELEASE', '+refs/tags/FENNEC_28_0b6_BUILD1:refs/tags/FENNEC_28_0b6_BUILD1', '+refs/tags/FIREFOX_28_0b4_RELEASE_RUNTIME:refs/tags/FIREFOX_28_0b4_RELEASE_RUNTIME', '+refs/tags/FIREFOX_28_0b4_BUILD1_RUNTIME:refs/tags/FIREFOX_28_0b4_BUILD1_RUNTIME', '+refs/tags/FIREFOX_28_0b4_RELEASE:refs/tags/FIREFOX_28_0b4_RELEASE', '+refs/tags/FIREFOX_28_0b4_BUILD1:refs/tags/FIREFOX_28_0b4_BUILD1', '+refs/tags/FENNEC_28_0b4_RELEASE:refs/tags/FENNEC_28_0b4_RELEASE'] in /opt/vcs2vcs/vcs_sync/build/conversion/build-tools/.git
09:21:06     INFO - Copy/paste: git push -f [e-mail]:mozilla/build-tools.git +refs/heads/master:refs/heads/master +refs/tags/FIREFOX_28_0b6_RELEASE:refs/tags/FIREFOX_28_0b6_RELEASE +refs/tags/FIREFOX_28_0b6_BUILD1:refs/tags/FIREFOX_28_0b6_BUILD1 +refs/tags/FENNEC_28_0b6_RELEASE:refs/tags/FENNEC_28_0b6_RELEASE +refs/tags/FENNEC_28_0b6_BUILD1:refs/tags/FENNEC_28_0b6_BUILD1 +refs/tags/FIREFOX_28_0b4_RELEASE_RUNTIME:refs/tags/FIREFOX_28_0b4_RELEASE_RUNTIME +refs/tags/FIREFOX_28_0b4_BUILD1_RUNTIME:refs/tags/FIREFOX_28_0b4_BUILD1_RUNTIME +refs/tags/FIREFOX_28_0b4_RELEASE:refs/tags/FIREFOX_28_0b4_RELEASE +refs/tags/FIREFOX_28_0b4_BUILD1:refs/tags/FIREFOX_28_0b4_BUILD1 +refs/tags/FENNEC_28_0b4_RELEASE:refs/tags/FENNEC_28_0b4_RELEASE
09:21:06     INFO - Using partial env: {'GIT_SSH': '/opt/vcs2vcs/vcs_sync/mozharness/external_tools/git-ssh-wrapper.sh',
09:21:06     INFO -  'GIT_SSH_KEY': '~/.ssh/releng-github-id_rsa'}
09:21:06     INFO - Calling ['git', 'push', '-f', '[e-mail]:mozilla/build-tools.git', '+refs/heads/master:refs/heads/master', '+refs/tags/FIREFOX_28_0b6_RELEASE:refs/tags/FIREFOX_28_0b6_RELEASE', '+refs/tags/FIREFOX_28_0b6_BUILD1:refs/tags/FIREFOX_28_0b6_BUILD1', '+refs/tags/FENNEC_28_0b6_RELEASE:refs/tags/FENNEC_28_0b6_RELEASE', '+refs/tags/FENNEC_28_0b6_BUILD1:refs/tags/FENNEC_28_0b6_BUILD1', '+refs/tags/FIREFOX_28_0b4_RELEASE_RUNTIME:refs/tags/FIREFOX_28_0b4_RELEASE_RUNTIME', '+refs/tags/FIREFOX_28_0b4_BUILD1_RUNTIME:refs/tags/FIREFOX_28_0b4_BUILD1_RUNTIME', '+refs/tags/FIREFOX_28_0b4_RELEASE:refs/tags/FIREFOX_28_0b4_RELEASE', '+refs/tags/FIREFOX_28_0b4_BUILD1:refs/tags/FIREFOX_28_0b4_BUILD1', '+refs/tags/FENNEC_28_0b4_RELEASE:refs/tags/FENNEC_28_0b4_RELEASE'] with output_timeout 1800
09:21:08    ERROR -  error: src refspec refs/tags/FIREFOX_28_0b6_RELEASE does not match any.
09:21:08    ERROR -  error: src refspec refs/tags/FIREFOX_28_0b6_BUILD1 does not match any.
09:21:08    ERROR -  error: src refspec refs/tags/FENNEC_28_0b6_RELEASE does not match any.
09:21:08    ERROR -  error: src refspec refs/tags/FENNEC_28_0b6_BUILD1 does not match any.
09:21:08    ERROR -  error: failed to push some refs to '[e-mail]:mozilla/build-tools.git'
09:21:08    ERROR - Return code: 1
09:21:08    ERROR - build-tools: Can't push /opt/vcs2vcs/vcs_sync/build/conversion/build-tools to build-tools-github!
09:21:08    ERROR - Unable to push build-tools.
09:21:08     INFO - Writing to file /opt/vcs2vcs/vcs_sync/build/upload/repo_update.json
09:21:08     INFO - Contents:
09:21:08     INFO -  {
09:21:08     INFO -      "last_pull_datetime": "2014-02-25 09:21 UTC", 
09:21:08     INFO -      "last_pull_timestamp": 1393320065, 
--
09:21:08     INFO -              "push_datetime": "2014-02-24 20:30 UTC", 
09:21:08     INFO -              "push_timestamp": 1393273806
09:21:08     INFO -          }
09:21:08     INFO -      }
09:21:08     INFO -  }
09:21:08    FATAL - Unable to push these repos:
09:21:08    FATAL - build-tools: Can't push /opt/vcs2vcs/vcs_sync/build/conversion/build-tools to build-tools-github!
09:21:08    FATAL - 
09:21:08    FATAL - Running post_fatal callback...
09:21:08     INFO - Job took 6 seconds.
09:21:08     INFO - Getting output from command: ['egrep', '-C5', '^[0-9:]+ +(ERROR|CRITICAL|FATAL) -', '/opt/vcs2vcs/vcs_sync/logs/build-repos_info.log']
09:21:08     INFO - Copy/paste: egrep -C5 "^[0-9:]+ +(ERROR|CRITICAL|FATAL) -" /opt/vcs2vcs/vcs_sync/logs/build-repos_info.log

Comment 1

[Pete Moore [:pmoore][:pete]]

Problem is with hg2git conversion process, not the tags per se.

The hg changeset failing to convert is:
# HG changeset patch
# User Nick Thomas <[e-mail]>
# Date 1393273273 -46800
# Node ID 85fece495d379673e47ae8b481031bf53689ee31
# Parent  268665fc0baca08335efc777ab2f98ea78a14a3f
Bug 974057, Support modifying update-settings.ini when doing update verify, r=bhearsum"

There looks to be nothing wrong with this changeset from an "hg export" - suspect local repo corruption on vcs2vcs server.

I'll move the working dir to a safe location, so a new one gets created. My guess is with a fresh repo, the problem will be solved. Will also watch out to make sure we don't get new hashes.

Afterwards, if this was the cause, I will try to investigate how the repo got corrupted (fallout from previous http 500s?)...

Pete

Comment 2

[Pete Moore [:pmoore][:pete]]

Immediate issue resolved.

Still need to investigate *why* hg working directory got corrupted in the first place...

Comment 3

[Gregory Szorc [:gps]]

You should set [server] validate=True in your hgrc to avoid corruption.

See also bug 963778.

Comment 4

[Pete Moore [:pmoore][:pete]]

Wonderful, thanks Gregory! I'll give it a bash. =)