827324 - updates builder was unable to retry and succeed

Status: RESOLVED WONTFIX

(Release Engineering :: Release Automation, defect, P3)

Description

[[e-mail] (:bhearsum)]

It seemed to have lost a push race, but just kept trying to push without resolving anything:
Executing: ['hg', 'push', '-e', 'ssh -l ffxbld -i /home/mock_mozilla/.ssh/ffxbld_dsa', '-f', 'ssh://hg.mozilla.org/build/tools']
pushing to ssh://hg.mozilla.org/build/tools
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 2 changesets with 6 changes to 6 files (+1 heads)
remote: You may not push multiple heads to this repository.
remote: transaction abort!
remote: rollback completed
remote: abort: pretxnchangegroup.a_singlehead hook exited with status 1
retry: Failed, sleeping 8 seconds before retrying
retry: Calling <function run_with_timeout at 0xf76348b4> with args: (['hg', 'push', '-e', 'ssh -l ffxbld -i /home/mock_mozilla/.ssh/ffxbld_dsa', '-f', 'ssh://hg.mozilla.org/build/tools'], 1260, None, None, False, True), kwargs: {}, attempt #5
Executing: ['hg', 'push', '-e', 'ssh -l ffxbld -i /home/mock_mozilla/.ssh/ffxbld_dsa', '-f', 'ssh://hg.mozilla.org/build/tools']
pushing to ssh://hg.mozilla.org/build/tools
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 2 changesets with 6 changes to 6 files (+1 heads)
remote: You may not push multiple heads to this repository.
remote: transaction abort!
remote: rollback completed
remote: abort: pretxnchangegroup.a_singlehead hook exited with status 1
retry: Giving up on <function run_with_timeout at 0xf76348b4>

I guess this is because it's using a really simply retry.py, not something that's wrapped in apply_and_push -- it has no logic to be able to resolve a push race, I think.

Comment 1

[[e-mail] (:bhearsum)]

hard to fix because updates isn't in a script. also not a common problem and easy to recover from. P3.

Comment 2

[Aki Sasaki (not active)]

To fix in bug 748798 since balrog won't completely kill this builder?

Comment 3

[[e-mail] (:bhearsum)]

I bet the updates builder will die before we ever hit this again.