595027 - Failing hg clone step was green

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Nick Thomas [:nthomas] (UTC+12)]

A couple of linux64 slaves timed out cloning the try repo 
  command timed out: 3600 seconds without output, killing pid 2734
  elapsedTime=3600.068265
  program finished with exit code -1
but the step was green so the job continued (failing at the very start of compile).

Jobs refs: 
Linux x86-64 tryserver build #2793
Linux x86-64 tryserver leak test #2763

Suspected regression from bug 505512.

Comment 1

[[e-mail] (:bhearsum)]

I'll have a look.

Comment 2

[[e-mail] (:bhearsum)]

So, the upstream code doesn't check the return code of the command if log_eval_func is set -- this changed mid-development, which is probably why regex_log_evaluator isn't checking the return code. The following upstream changeset will fix us up:
http://github.com/buildbot/buildbot/commit/e8e5bcb7434aa90e2a76423d7ebe3394bd347d00

Requires a Buildbot restart on tm*, pm01 and pm03.

Comment 3

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #2)
> Requires a Buildbot restart on tm*, pm01 and pm03.

Do we need to restart try master as well?

Comment 4

[[e-mail] (:bhearsum)]

Yes, good catch!

Comment 5

[[e-mail] (:bhearsum)]

Tested this fix in staging today, worked fine:
http://staging-master.build.mozilla.org:8011/builders/WINNT%205.2%20mozilla-central%20nightly/builds/22

Failed to clone build tools, halted there, the step was white.

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

It'd be great if we could have buildbot return 'version-rev' for 'buildbot --version', where rev is the revision of the code in our buildbot repo. I think that should be as easy as modifying master/buildbot/__init__.py after landing any changes.

Comment 9

[[e-mail] (:bhearsum)]

This landed.