851059 - snippet comparison timed out

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect)

Description

[[e-mail] (:bhearsum)]

========= Started snippet compare failed (results: 2, elapsed: 10 mins, 0 secs) (at 2013-03-14 00:59:59.201411) =========
bash /builds/slave/rel-m-beta-updates-00000000000/tools/release/compare-channel-snippets.sh aus2 beta aus2.test releasetest
 in dir /builds/slave/rel-m-beta-updates-00000000000/. (timeout 600 secs)
 watching logfiles {}
 argv: ['bash', '/builds/slave/rel-m-beta-updates-00000000000/tools/release/compare-channel-snippets.sh', 'aus2', 'beta', 'aus2.test', 'releasetest']
 environment:
  BUILDBOTCUSTOM=http://hg.mozilla.org/build/buildbotcustom
  BUILDBOT_CONFIGS=http://hg.mozilla.org/build/buildbot-configs
  CCACHE_HASHDIR=
  CLOBBERER_URL=http://clobberer.pvt.build.mozilla.org/always_clobber.php
  G_BROKEN_FILENAMES=1
  HISTCONTROL=ignoredups
  HISTSIZE=1000
  HOME=/home/cltbld
  HOSTNAME=bld-linux64-ec2-326.build.aws-us-east-1.mozilla.com
  LANG=en_US.UTF-8
  LESSOPEN=|/usr/bin/lesspipe.sh %s
  LOGNAME=cltbld
  MAIL=/var/spool/mail/cltbld
  PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin
  PWD=/builds/slave/rel-m-beta-updates-00000000000
  SHELL=/bin/bash
  SHLVL=1
  TERM=linux
  USER=cltbld
  _=/tools/buildbot/bin/python
 using PTY: False

command timed out: 600 seconds without output, attempting to kill
process killed by signal 9
program finished with exit code -1
elapsedTime=600.107116

We should make this script print progress to avoid the timeout...

Comment 1

[[e-mail] (:bhearsum)]

Attachment: make snippet comparison noisier

Tested this on the machine that did 20.0b5 updates. It took a couple of min to run the initial find command and then started printing things like:
Processing: ./Firefox/13.0/Darwin_x86_64-gcc3-u-i386-x86_64/20120508180640/af/beta/complete.txt
Processing: ./Firefox/13.0/Darwin_x86_64-gcc3-u-i386-x86_64/20120508180640/nb-NO/beta/complete.txt
Processing: ./Firefox/13.0/Darwin_x86_64-gcc3-u-i386-x86_64/20120508180640/br/beta/complete.txt
Processing: ./Firefox/13.0/Darwin_x86_64-gcc3-u-i386-x86_64/20120508180640/rm/beta/complete.txt
Processing: ./Firefox/13.0/Darwin_x86_64-gcc3-u-i386-x86_64/20120508180640/km/beta/complete.txt
Processing: ./Firefox/13.0/Darwin_x86_64-gcc3-u-i386-x86_64/20120508180640/lt/beta/complete.txt
Processing: ./Firefox/13.0/Darwin_x86_64-gcc3-u-i386-x86_64/20120508180640/lij/beta/complete.txt