e.g. https://tbpl.mozilla.org/php/getParsedLog.php?id=20792981&tree=Mozilla-Inbound b2g_mozilla-inbound_panda_dep on 2013-03-18 12:59:14 PDT for push d764382ed4cf slave: bld-linux64-ix-008 3:44:37 INFO - Target system fs tarball: out/target/product/panda/system.tar.bz2 13:44:37 INFO - build/tools/mktarball.sh out/host/linux-x86/bin/fs_get_stats out/target/product/panda system out/target/product/panda/system.tar out/target/product/panda/system.tar.bz2 command timed out: 3600 seconds without output, attempting to kill process killed by signal 9 program finished with exit code -1
I took a look at this. A failing log looks like this: 07:57:20 INFO - Created filesystem with 707/32768 inodes and 33712/131072 blocks 07:57:20 INFO - Install system fs image: out/target/product/unagi/system.img 07:57:21 INFO - out/target/product/unagi/system.img+ total size is 131546892 command timed out: 3600 seconds without output, attempting to kill If it doesn't stall it's: 10:09:59 INFO - Created filesystem with 707/32768 inodes and 33761/131072 blocks 10:09:59 INFO - Install system fs image: out/target/product/unagi/system.img 10:09:59 INFO - out/target/product/unagi/system.img+ total size is 131747596 10:09:59 INFO - real 47m27.244s 10:09:59 INFO - user 147m20.280s 10:09:59 INFO - sys 21m23.225s 10:09:59 INFO - Run |./flash.sh| to flash all partitions of your device 10:10:00 INFO - Return code: 0 The time output comes from https://github.com/mozilla-b2g/B2G/blob/master/build.sh#L61: time nice -n19 make $MAKE_FLAGS $@ ie make isn't finishing Catching a machine (bld-centos5-hp-007) in the act yields this process list: 2213 ? Sl 0:01 /tools/buildbot-0.8.4-pre-moz2/bin/python2.7 /tools/buildbot/bin/twistd --no_save --logfile /builds/slave/twistd.log --python /builds/slave/buildbot.tac 2336 ? S 0:02 \_ python scripts/scripts/b2g_build.py --target unagi --config b2g/releng.py --gaia-languages-file locales/languages_dev.json --gecko-languages-file gecko/b2g/locales/all-locales 2840 ? S 0:00 \_ /usr/bin/python -tt /usr/sbin/mock_mozilla -r mozilla-centos6-i386 -q --cwd /builds/slave/b2g_m-in_unagi_dep-00000000000/build --unpriv --shell /usr/bin/env VARIANT=user GAIA_OPTIMIZE=1 "LESSOPEN=|/usr/bin/lesspipe 2867 ? S 0:00 \_ /bin/bash ./build.sh 3253 ? SN 0:05 \_ make -j6 strace says 3253 is trying to read from fd 11, which is a closed pipe.
catlee was doing emulator builds today and saw a similar issue, and traced it an 'adb fork-server server' process. Killing that unwedged make. I haven't confirmed that in the panda or unagi builds yet.
https://github.com/mozilla-b2g/gaia/commit/b9f33e564b7ac25f43e78727af01ea35e3b21d0c may have caused this issue. https://github.com/mozilla-b2g/gaia/commit/676956f3759baafce88dc9cc801c5ea3ac605d75 disables part of that commit and hopefully fixes this issue.
Thanks for tracking that down mwu. FTR, inbound rev 805254513390 is the first build to use that.
22 builds since the landing and no failures.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
Component: General Automation → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.