Closed Bug 895943 Opened 12 years ago Closed 12 years ago

duplicate jobs triggered in 23.0b6 b7 and b8

Categories

(Release Engineering :: General, defect)

Product:
Release Engineering
Component:
General
Platform:
x86_64
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: bhearsum, Unassigned)

Details

Details incoming.
Examples from each release: 23.0b6: - 6 linux partner repacks: -- http://buildbot-master58:8001/builders/release-mozilla-beta-linux_partner_repack/builds/3 & 4 & 5 --- all started at Mon Jul 15 19:14:50 2013 -- http://buildbot-master64:8001/builders/release-mozilla-beta-linux_partner_repack/builds/0 & 1 & 2 --- all started at Mon Jul 15 19:15:12 2013 - 4 linux64 partner repacks: -- http://buildbot-master58:8001/builders/release-mozilla-beta-linux64_partner_repack/builds/1 --- started at Mon Jul 15 18:47:56 2013 -- http://buildbot-master62:8001/builders/release-mozilla-beta-linux64_partner_repack/builds/2 --- started at Mon Jul 15 18:47:32 2013 -- http://buildbot-master64:8001/builders/release-mozilla-beta-linux64_partner_repack/builds/0 --- started at Mon Jul 15 18:47:42 2013 -- http://buildbot-master66.srv.releng.usw2.mozilla.com:8001/builders/release-mozilla-beta-linux64_partner_repack/builds/3 --- started at Mon Jul 15 18:47:20 2013 - 3 macosx64 partner repacks -- http://buildbot-master58:8001/builders/release-mozilla-beta-macosx64_partner_repack/builds/1 --- started at Mon Jul 15 18:48:31 2013 -- http://buildbot-master66.srv.releng.usw2.mozilla.com:8001/builders/release-mozilla-beta-macosx64_partner_repack/builds/1 & 2 --- started at Mon Jul 15 18:48:11 2013 & Mon Jul 15 18:48:12 2013 23.0b7: - 2 linux64 partner repacks: -- http://buildbot-master64:8001/builders/release-mozilla-beta-linux64_partner_repack/builds/1 --- started at Thu Jul 18 21:44:12 2013 -- http://buildbot-master66.srv.releng.usw2.mozilla.com:8001/builders/release-mozilla-beta-linux64_partner_repack/builds/4 --- started at Thu Jul 18 21:44:20 2013 - 2 win32 update verifies: -- http://buildbot-master62:8001/builders/release-mozilla-beta-win32_update_verify_1%2F4/builds/3 and 4 --- both started at Fri Jul 19 01:56:37 2013 There were lots of other duplicates too, looking at https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/23.0b6-candidates/build1/logs/ and https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/23.0b7-candidates/build1/logs/ is an easy way to find them. I tried to find additional details of these in the buildbot db, but unfortunately my sql-fu wasn't good enough. Going to dump some twistd.log info for the 23.0b7 ones in the next comment, though.
From bm62: twistd.log.6:2013-07-19 01:56:36-0700 [-] prioritizeBuilders: 0.27s prioritized 12 builder(s): [((0, 0L, 100, 1374224177L), 'release-mozilla-beta-win32_update_verify_1/4'), ((0, 0L, 100, 1374224177L), 'release-mozilla-beta-win32_update_verify_3/4'), ((0, 0L, 100, 1374224183L), 'release-mozilla-beta-linux_update_verify_2/4'), ((0, 0L, 100, 1374224183L), 'release-mozilla-beta-linux64_update_verify_4/4'), ((0, 0L, 100, 1374224183L), 'release-mozilla-beta-linux_update_verify_1/4'), ((0, 0L, 100, 1374224183L), 'release-mozilla-beta-linux64_update_verify_3/4'), ((0, 0L, 100, 1374224183L), 'release-mozilla-beta-linux64_update_verify_1/4'), ((0, 0L, 100, 1374224183L), 'release-mozilla-beta-linux64_update_verify_2/4'), ((0, 0L, 100, 1374224183L), 'release-mozilla-beta-win32_update_verify_2/4'), ((0, 0L, 100, 1374224183L), 'release-mozilla-beta-win32_update_verify_4/4'), ((0, 0L, 100, 1374224183L), 'release-mozilla-beta-linux_update_verify_3/4'), ((0, 0L, 100, 1374224183L), 'release-mozilla-beta-linux_update_verify_4/4')] twistd.log.6:2013-07-19 01:56:36-0700 [-] prioritizeBuilders: 0.28s important builder release-mozilla-beta-win32_update_verify_1/4 (p == (0, 0L, 100, 1374224177L)) twistd.log.6:2013-07-19 01:56:36-0700 [-] prioritizeBuilders: 0.28s found 10 important builder(s): ['release-mozilla-beta-linux_update_verify_3/4', 'release-mozilla-beta-linux64_update_verify_4/4', 'release-mozilla-beta-linux64_update_verify_3/4', 'release-mozilla-beta-linux_update_verify_1/4', 'release-mozilla-beta-linux64_update_verify_1/4', 'release-mozilla-beta-win32_update_verify_3/4', 'release-mozilla-beta-win32_update_verify_1/4', 'release-mozilla-beta-linux64_update_verify_2/4', 'release-mozilla-beta-linux_update_verify_4/4', 'release-mozilla-beta-linux_update_verify_2/4'] twistd.log.6:2013-07-19 01:56:37-0700 [-] starting build <Build release-mozilla-beta-win32_update_verify_1/4> using slave <SlaveBuilder builder='release-mozilla-beta-win32_update_verify_1/4' slave='w64-ix-slave06'> twistd.log.6:2013-07-19 01:56:37-0700 [-] starting build <Build release-mozilla-beta-win32_update_verify_1/4>.. pinging the slave <SlaveBuilder builder='release-mozilla-beta-win32_update_verify_1/4' slave='w64-ix-slave06'> twistd.log.6:2013-07-19 01:56:37-0700 [-] starting build <Build release-mozilla-beta-win32_update_verify_1/4> using slave <SlaveBuilder builder='release-mozilla-beta-win32_update_verify_1/4' slave='w64-ix-slave119'> twistd.log.6:2013-07-19 01:56:37-0700 [-] starting build <Build release-mozilla-beta-win32_update_verify_1/4>.. pinging the slave <SlaveBuilder builder='release-mozilla-beta-win32_update_verify_1/4' slave='w64-ix-slave119'> twistd.log.6:2013-07-19 01:56:37-0700 [Broker,6779,10.12.40.151] <Build release-mozilla-beta-win32_update_verify_1/4>.startBuild twistd.log.6:2013-07-19 01:56:37-0700 [Broker,6897,10.12.40.26] <Build release-mozilla-beta-win32_update_verify_1/4>.startBuild From bm64: 2013-07-18 21:44:11-0700 [-] prioritizeBuilders: 0.10s prioritized 2 builder(s): [((0, 0L, 100, 1374209048L), 'release-mozilla-beta-linux64_partner_repack'), ((6, 0L, 100, 1373746552L), 'fuzzer-linux64')] 2013-07-18 21:44:11-0700 [-] prioritizeBuilders: 0.10s important builder release-mozilla-beta-linux64_partner_repack (p == (0, 0L, 100, 1374209048L)) 2013-07-18 21:44:11-0700 [-] prioritizeBuilders: 0.10s found 2 important builder(s): ['fuzzer-linux64', 'release-mozilla-beta-linux64_partner_repack'] 2013-07-18 21:44:12-0700 [-] starting build <Build release-mozilla-beta-linux64_partner_repack> using slave <SlaveBuilder builder='release-mozilla-beta-linux64_partner_repack' slave='bld-lion-r5-074'> 2013-07-18 21:44:12-0700 [-] starting build <Build release-mozilla-beta-linux64_partner_repack>.. pinging the slave <SlaveBuilder builder='release-mozilla-beta-linux64_partner_repack' slave='bld-lion-r5-074'> 2013-07-18 21:44:12-0700 [Broker,41718,10.26.52.94] <Build release-mozilla-beta-linux64_partner_repack>.startBuild 2013-07-18 21:44:41-0700 [-] <Build release-mozilla-beta-linux64_partner_repack>: build finished From bm66: twistd.log.4:2013-07-18 21:44:19-0700 [-] prioritizeBuilders: 0.11s prioritized 1 builder(s): [((0, 0L, 100, 1374209054L), 'release-mozilla-beta-linux64_partner_repack')] twistd.log.4:2013-07-18 21:44:19-0700 [-] prioritizeBuilders: 0.11s important builder release-mozilla-beta-linux64_partner_repack (p == (0, 0L, 100, 1374209054L)) twistd.log.4:2013-07-18 21:44:19-0700 [-] prioritizeBuilders: 0.11s found 1 important builder(s): ['release-mozilla-beta-linux64_partner_repack'] twistd.log.4:2013-07-18 21:44:20-0700 [-] starting build <Build release-mozilla-beta-linux64_partner_repack> using slave <SlaveBuilder builder='release-mozilla-beta-linux64_partner_repack' slave='bld-lion-r5-080'> twistd.log.4:2013-07-18 21:44:20-0700 [-] starting build <Build release-mozilla-beta-linux64_partner_repack>.. pinging the slave <SlaveBuilder builder='release-mozilla-beta-linux64_partner_repack' slave='bld-lion-r5-080'> twistd.log.4:2013-07-18 21:44:20-0700 [Broker,31897,10.26.52.100] <Build release-mozilla-beta-linux64_partner_repack>.startBuild twistd.log.4:2013-07-18 21:44:47-0700 [-] <Build release-mozilla-beta-linux64_partner_repack>: build finished From bm81 (the scheduler): master/twistd.log.4:2013-07-18 21:44:08-0700 [-] AggregatingScheduler(release-mozilla-beta-linux64_firefox_l10n_done) <id=50825480>: new buildset: branch=releases/mozilla-beta, ssid=5574135, builders: release-mozilla-beta-linux64_partner_repack master/twistd.log.4:2013-07-18 21:44:14-0700 [-] AggregatingScheduler(release-mozilla-beta-linux64_firefox_l10n_done) <id=50825480>: new buildset: branch=releases/mozilla-beta, ssid=5574136, builders: release-mozilla-beta-linux64_partner_repack master/twistd.log.3:2013-07-19 01:56:17-0700 [-] AggregatingScheduler(release-mozilla-beta-updates_done) <id=50775824>: new buildset: branch=releases/mozilla-beta, ssid=5575297, builders: release-mozilla-beta-linux_update_verify_1/4, release-mozilla-beta-linux_update_verify_2/4, release-mozilla-beta-linux_update_verify_3/4, release-mozilla-beta-linux_update_verify_4/4, release-mozilla-beta-linux64_update_verify_1/4, release-mozilla-beta-linux64_update_verify_2/4, release-mozilla-beta-linux64_update_verify_3/4, release-mozilla-beta-linux64_update_verify_4/4, release-mozilla-beta-macosx64_update_verify_1/4, release-mozilla-beta-macosx64_update_verify_2/4, release-mozilla-beta-macosx64_update_verify_3/4, release-mozilla-beta-macosx64_update_verify_4/4, release-mozilla-beta-win32_update_verify_1/4, release-mozilla-beta-win32_update_verify_2/4, release-mozilla-beta-win32_update_verify_3/4, release-mozilla-beta-win32_update_verify_4/4 master/twistd.log.3:2013-07-19 01:56:22-0700 [-] AggregatingScheduler(release-mozilla-beta-updates_done) <id=50775824>: new buildset: branch=releases/mozilla-beta, ssid=5575298, builders: release-mozilla-beta-linux_update_verify_1/4, release-mozilla-beta-linux_update_verify_2/4, release-mozilla-beta-linux_update_verify_3/4, release-mozilla-beta-linux_update_verify_4/4, release-mozilla-beta-linux64_update_verify_1/4, release-mozilla-beta-linux64_update_verify_2/4, release-mozilla-beta-linux64_update_verify_3/4, release-mozilla-beta-linux64_update_verify_4/4, release-mozilla-beta-macosx64_update_verify_1/4, release-mozilla-beta-macosx64_update_verify_2/4, release-mozilla-beta-macosx64_update_verify_3/4, release-mozilla-beta-macosx64_update_verify_4/4, release-mozilla-beta-win32_update_verify_1/4, release-mozilla-beta-win32_update_verify_2/4, release-mozilla-beta-win32_update_verify_3/4, release-mozilla-beta-win32_update_verify_4/4 The double "new buildset" entries are pretty indicative of a problem. Makes me suspect either time problems on the scheduler master or dataloss in the db. I'm not smart enough to back either of those up though.
This is almost certainly because of bug 894201.
Summary: duplicate jobs triggered in 23.0b6 and b7 → duplicate jobs triggered in 23.0b6 b7 and b8
Didn't get any duplicate jobs in b9. I'm pretty sure that bug 894201 fixed this.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.