551086 - "compare previous leak logs failed" on Windows debug build

Status: RESOLVED WORKSFORME

(Release Engineering :: General, defect)

Description

[Myk Melez [:myk] [@mykmelez]]

The Windows debug build for two changesets I pushed to the 1.9.2 branch today failed with the message "compare previous leak logs failed" <http://tinderbox.mozilla.org/showlog.cgi?log=Firefox3.6/1268087462.1268091928.15187.gz&fulltext=1>:

The following build log snippet is relevant:

  obj-firefox\dist\bin\leakstats.exe starting at Mon Mar 08 15:42:01 2010
  obj-firefox\dist\bin\leakstats.exe: no callsite for 'M' (368656)!
  Unknown event type �
  obj-firefox\dist\bin\leakstats.exe: log file incomplete
  program finished with exit code 1
  elapsedTime=1.547000
  Unable to parse leakstats output

I chatted with Aki and nthomas on IRC, who thought it was more likely to be a build machine issue or a race on log push/pull than a problem with the changesets, and Aki started another debug build for me, but it too failed with the same problem <http://tinderbox.mozilla.org/showlog.cgi?log=Firefox3.6/1268093726.1268099214.1256.gz&fulltext=1>:

  obj-firefox\dist\bin\leakstats.exe starting at Mon Mar 08 17:38:37 2010
  obj-firefox\dist\bin\leakstats.exe: log file incomplete
  program finished with exit code 1
  elapsedTime=2.750000
  Unable to parse leakstats output

The two changesets for which the problem is occurring:

  http://hg.mozilla.org/releases/mozilla-1.9.2/rev/a57bc2f49c9d
  http://hg.mozilla.org/releases/mozilla-1.9.2/rev/b54d9f55473a

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

Looks like both failures were for rev a57bc2f49c9d.

The second case is definitely a problem with the log being replaced under while apache is serving it to another job. The build starting at 16:53 got a slave with a recent objdir and was uploading malloc.log between 17:33:58 and 17:36:20. The build starting at 16:15 (the second build of a57bc2f49c9d) got a slave without a prior build and was downloading malloc.log between 17:34 and 17:38. Hence the incomplete log.

Haven't found a smoking gun for the first failing build yet. Also note that the debug builds have been green twice since.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Can't find any reason for the failure in the 14:31 build. The preceeding build (at 14:11) didn't have any problems and ran green, but I've clobbered mw32-ix-slave15 as a precaution.

bug 539334 covers the race.