475500 - Need core dump for macosx/crashtest running patch in bug 474771

Status: RESOLVED INVALID

(Release Engineering :: General, defect, P1)

Description

[Andreas Gal :gal]

When trying to land 474771 the macosx tinderbox crashed during the talos run. I was not able to reproduce the crash offline or using the try server. I need the core dump of a talos VM crashing from this patch, along with the executable (with debug info would be fantastic). 

474771 is a P1 blocker for JS.

Comment 1

[Andreas Gal :gal]

Requesting b+ since its blocking a blocker.

Comment 2

[Andreas Gal :gal]

This is the offending patch:

https://bugzilla.mozilla.org/attachment.cgi?id=359016

Comment 3

[[e-mail] (:bhearsum)]

I don't know how to do this. Ted tells me that we don't keep symbols for hourly builds so getting a core dump from any old run would be basically useless. Are we supposed to manually do a build, upload symbols and then manually run it through Talos and wait for a crash?

Didn't we have to do something like this for b1 or b2? Anyone remember what we did there?

Comment 4

[[e-mail] (:bhearsum)]

Now that I think about it I think a bit more we may have given someone access to a Talos machine so they could reproduce it themselves. Not entirely sure about that. Can anyone confirm that?

Comment 5

[Phil Ringnalda (:philor)]

bug 461020 - you pulled the try talos box out of production, scrubbed it, and gave it to mrbkap.

Comment 6

[Brendan Eich [:brendan]]

mrbkap got access to do something like this, once.

/be

Comment 7

[[e-mail] (:bhearsum)]

Okay, I'll have to defer to John and/or Alice here, they know what the process is.

Comment 8

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Andreas: 

So this crashed on one Talos mac, but not a try Talos mac? These machines are imaged the same, so maybe this is intermittent crash? Can you try sending the same patch 3-4 times to the try server? If *any* of your try runs crash, we can pull that slave out of the pool for you to poke around (and not get clobbered by the next incoming request!). Let us know when you are starting to send jobs to try server, so we can be ready-and-waiting as the runs complete.

(Comment#5, comment#6 are both correct: we did exactly the same when mrbkap was chasing an intermittent crash in bug#461020)

Comment 9

[Mike Shaver (:shaver emeritus)]

Last comment in 474771 indicates that it didn't even crash Talos the last time it was landed on TM -- got backed out for unit-test crash and reftest failure (!), so reducing this in priority and reassigning to Andreas until we're sure that we need it.  Sorry for the noise, releng!

Comment 10

[Andreas Gal :gal]

We submitted the patch last night again to confirm that its still crashing and talos cycled now and dep unit failed. I will try to reproduce the latter offline (we only tried talos the last 2 days, not dep unit, since that used to pass)

Comment 11

[Andreas Gal :gal]

Ok, I am unable to reproduce this offline (again) using the following command line:

../../debug-build/dist/MinefieldDebug.app/Contents/MacOS/firefox -console -P default -reftest crashtests.list -no-remote

Is there anything else we could do offline to avoid pulling a core straight from the VM running the test?

Comment 12

[Nick Thomas [:nthomas] (UTC+12)]

Mac/unit on tracemonkey seems to be green since the patch landed there b0592ff0f21a/Tue Jan 27 16:41. But in the merge to m-c today there's a

REFTEST TEST-UNEXPECTED-FAIL | file:///builds/moz2_slave/mozilla-central-macosx-unittest/build/layout/base/crashtests/403175-1.html | timed out waiting for reftest-wait to be removed (after onload fired)

Is that the signature of this issue ?

Comment 13

[Andreas Gal :gal]

Its hard to tell. We had a bunch of sporadic crashes in both TM and MC. We can't clearly associate any of them with any particular patch. I will invalide this bug for the time being. I think a mechanism to get crash info of the tinderboxes is in the works as we speak.