Closed Bug 1149476 Opened 9 years ago Closed 9 years ago

Intermittent ecma/Date/15.9.5.23-1.js,test262/ch10/10.6/10.6-14-1-s.js,flags.js | (args: ""),test262/ch08/8.12/8.12.1/8.12.1-1_27.js | Assertion failure: stackBase, at /builds/slave/m-in_l64-d_sm-compacting-00000/src/js/src/jsnativestack.cpp:155

Categories

(Core :: JavaScript Engine, defect)

Product:
Core
Component:
JavaScript Engine
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla40
Tracking Flags:
Tracking Status
firefox38 --- fixed
firefox38.0.5 --- fixed
firefox39 --- fixed
firefox40 --- fixed
firefox-esr31 --- unaffected

People

(Reporter: cbook, Assigned: sfink)

References

(
URL
)

Details

(Keywords: crash, intermittent-failure)

Attachments

(2 files, 2 obsolete files)

Lengthen timeout because we are hitting it with SM(cgc), (also jonco for a more complex version)
1.26 KB, patch
Details | Diff | Splinter Review
Extend autospider.sh timeout and emit a log message
1.57 KB, patch
jonco
: review+
Details | Diff | Splinter Review 
Linux x86-64 mozilla-inbound leak test spidermonkey_tier_1-compacting build

https://treeherder.mozilla.org/logviewer.html#?job_id=8272344&repo=mozilla-inbound


TEST-UNEXPECTED-FAIL | ecma/Date/15.9.5.23-1.js | (args: "") 
Assertion failure: stackBase, at /builds/slave/m-in_l64-d_sm-compacting-00000/src/js/src/jsnativestack.cpp:155
Only happening on compacting builds it seems.
Flags: needinfo?(jcoppeard)
(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #5)
The only way this can happen is if pthread_attr_getstack() leaves the stackBase out param as zero without returning an error code.  So this doesn't look like it can be related to compacting GC.

I had a look in the logs and the failures are all preceded by lines like:

  WARNING: Forcibly unmounting '/builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/slave' from chroot.
  State Changed: unlock buildroot

I don't know what this means but it sounds infra-related.  Ryan have you seen something like this before?
Flags: needinfo?(jcoppeard) → needinfo?(ryanvm)
SM test only output to the log if there's a failure, no? In other words, those could be happening on green runs but you wouldn't see them. So no, I don't think they're related.
Flags: needinfo?(ryanvm)
Can we safely rule out infra now? :)
Flags: needinfo?(jcoppeard)
(In reply to Jon Coppeard (:jonco) from comment #6)
> (In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #5)
> The only way this can happen is if pthread_attr_getstack() leaves the
> stackBase out param as zero without returning an error code.  So this
> doesn't look like it can be related to compacting GC.
> 
> I had a look in the logs and the failures are all preceded by lines like:
> 
>   WARNING: Forcibly unmounting
> '/builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/slave' from chroot.
>   State Changed: unlock buildroot
> 
> I don't know what this means but it sounds infra-related.  Ryan have you
> seen something like this before?

Look back a little farther, and you'll see something like

TEST-PASS | js1_7/geniter/regress-349012-03.js | (args: "")
TEST-PASS | js1_7/geniter/regress-351120.js | (args: "")
TEST-PASS | js1_7/geniter/regress-390918.js | (args: "")
++ kill 24669
WARNING: Command failed. See logs for output.
# umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/slave

That's a (relatively new) autospider.sh timeout. I should echo something more helpful to the log. That's a 7200 second timeout, and the job I'm looking at went 133 minutes.

A green SM(cgc) run I looked at went 124 seconds. So I'm guessing that SM(cgc) is very close to the timeout threshold, and when it times out, it kills the parent process before all of the tests processes are dead, which allows mock to forcibly unmount, which screws up some address space mapping in a way that results in this symptom. Ugh.
Flags: needinfo?(jcoppeard)
I was previously closing stdout and stderr out of paranoia, thinking it might hold things alive artificially. It probably won't, and we'll know very quickly if it does.
Attachment #8597017 - Flags: review?(jcoppeard)
Assignee: nobody → sphink
Status: NEW → ASSIGNED
Comment on attachment 8597017 [details] [diff] [review]
Extend autospider.sh timeout and emit a log message

Review of attachment 8597017 [details] [diff] [review]:
-----------------------------------------------------------------

Right, that sounds plausible!  Fix looks good.
Attachment #8597017 - Flags: review?(jcoppeard) → review+
Summary: Intermittent ecma/Date/15.9.5.23-1.js | (args: "") | Assertion failure: stackBase, at /builds/slave/m-in_l64-d_sm-compacting-00000/src/js/src/jsnativestack.cpp:155 → Intermittent ecma/Date/15.9.5.23-1.js | (args: ""),test262/ch08/8.12/8.12.1/8.12.1-1_27.js | Assertion failure: stackBase, at /builds/slave/m-in_l64-d_sm-compacting-00000/src/js/src/jsnativestack.cpp:155
The previous attempt seemed to cause problems with preventing the test script from exiting, which I was defending against by closing all file descriptors (even though I wasn't sure if it was necessary.) Sadly, I have no way to report the failure, but for now just lengthening the timeout seems like the simplest to make the intermittents go away.
The previous attempt seems to have kept the shell alive on regular runs (until it hits the no-output timeout.) Here's another attempt. I have a try push going to verify it this time; I have no idea whether it's going to work. But I'll be a jerk and r? now anyway, since I need to run for a bit and by the time you see this, it'll probably be done.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a68b6a5d3360
Attachment #8598236 - Flags: review?(jcoppeard)
Attachment #8597017 - Attachment is obsolete: true
Summary: Intermittent ecma/Date/15.9.5.23-1.js | (args: ""),test262/ch08/8.12/8.12.1/8.12.1-1_27.js | Assertion failure: stackBase, at /builds/slave/m-in_l64-d_sm-compacting-00000/src/js/src/jsnativestack.cpp:155 → Intermittent ecma/Date/15.9.5.23-1.js,test262/ch10/10.6/10.6-14-1-s.js | (args: ""),test262/ch08/8.12/8.12.1/8.12.1-1_27.js | Assertion failure: stackBase, at /builds/slave/m-in_l64-d_sm-compacting-00000/src/js/src/jsnativestack.cpp:155
Summary: Intermittent ecma/Date/15.9.5.23-1.js,test262/ch10/10.6/10.6-14-1-s.js | (args: ""),test262/ch08/8.12/8.12.1/8.12.1-1_27.js | Assertion failure: stackBase, at /builds/slave/m-in_l64-d_sm-compacting-00000/src/js/src/jsnativestack.cpp:155 → Intermittent ecma/Date/15.9.5.23-1.js,test262/ch10/10.6/10.6-14-1-s.js,flags.js | (args: ""),test262/ch08/8.12/8.12.1/8.12.1-1_27.js | Assertion failure: stackBase, at /builds/slave/m-in_l64-d_sm-compacting-00000/src/js/src/jsnativestack.cpp:155
...except I'm an idiot, and kill -1 is not SIGINT. That would be kill -2.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=017b12b57d5a
Attachment #8598332 - Flags: review?(jcoppeard)
Attachment #8598236 - Attachment is obsolete: true
Attachment #8598236 - Flags: review?(jcoppeard)
Attachment #8598332 - Flags: review?(jcoppeard) → review+
https://hg.mozilla.org/mozilla-central/rev/a7f8556dca89
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
status-firefox40: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla40
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: