1686468 - Intermittent dom/canvas/test/webgl-conf/generated/<anything>.html | application crashed [@ js::jit::WarpOracle::createSnapshot()] post Assertion failure: cx_->hadNondeterministicException(), at js/src/jit/WarpOracle.cpp:190

Status: RESOLVED DUPLICATE of bug 1698609

(Core :: JavaScript Engine: JIT, defect, P3)

Description

[Treeherder Bug Filer]

Filed by: rmaries [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=326591286&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/IkP_DeRiT5m0b9EYjmkFtw/runs/0/artifacts/public/logs/live_backing.log

---

[task 2021-01-13T14:22:15.514Z] 14:21:44     INFO -  499 INFO TEST-START | dom/canvas/test/webgl-conf/generated/test_deqp__data__gles2__shaders__functions.html
[task 2021-01-13T14:22:15.514Z] 14:22:08     INFO -  wait for org.mozilla.geckoview.test complete; top activity=com.bitbar.testdroid.monitor
[task 2021-01-13T14:22:15.514Z] 14:22:09     INFO -  remoteautomation.py | Application ran for: 0:09:15.322286
[task 2021-01-13T14:22:15.514Z] 14:22:10     INFO -  mozcrash Copy/paste: /builds/task_161054700821469/fetches/minidump_stackwalk/minidump_stackwalk /tmp/tmphLt9da/08d45090-7410-72c9-5d2e-51778dec594e.dmp /builds/task_161054700821469/workspace/build/symbols
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -  mozcrash Saved minidump as /builds/task_161054700821469/workspace/build/blobber_upload_dir/08d45090-7410-72c9-5d2e-51778dec594e.dmp
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -  mozcrash Saved app info as /builds/task_161054700821469/workspace/build/blobber_upload_dir/08d45090-7410-72c9-5d2e-51778dec594e.extra
[task 2021-01-13T14:22:15.514Z] 14:22:15  WARNING -  PROCESS-CRASH | dom/canvas/test/webgl-conf/generated/test_deqp__data__gles2__shaders__functions.html | application crashed [@ js::jit::WarpOracle::createSnapshot()]
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -  Mozilla crash reason: MOZ_ASSERT(cx_->hadNondeterministicException())
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -  Crash dump filename: /tmp/tmphLt9da/08d45090-7410-72c9-5d2e-51778dec594e.dmp
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -  Operating system: Android
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -                    0.0.0 Linux 4.4.56-g594d847d09a1 #1 SMP PREEMPT Thu Oct 26 22:34:08 UTC 2017 aarch64
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -  CPU: arm64
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       8 CPUs
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -  GPU: UNKNOWN
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -  Crash reason:  SIGSEGV /SEGV_MAPERR
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -  Crash address: 0x0
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -  Process uptime: not available
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -  Thread 11 (crashed)
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -   0  libxul.so!js::jit::WarpOracle::createSnapshot() [WarpOracle.cpp:efbb163e768156dceb4270720525cda5bcdb85ee : 190 + 0x34]
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       x0 = 0x0000000000000085    x1 = 0x000000749f7e4850
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       x2 = 0x0000000000000004    x3 = 0x0000000000000003
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       x4 = 0x0000000040100401    x5 = 0x40404000a800a800
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       x6 = 0x0000000000000000    x7 = 0x7f7f7f7f7f7f7f7f
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       x8 = 0x0000000000000000    x9 = 0x000000749eeb9d80
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x10 = 0x000000749c46a42a   x11 = 0x0000000000000079
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x12 = 0x000000749f7e4b28   x13 = 0xffffffffffffffff
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x14 = 0xff00000000000000   x15 = 0xffffffffffffffff
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x16 = 0x00000074b6bf1cd0   x17 = 0x00000074b6afd268
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x18 = 0x0000000000000000   x19 = 0x00000000000000be
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x20 = 0x0000007408dc0fb8   x21 = 0x00000074088150a0
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x22 = 0x000000749f7e5068   x23 = 0x000000749f7e8588
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x24 = 0x0000000000000003   x25 = 0x0000007409d12580
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x26 = 0x0000000000000000   x27 = 0x000000749f7e8588
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x28 = 0x0000000000000003    fp = 0x000000749f7e4fe0
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       lr = 0x00000074998632d8    sp = 0x000000749f7e4f60
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       pc = 0x00000074998632f0
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      Found by: given as instruction pointer in context
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -   1  libxul.so!js::jit::CreateWarpSnapshot(JSContext*, js::jit::MIRGenerator*, JS::Handle<JSScript*>) [Ion.cpp:efbb163e768156dceb4270720525cda5bcdb85ee : 1619 + 0x4]
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x19 = 0x000000749f7e5330   x20 = 0x0000007408dbb170
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x21 = 0x000000749e62f000   x22 = 0x000000749f7e8588
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x23 = 0x000000749f7e5040   x24 = 0x000000749e621000
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x25 = 0x0000007408dbb108   x26 = 0x0000007405cd5540
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x27 = 0x000000749f7e8588   x28 = 0x0000000000000003
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       fp = 0x000000749f7e5120    sp = 0x000000749f7e5030
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       pc = 0x00000074994c99fc
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      Found by: call frame info
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -   2  libxul.so!js::jit::IonCompile(JSContext*, JS::Handle<JSScript*>, js::jit::BaselineFrame*, unsigned int, unsigned char*, bool, js::jit::OptimizationLevel) [Ion.cpp:efbb163e768156dceb4270720525cda5bcdb85ee : 1704 + 0xc]
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x19 = 0x000000749e62f000   x20 = 0x0000000000000000
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x21 = 0x000000740d0cd56c   x22 = 0x0000007408dbb170
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x23 = 0x0000007408dbb0d0   x24 = 0x000000749e621000
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x25 = 0x0000007408dbb108   x26 = 0x0000007405cd5540
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x27 = 0x000000749f7e8588   x28 = 0x0000000000000003
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       fp = 0x000000749f7e5240    sp = 0x000000749f7e5160
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       pc = 0x00000074994c93f0
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      Found by: call frame info
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -   3  libxul.so!js::jit::Compile(JSContext*, JS::Handle<JSScript*>, js::jit::BaselineFrame*, unsigned int, unsigned char*, bool) [Ion.cpp:efbb163e768156dceb4270720525cda5bcdb85ee : 1961 + 0x10]
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x19 = 0x000000749f7e5330   x20 = 0x000000749e62f000
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x21 = 0x000000740d0cd56c   x22 = 0x0000000000000000
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x23 = 0x0000000000000001   x24 = 0x0000007405cd5540
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x25 = 0x000000749f7e8588   x26 = 0x0000007405cd46f0
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      x27 = 0x000000749e32b000   x28 = 0x000000749f7e5420
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       fp = 0x000000749f7e52c0    sp = 0x000000749f7e52a0
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -       pc = 0x00000074994a8b80
[task 2021-01-13T14:22:15.514Z] 14:22:15     INFO -      Found by: call frame info
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -   4  libxul.so!IonCompileScriptForBaseline(JSContext*, js::jit::BaselineFrame*, unsigned int, unsigned char*) [Ion.cpp:efbb163e768156dceb4270720525cda5bcdb85ee : 2202 + 0x10]
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x19 = 0x000000749e62f000   x20 = 0x000000749f7e5480
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x21 = 0x000000740d0cd56c   x22 = 0x000000749f7e5330
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x23 = 0x0000000000000093   x24 = 0x000000749f7e8588
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x25 = 0x000000749e32b000   x26 = 0x000000749e32b7a2
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x27 = 0x000000749e32b000   x28 = 0x000000749f7e5420
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -       fp = 0x000000749f7e5340    sp = 0x000000749f7e5320
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -       pc = 0x00000074994a93a8
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      Found by: call frame info
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -   5  libxul.so!js::jit::IonCompileScriptForBaselineOSR(JSContext*, js::jit::BaselineFrame*, unsigned int, unsigned char*, js::jit::IonOsrTempData**) [Ion.cpp:efbb163e768156dceb4270720525cda5bcdb85ee : 2316 + 0x10]
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x19 = 0x000000749f7e5430   x20 = 0x000000749f7e5480
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x21 = 0x000000749e62f000   x22 = 0x0000000000000068
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x23 = 0x000000740d0cd56c   x24 = 0x000000749f7e8588
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x25 = 0x000000749f7e5878   x26 = 0x000000749e62f000
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x27 = 0x000000740e610b10   x28 = 0x000000749f7e5420
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -       fp = 0x000000749f7e53c0    sp = 0x000000749f7e53a0
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -       pc = 0x00000074994a9b9c
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      Found by: call frame info
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -   6  0x740e61d970
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x19 = 0x000000749e480bc0   x20 = 0x0000000000000000
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x21 = 0x000000749f7e5770   x22 = 0x0000000000000002
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x23 = 0x000000749f7e54c8   x24 = 0xfffe007409f13228
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x25 = 0x000000749f7e5878   x26 = 0x000000749e62f000
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      x27 = 0x000000740e610b10   x28 = 0x000000749f7e5420
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -       fp = 0x000000749f7e5830    sp = 0x000000749f7e5420
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -       pc = 0x000000740e61d974
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      Found by: call frame info
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -   7  0x740e61d970
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -       fp = 0x00000074994e59e0    lr = 0x000000749f7e5bc0
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -       sp = 0x000000749f7e5840    pc = 0x000000740e61d974
[task 2021-01-13T14:22:15.515Z] 14:22:15     INFO -      Found by: previous frame's frame pointer```

Comment 1

[Nicolas B. Pierron [:nbp]]

Iain, would you be the right person to investigate WarpOracle issues?

Comment 2

[Iain Ireland [:iain]]

I am one of the right people in general, and for this bug in particular I am the exact culprit, because I just landed this assertion last week.

This is the assertion to verify that we don't have bailout loops. It works using a checksum of the ICScript. It's not immediately obvious whether we're intermittently catching a real bug, or whether this is just hash collision. The fuzzer is still finding real bugs, so I'm going to fix those first. After that, if we're still seeing intermittent failures like this, I'll increase the hash from 32 to 64 bits.

Comment 3

[Intermittent Failures Robot]

5 failures in 3810 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 5

Platform breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1686468&startday=2021-01-11&endday=2021-01-17&tree=all

Comment 4

[Nicolas B. Pierron [:nbp]]

(In reply to Iain Ireland [:iain] from comment #2)

This is the assertion to verify that we don't have bailout loops. It works using a checksum of the ICScript. It's not immediately obvious whether we're intermittently catching a real bug, or whether this is just hash collision. The fuzzer is still finding real bugs, so I'm going to fix those first. After that, if we're still seeing intermittent failures like this, I'll increase the hash from 32 to 64 bits.

Could this be a flag which is only enabled when fuzzing? Especially since I would think that the crash-stat report are non-actionable for finding where the bailout loop comes from.

Maybe a profiler marker might be a better report than an assertion, to get better feedback from actual users, than crashing and losing users …

Comment 5

[Iain Ireland [:iain]]

It's a debug assertion. It will not show up in crash-stats.

Comment 6

[Intermittent Failures Robot]

5 failures in 3488 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 5

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 5
  • debug: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1686468&startday=2021-01-18&endday=2021-01-24&tree=all

Comment 7

[Intermittent Failures Robot]

6 failures in 3891 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2
• autoland: 4

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 5
  • debug: 5
• windows10-64: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1686468&startday=2021-01-25&endday=2021-01-31&tree=all

Comment 8

[Intermittent Failures Robot]

11 failures in 3966 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 9
• mozilla-central: 2

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 10
  • debug: 10
• windows7-32-mingwclang: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1686468&startday=2021-02-01&endday=2021-02-07&tree=all

Comment 9

[Intermittent Failures Robot]

13 failures in 4322 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 7
• mozilla-central: 6

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 13
  • debug: 13

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1686468&startday=2021-02-08&endday=2021-02-14&tree=all

Comment 10

[Intermittent Failures Robot]

7 failures in 3737 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 7

Platform and build breakdown:

• macosx1014-64-qr: 1
  • debug: 1
• android-hw-p2-8-0-android-aarch64-qr: 5
  • debug: 5
• windows7-32-mingwclang: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1686468&startday=2021-02-15&endday=2021-02-21&tree=all

Comment 11

[Intermittent Failures Robot]

9 failures in 4327 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2
• autoland: 7

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 9
  • debug: 9

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1686468&startday=2021-02-22&endday=2021-02-28&tree=all

Comment 12

[Intermittent Failures Robot]

8 failures in 3683 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 5
• mozilla-central: 3

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 7
  • debug: 7
• windows10-64-mingwclang: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1686468&startday=2021-03-01&endday=2021-03-07&tree=all

Comment 13

[Intermittent Failures Robot]

18 failures in 4622 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 4
• autoland: 14

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 17
  • debug: 17
• windows10-64: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1686468&startday=2021-03-08&endday=2021-03-14&tree=all

Comment 14

[Intermittent Failures Robot]

7 failures in 4257 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 5
• mozilla-central: 2

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 7
  • debug: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1686468&startday=2021-03-15&endday=2021-03-21&tree=all

Comment 15

[Iain Ireland [:iain]]

This may be a duplicate of bug 1698609.

Comment 16

[Intermittent Failures Robot]

4 failures in 3967 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 3
• autoland: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 4
  • debug: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1686468&startday=2021-03-22&endday=2021-03-28&tree=all

Comment 17

[Intermittent Failures Robot]

1 failures in 3125 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform and build breakdown:

• linux1804-64: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1686468&startday=2021-03-29&endday=2021-04-04&tree=all

Comment 18

[Iain Ireland [:iain]]

The only failure since the fix for bug 1698609 landed was a try build based on a much older version of central. I think it's safe to conclude that this was a duplicate of that bug.