1292976 - Intermittent TEST-UNEXPECTED-TIMEOUT | dom/media/tests/mochitest/test_peerConnection_restartIce.html | application timed out after 330 seconds with no output

Status: RESOLVED INCOMPLETE

(Core :: WebRTC: Networking, defect, P3)

Description

[Treeherder Bug Filer]

Filed by: philringnalda [at] gmail.com

https://treeherder.mozilla.org/logviewer.html#?job_id=1480334&repo=autoland

http://archive.mozilla.org/pub/firefox/tinderbox-builds/autoland-macosx64/1470403041/autoland_yosemite_r7_test-mochitest-media-e10s-bm136-tests1-macosx-build85.txt.gz

Comment 1

[Nils Ohlmeier [:drno]]

Hmm something apparently calls some kind of abort() function here, which then results in a test timeout laster:

09:30:28     INFO -  (ice/NOTICE) ICE(PC:1470414627968384 (id=2147483869 url=http://mochi.test:8888/tests/dom/media/tests/mochitest/test_peerConnection_restartIce.html)): peer (PC:1470414627968384 (id=2147483869 url=http://mochi.test:8888/tests/dom/media/tests/mochitest/test_peerConnection_restartIce.html):default) no streams with pre-answer requests
09:30:28     INFO -  (ice/NOTICE) ICE(PC:1470414627968384 (id=2147483869 url=http://mochi.test:8888/tests/dom/media/tests/mochitest/test_peerConnection_restartIce.html)): peer (PC:1470414627968384 (id=2147483869 url=http://mochi.test:8888/tests/dom/media/tests/mochitest/test_peerConnection_restartIce.html):default) no checks to start
09:30:28     INFO -  Redirecting call to abort() to mozalloc_abort
09:30:29     INFO -  ###!!! [Parent][MessageChannel] Error: (msgtype=0x2E007D,name=PBrowser::Msg_Destroy) Channel error: cannot send/recv
09:35:59     INFO -  2111 INFO TEST-PASS | dom/media/tests/mochitest/test_peerConnection_restartIce.html | A valid string reason is expected
09:35:59     INFO -  2112 INFO TEST-PASS | dom/media/tests/mochitest/test_peerConnection_restartIce.html | Reason cannot be empty
09:35:59     INFO -  2113 INFO Skipping ICE Server for this test

Comment 2

[Byron Campen [:bwc]]

Looks like something has gone wrong in mozilla::layers::ImageContainer::GetCurrentImages when it tries to unlock a mutex.

09:36:18     INFO -   0  libmozglue.dylib!mozalloc_abort(char const*) [mozalloc_abort.cpp:192132a4cbc7 : 33 + 0x0]
09:36:18     INFO -      rax = 0x0000000000000001   rdx = 0x00007fff7603c1f8
09:36:18     INFO -      rcx = 0x0000000000000000   rbx = 0x00007fff7603cc50
09:36:18     INFO -      rsi = 0x001c5700001c5700   rdi = 0x001c5600001c5703
09:36:18     INFO -      rbp = 0x00000001221806d0   rsp = 0x00000001221806c0
09:36:18     INFO -       r8 = 0x0000000122180650    r9 = 0x0000000122181000
09:36:18     INFO -      r10 = 0x0000000122180690   r11 = 0x0000000000000246
09:36:18     INFO -      r12 = 0x8000000400000000   r13 = 0x0000000122180d50
09:36:18     INFO -      r14 = 0x0000000110f30000   r15 = 0x0000000000000040
09:36:18     INFO -      rip = 0x0000000110b4d3a4
09:36:18     INFO -      Found by: given as instruction pointer in context
09:36:18     INFO -   1  libmozglue.dylib!abort [mozalloc_abort.cpp:192132a4cbc7 : 80 + 0xc]
09:36:18     INFO -      rbx = 0x0000000110f300c0   rbp = 0x00000001221806e0
09:36:18     INFO -      rsp = 0x00000001221806e0   r12 = 0x8000000400000000
09:36:18     INFO -      r13 = 0x0000000122180d50   r14 = 0x0000000110f30000
09:36:18     INFO -      r15 = 0x0000000000000040   rip = 0x0000000110b4d3d0
09:36:18     INFO -      Found by: call frame info
09:36:18     INFO -   2  libsystem_pthread.dylib!__pthread_abort + 0x31
09:36:18     INFO -      rbx = 0x0000000110f300c0   rbp = 0x00000001221806f0
09:36:18     INFO -      rsp = 0x00000001221806f0   r12 = 0x8000000400000000
09:36:18     INFO -      r13 = 0x0000000122180d50   r14 = 0x0000000110f30000
09:36:18     INFO -      r15 = 0x0000000000000040   rip = 0x00007fff8f716e1f
09:36:18     INFO -      Found by: call frame info
09:36:18     INFO -   3  libsystem_pthread.dylib!__pthread_abort_reason + 0xb4
09:36:18     INFO -      rbp = 0x00000001221807e0   rsp = 0x0000000122180700
09:36:18     INFO -      rip = 0x00007fff8f716ed3
09:36:18     INFO -      Found by: previous frame's frame pointer
09:36:18     INFO -   4  libsystem_pthread.dylib!pthread_mutex_unlock + 0xf6
09:36:18     INFO -      rbp = 0x0000000122180820   rsp = 0x00000001221807f0
09:36:18     INFO -      rip = 0x00007fff8f711aa1
09:36:18     INFO -      Found by: previous frame's frame pointer
09:36:18     INFO -   5  XUL!mozilla::layers::ImageContainer::GetCurrentImages(nsTArray<mozilla::layers::ImageContainer::OwningImage>*, unsigned int*) [ReentrantMonitor.h:192132a4cbc7 : 74 + 0x5]
09:36:18     INFO -      rbp = 0x0000000122180850   rsp = 0x0000000122180830
09:36:18     INFO -      rip = 0x0000000107d956ac
09:36:18     INFO -      Found by: previous frame's frame pointer

Comment 3

[Nils Ohlmeier [:drno]]

Makes we wonder if this related to bug 1292785 and bug 1292774?

Comment 4

[Intermittent Failures Robot]

1 failures in 792 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292976&startday=2017-11-27&endday=2017-12-03&tree=all

Comment 5

[Intermittent Failures Robot]

1 failures in 889 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292976&startday=2017-12-04&endday=2017-12-10&tree=all

Comment 6

[Byron Campen [:bwc]]

ASAN crash (read of unknown address) in copy_and_extend_plane /builds/worker/workspace/build/src/media/libvpx/libvpx/vp8/common/extend.c:37:23

Comment 7

[Randell Jesup [:jesup] (needinfo me)]

We've had crashes here before.  This is a buffer-overrun (bounds) failure almost certainly.  In the past this sort of problem was due to resolution changes that hadn't rippled down into the VPX encoder when the frame got down there.   These buffers are sized based on the resolution the frame is configured to.  Alternatively something might have trashed the size between when it was checked and here (very unlikely).

Comment 8

[Randell Jesup [:jesup] (needinfo me)]

So the last crash reported appears to be a separate issue, and not a timeout really.

Comment 9

[Phil Ringnalda (:philor)]

Yeah, welcome to the new world order, where everybody insists on shoving PIDs and paths into every line of output, and doing their crashing and asserting OMT, and the only thing we have that we can get a suggestible bug out of is the main thread timeout. Sorry, you'll have to take it from there, since neither treeherder nor the people creating the output are willing to take all the crap that breaks bug suggestions out.

Comment 10

[Intermittent Failures Robot]

1 failures in 423 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-esr52: 1

Platform breakdown:
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292976&startday=2017-12-11&endday=2017-12-17&tree=all

Comment 11

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 12

[Intermittent Failures Robot]

1 failures in 657 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292976&startday=2018-01-15&endday=2018-01-21&tree=all

Comment 13

[Intermittent Failures Robot]

7 failures in 701 pushes (0.01 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 2
* mozilla-inbound: 2
* autoland: 2
* mozilla-central: 1

Platform breakdown:
* linux32-stylo-disabled: 2
* linux32: 2
* linux64-stylo-disabled: 1
* linux64-qr: 1
* linux64-devedition: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292976&startday=2018-01-22&endday=2018-01-28&tree=all

Comment 14

[Intermittent Failures Robot]

11 failures in 735 pushes (0.015 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 6
* mozilla-inbound: 2
* mozilla-central: 2
* try: 1

Platform breakdown:
* linux64: 5
* linux32: 3
* linux32-stylo-disabled: 2
* linux64-stylo-disabled: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292976&startday=2018-01-29&endday=2018-02-04&tree=all

Comment 15

[Intermittent Failures Robot]

1 failures in 702 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 1

Platform breakdown:
* linux64-stylo-disabled: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292976&startday=2018-02-05&endday=2018-02-11&tree=all

Comment 16

[Intermittent Failures Robot]

9 failures in 685 pushes (0.013 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 5
* mozilla-inbound: 2
* mozilla-central: 2

Platform breakdown:
* linux32: 8
* linux32-stylo-disabled: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292976&startday=2018-02-12&endday=2018-02-18&tree=all

Comment 17

[Intermittent Failures Robot]

3 failures in 675 pushes (0.004 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 3

Platform breakdown:
* linux32: 2
* linux32-stylo-disabled: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292976&startday=2018-02-19&endday=2018-02-25&tree=all

Comment 18

[Intermittent Failures Robot]

3 failures in 831 pushes (0.004 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 2
* mozilla-beta: 1

Platform breakdown:
* linux32: 2
* linux64-stylo-disabled: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292976&startday=2018-02-26&endday=2018-03-04&tree=all

Comment 19

[Intermittent Failures Robot]

1 failures in 747 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-release: 1

Platform breakdown:
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292976&startday=2018-03-05&endday=2018-03-11&tree=all

Comment 20

[Intermittent Failures Robot]

1 failures in 822 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* linux32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1292976&startday=2018-03-20&endday=2018-03-26&tree=trunk

Comment 21

[Intermittent Failures Robot]

2 failures in 718 pushes (0.003 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 2

Platform breakdown:
* linux64-qr: 1
* linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1292976&startday=2018-03-26&endday=2018-04-01&tree=trunk

Comment 22

[Intermittent Failures Robot]

1 failures in 765 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 1

Platform breakdown:
* linux32-stylo-disabled: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1292976&startday=2018-04-02&endday=2018-04-08&tree=trunk

Comment 23

[Intermittent Failures Robot]

1 failures in 782 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* linux32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1292976&startday=2018-04-16&endday=2018-04-22&tree=trunk

Comment 24

[Intermittent Failures Robot]

1 failures in 551 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-esr52: 1

Platform breakdown:
* windowsxp: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1292976&startday=2018-04-30&endday=2018-05-06&tree=trunk

Comment 25

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 26

[Intermittent Failures Robot]

1 failures in 408 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-esr52: 1

Platform breakdown:
* windowsxp: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1292976&startday=2018-06-11&endday=2018-06-17&tree=trunk