Closed Bug 1197977 Opened 9 years ago Closed 9 years ago

Intermittent test_bug495300.html,test_seek-5.html,test_seek-10.html | application crashed [@ PR_GetThreadPrivate]

Categories

(Core :: Audio/Video: MediaStreamGraph, defect)

Product:
Core
Component:
Audio/Video: MediaStreamGraph
Platform:
ARM
Gonk (Firefox OS)
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla43
Tracking Flags:
Tracking Status
firefox41 --- unaffected
firefox42 --- fixed
firefox43 --- fixed
firefox-esr38 --- unaffected

People

(Reporter: RyanVM, Assigned: jya)

References

Details

(Keywords: crash, intermittent-failure)

Attachments

(3 files)

[webm]. Ensure resource length calculation never underflows.
2.10 KB, patch
kinetik
: review+
Details | Diff | Splinter Review
[webm] P1. Explicitly differentiate WebM usage for mediasource.
6.28 KB, patch
kinetik
: review+
Details | Diff | Splinter Review
[MSE/webm] P2. Construct WebMDemuxer with special MSE handling.
1020 bytes, patch
kinetik
: review+
Details | Diff | Splinter Review 
13:00:24 WARNING - PROCESS-CRASH | dom/media/test/test_seek-10.html | application crashed [@ PR_GetThreadPrivate]
13:00:24 INFO - Crash dump filename: /tmp/tmpQx1FWf/1a9675cd-645d-75b1-53424ba7-31482247.dmp
13:00:24 INFO - Operating system: Android
13:00:24 INFO - 0.0.0 Linux 2.6.29-g41a03df #22 Thu Jun 26 10:59:09 CST 2014 armv7l Android/full/generic:4.0.4.0.4.0.4/OPENMASTER/eng.cltbld.20150824.100905:eng/test-keys
13:00:24 INFO - CPU: arm
13:00:24 INFO - ARMv0
13:00:24 INFO - 0 CPUs
13:00:24 INFO - Crash reason: SIGSEGV
13:00:24 INFO - Crash address: 0x855ed241
13:00:24 INFO - Process uptime: not available
13:00:24 INFO - Thread 44 (crashed)
13:00:24 INFO - 0 libnss3.so!PR_GetThreadPrivate [prtpd.c:3f9990170a1a : 205 + 0x0]
13:00:24 INFO - r0 = 0x855ed21d r1 = 0x05328e35 r2 = 0xffffff38 r3 = 0x4857ff00
13:00:24 INFO - r4 = 0x00000001 r5 = 0x49cf244c r6 = 0x4857f8cc r7 = 0x49cf2488
13:00:24 INFO - r8 = 0x00000000 r9 = 0x00000000 r10 = 0x0000283b r12 = 0x407d1e08
13:00:24 INFO - fp = 0x00000000 sp = 0x4857f868 lr = 0x4076b121 pc = 0x4076c764
13:00:24 INFO - Found by: given as instruction pointer in context
13:00:24 INFO - 1 libxul.so!mozilla::BlockingResourceBase::Release [BlockingResourceBase.cpp:3f9990170a1a : 345 + 0x3]
13:00:24 INFO - r4 = 0x49cf244c r5 = 0x49cf244c r6 = 0x4857f8cc r7 = 0x49cf2488
13:00:24 INFO - r8 = 0x00000000 r9 = 0x00000000 r10 = 0x0000283b fp = 0x00000000
13:00:24 INFO - sp = 0x4857f870 pc = 0x40bf3489
13:00:24 INFO - Found by: call frame info
13:00:24 INFO - 2 libxul.so!mozilla::OffTheBooksMutex::Unlock [BlockingResourceBase.cpp:3f9990170a1a : 389 + 0x3]
13:00:24 INFO - r0 = 0x49cf244c r1 = 0x05328e35 r4 = 0x49cf244c r5 = 0x49cf2440
13:00:24 INFO - r6 = 0x4857f8cc r7 = 0x49cf2488 r8 = 0x00000000 r9 = 0x00000000
13:00:24 INFO - r10 = 0x0000283b fp = 0x00000000 sp = 0x4857f888 pc = 0x40bf357d
13:00:24 INFO - Found by: call frame info
13:00:24 INFO - 3 libxul.so!mozilla::CDMCaps::AutoLock::~AutoLock [Monitor.h : 36 + 0x5]
13:00:24 INFO - r0 = 0x49cf244c r1 = 0x05328e35 r4 = 0x4857f8d4 r5 = 0x49cf2440
13:00:24 INFO - r6 = 0x4857f8cc r7 = 0x49cf2488 r8 = 0x00000000 r9 = 0x00000000
13:00:24 INFO - r10 = 0x0000283b fp = 0x00000000 sp = 0x4857f898 pc = 0x40bdcdc3
13:00:24 INFO - Found by: call frame info
13:00:24 INFO - 4 libxul.so!mozilla::FileBlockCache::Read [FileBlockCache.cpp:3f9990170a1a : 290 + 0x5]
13:00:24 INFO - r4 = 0x000057c5 r5 = 0x49cf2440 r6 = 0x4857f8cc r7 = 0x49cf2488
13:00:24 INFO - r8 = 0x00000000 r9 = 0x00000000 r10 = 0x0000283b fp = 0x00000000
13:00:24 INFO - sp = 0x4857f8a0 pc = 0x41902555
13:00:24 INFO - Found by: call frame info
13:00:24 INFO - 5 libxul.so!mozilla::MediaCacheStream::Read [MediaCache.cpp:3f9990170a1a : 2298 + 0x15]
13:00:24 INFO - r4 = 0x0002a83b r5 = 0x00000000 r6 = 0x00000001 r7 = 0x471f46a0
13:00:24 INFO - r8 = 0x46b5c8e0 r9 = 0x4857fa6f r10 = 0x00000005 fp = 0x4411669c
13:00:24 INFO - sp = 0x4857f900 pc = 0x4191f959
13:00:24 INFO - Found by: call frame info
13:00:24 INFO - 6 libxul.so!mozilla::MediaCacheStream::ReadAt [MediaCache.cpp:3f9990170a1a : 2329 + 0x9]
13:00:24 INFO - r4 = 0x0002a83b r5 = 0x00000000 r6 = 0x4857f984 r7 = 0x46b5c8e0
13:00:24 INFO - r8 = 0x4857fa3c r9 = 0x4857fa6f r10 = 0x48a63408 fp = 0x00000000
13:00:24 INFO - sp = 0x4857f978 pc = 0x4191faef
13:00:24 INFO - Found by: call frame info
13:00:24 INFO - 7 libxul.so!mozilla::ChannelMediaResource::ReadAt [MediaResource.cpp:3f9990170a1a : 728 + 0x15]
13:00:24 INFO - r4 = 0x0002a83b r5 = 0x00000000 r6 = 0x46b5c800 r7 = 0x4857f9e4
13:00:24 INFO - r8 = 0x4857fa3c r9 = 0x4857fa6f r10 = 0x48a63408 fp = 0x00000000
13:00:24 INFO - sp = 0x4857f9a0 pc = 0x4191fb4d
13:00:24 INFO - Found by: call frame info
13:00:24 INFO - 8 libxul.so!mozilla::MediaResourceIndex::ReadAt [MediaResource.cpp:3f9990170a1a : 1752 + 0x15]
13:00:24 INFO - r0 = 0x4857fa6f r1 = 0xffff66b0 r2 = 0x4857f9e4 r3 = 0x00000000
13:00:24 INFO - r4 = 0x0002a83b r5 = 0x00000000 r6 = 0xffff66b0 r7 = 0x4857fa6f
13:00:24 INFO - r8 = 0x4857fa3c r9 = 0x4857fa6f r10 = 0x48a63408 fp = 0x00000000
13:00:24 INFO - sp = 0x4857f9c8 pc = 0x41911c05
13:00:24 INFO - Found by: call frame info
Fallout from bug 1196112?
Summary: Intermittent test_seek-10.html | application crashed [@ PR_GetThreadPrivate] → Intermittent test_seek-5.html,test_seek-10.html | application crashed [@ PR_GetThreadPrivate]
Strike two. Backing out bug 1196112.
(In reply to Treeherder Robot from comment #6)

Dammit.
Maybe bug 1195632 is a more-likely candidate?
Flags: needinfo?(jwwang)
I don't think so. Both bug 1195632 and bug 1196112 are about capture stream which is a code path that test_seek-10.html will not exercise.
Flags: needinfo?(jwwang)
revision b69b2c0188e6: including bug 1196112.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b69b2c0188e6
39 runs for M15 and no crash so bug 1196112 should be clean.

resision 4e97250ab1b1: including bug 1196112 and bug 1195073 p1 ~ p7
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4e97250ab1b1
50 runs for M15 and 2 crashes.
https://treeherder.mozilla.org/logviewer.html#?job_id=10705606&repo=try
https://treeherder.mozilla.org/logviewer.html#?job_id=10712724&repo=try

It looks like bug 1195073 is response for this bug.
Flags: needinfo?(jyavenard)
webmdemux_read is shown in both crash stacks and bug 1195073 is about webm changes. Bug 1195073 is suspicious.
Will have a look, but your try test for b69b2c0188e6 is no proof. You're testing different base revision, and had a different number of runs.

I don't see how bug 1195073 could make a difference whatsoever; if you look how read is called, it's identical to before and after 1195073 ; the only effective difference is when called from MSE we may read less data.
For files, Read() will be called with exactly the same argument as before.

The issue seems to be about accessing sResourceAcqnChainFrontTPI static in BlockingResourceBase::Release ; it appears to be null.

sounds like a coincidence / timing change.
Flags: needinfo?(jyavenard)
Good test would be to run identical branches, one with "Bug 1195073: [MSE/webm] P4. Limit nestegg reads to the last block's boundaries. r=kinetik" in and one without.
As this commit is the only one affecting webm_read
Flags: needinfo?(jyavenard)
b69b2c0188e6 is an empty changeset to include try syntax. It is effectively the same as its parent which is b2eb913e58c9, the 2nd patch of bug 1196112.

b2eb913e58c9 is also the parent of 3e8526f57706 which is the 1st patch of bug 1195073.

Therefore, the 2 try pushes only differ in the patches of bug 1195073.
Summary: Intermittent test_seek-5.html,test_seek-10.html | application crashed [@ PR_GetThreadPrivate] → Intermittent test_bug495300.html,test_seek-5.html,test_seek-10.html | application crashed [@ PR_GetThreadPrivate]
Assignee: nobody → jyavenard
Fairly sure this is what the problem is.

WebMBufferedParser::GetLastBlockOffset() is only updated upon NotifyDataArrived callback. Which will only happen once download started.
It is possible that nestegg managed to read the entire file before NotifyDataArrived was called. If that's the case the MediaResource::Tell() could very well be ahead of what GetEndDataOffset() would now return. When that happen we would have an underflow when calculating length-position ; and we would read much more data than requested by nestegg ; causing an out of bound memory write. This obviously corrupted the memory and the mutex used by the MediaCache.

This situation would typically happen on slower machine that would have time to download the whole file, parse it and NotifyDataArrived got delayed for whatever reason.
Attachment #8653185 - Flags: review?(kinetik)
https://hg.mozilla.org/try/rev/234d0688f31e
Flags: needinfo?(jyavenard)
Attachment #8653185 - Flags: review?(kinetik) → review+
https://hg.mozilla.org/mozilla-central/rev/bf2d5ba56975
Status: NEW → RESOLVED
Closed: 9 years ago
status-firefox43: affected → fixed
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → mozilla43
Relying on the data being fully buffered or not turned out to not be such a great idea.
Attachment #8653824 - Flags: review?(kinetik)
Comment on attachment 8653824 [details] [diff] [review]
[webm] P1. Explicitly differentiate WebM usage for mediasource.

Review of attachment 8653824 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/media/webm/WebMDemuxer.h
@@ +55,5 @@
>  public:
>    explicit WebMDemuxer(MediaResource* aResource);
> +  // Indicate if the WebMDemuxer is to be used with MediaSource. In which
> +  // case the demuxer will stop reads to the last known complete block.
> +  explicit WebMDemuxer(MediaResource* aResource, bool aIsMediaSource);

Don't need explicit with two arg ctors.  I think I'd just get rid of the single arg ctor and have a default aIsMediaSource = false here.  Not a big deal either way.
Attachment #8653824 - Flags: review?(kinetik) → review+
Attachment #8653825 - Flags: review?(kinetik) → review+
Blocks: 1197083
Backed out for a youtube playback regression. See Bug 1199573.
https://hg.mozilla.org/releases/mozilla-aurora/rev/5bb661db5c6c
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Closed: 9 years ago → 9 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: