837051 - [video]some mpeg-4 format videos play abnormal.( the image of some videos is static when playing)

Status: VERIFIED FIXED

(Firefox OS Graveyard :: General, defect)

Description

[Firefox_Mozilla]

Attachment: the image of video is static

Steps to reproduce:
1.save some mpeg-4 format videos in TF card.
2.insert this TF card and go into videos;
3.select the mpeg-4 format video to play;
Expected results:
1.the video plays normally;
Actual results:
1.some mpeg-4 format videos play abnormal. the image of the video is static when playing.

Comment 1

[Firefox_Mozilla]

Attachment: the video play normally.

Comment 2

[Chiajung Hung [:chiajung]]

As I know, some Android devices does have similar issue since the hardware can not support some specific mpeg4 profile. Can you test the video can not play on the same device with Android?

Comment 3

[Firefox_Mozilla]

the video which can not be played on Firefox OS can be play normal in Android OS.

Comment 4

[Al Tsai [:atsai]]

Chiajung, can you check it's limited by the decoder we use or it's a bug?
Thanks.

Comment 5

[Chiajung Hung [:chiajung]]

(In reply to Al Tsai [:atsai] from comment #4)
> Chiajung, can you check it's limited by the decoder we use or it's a bug?
> Thanks.

If I can get any video has the problem I can check it later. But I have no such video in my hand...
Can you provide some?

Comment 6

[William Hsu [:whsu]]

Attachment: Video test sample

Comment 7

[William Hsu [:whsu]]

Hi, all,

Thanks for your help and information!
I have double confirmed this case.
The build: 20130218070202 (Have the same problem)
The build: 20130304230203 (The issue cannot be reproduced)

This case work for me with latest build and my test sample(Attached). So, close it.
If the issue happens again, please reopen it.
Thanks!

Comment 8

[Michael Vines [:m1] [:evilmachines]]

(tef- as no build information is provided in the description and comment 7 cannot reproduce)

Comment 9

[David Palomino [:dpalomino]]

Hi, 

According comment#7, RESOLVED-WORKSFORME. If you think it should be reopen, please proceed.

thx!
David

Comment 10

[Juan Perez-Bedmar [:juanpbf]]

Attachment: Smurf Test video

Hi all, 

In my sample, with build 20130304203846, I have played 3 different videos:

A) with the video from attachment 708965 [details] and I have experienced the problem described in the description of the bug. I will try to explain current behaviour better:

1- When you play the video it is normally viewed and heard, during few seconds (5 or 6).
2- After few seconds, sound and image are frozen for 2 or 3 seconds.
3- After those 2-3 seconds, the sound is resumed, but image is not.


At this point the image remains frozen unless you proceed as follows:

4- If you click in "pause", the music is paused (the image remains frozen)
5- Once you click again in "play", music is resumed and image is also resumed (and accelerated until it reaches the current reproduction point). After some seconds, behaviour becames again exactly the same as described in steps 2 and 3.


B) When playing video attached in comment #7, I don't see this issue, but it is too short, so I'm cannot be sure if the issue is present also with this video

C) When playing video attached in this comment, I don't see this issue. 


What do you think? could this be caused by a problem with the video de-coding? could be related to a RAM memory issue?

Would you please mind to check behaviour of the first video in other FFOS-based device, different from ours?

Bug reopened until clarification.

Thx!!

Comment 11

[Firefox_Mozilla]

It can see the reply of comment 10.

Comment 12

[Michael Vines [:m1] [:evilmachines]]

(Inder, can you please take a look at this from the decoder pov?)

Comment 13

[Andrew Overholt [:overholt]]

How are things going here?

Comment 14

[Sotaro Ikeda [:sotaro]]

I checked a playback of attachment 708965 [details] on unagi. When I tried to seek the video, video rendering update is not smooth.

Even during the payback, OMXCodec::read() took from 70ms-100ms. Even during playback, buffers of OMXCodec are always almost starved. Normally OMXCodec::read() takes less than 1ms at other videos.

And when seeking, OMXCodec::read()s called before and after nsMediaOmxReader::Seek() failed with following log. It says that OMXCodec::read() took 3sec then failed. But the log says OMXCodec owned 5 output bufers at the time. It seems strange. In normal video playback, 5 buffers might be enough to output videos.

> E OMXCodec: [OMX.qcom.video.decoder.mpeg4] Timed out waiting for output buffers: 0/5

Comment 15

[Sotaro Ikeda [:sotaro]]

Attachment: seek log of attachment 708965

Log out during seek of attachment 708965 [details]. Almost all log are added by manually.

Comment 16

[Andrew Overholt [:overholt]]

I spoke with Inder via email and he hasn't had a chance to investigate here but is planning on doing so soon.

Comment 17

[Inder]

The issue is indeed happening due to OMXCodec running out of output buffers. I am not sure why we are running out of output buffers though. Still investigating. On Android with the same OMXCodec implementation we don't see any issues.

Comment 18

[Sotaro Ikeda [:sotaro]]

Attachment: patch for b2g18 - store less buffers for video playback at nsMediaOmxReader

The patch reduces two buffers stored around nsMediaOmxReader.

Comment 19

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #18)
> Created attachment 732969 [details] [diff] [review]
> patch for b2g18 - store less buffers for video playback at nsMediaOmxReader
> 
> The patch reduces two buffers stored around nsMediaOmxReader.

By applying the patch, I confirmed that attachment 708965 [details] can be seeked as normal on unagi.

Comment 20

[Sotaro Ikeda [:sotaro]]

Comment on attachment 732969 [details] [diff] [review]
patch for b2g18 - store less buffers for video playback at nsMediaOmxReader

:doublec, can you review the patch?
I confirmed the patch works on unagi. The patch reduces 2 buffers stored around nsMediaOmxReader. Reducing 1 buffer did not work. Reducing 2 buffers are necessary.

Comment 21

[cajbir (:cajbir)]

(In reply to Sotaro Ikeda [:sotaro] from comment #20)
> Comment on attachment 732969 [details] [diff] [review]
> patch for b2g18 - store less buffers for video playback at nsMediaOmxReader
> 
> :doublec, can you review the patch?
> I confirmed the patch works on unagi. The patch reduces 2 buffers stored
> around nsMediaOmxReader. Reducing 1 buffer did not work. Reducing 2 buffers
> are necessary.

This patch seems to do more than just change the buffer size. Was that intended?

Comment 22

[cajbir (:cajbir)]

I've removed review for now until I get confirmation about the additional things in the patch from Sotaro.

Comment 23

[Sotaro Ikeda [:sotaro]]

(In reply to Chris Double (:doublec) from comment #21)
> (In reply to Sotaro Ikeda [:sotaro] from comment #20)
> > :doublec, can you review the patch?
> > I confirmed the patch works on unagi. The patch reduces 2 buffers stored
> > around nsMediaOmxReader. Reducing 1 buffer did not work. Reducing 2 buffers
> > are necessary.
> 
> This patch seems to do more than just change the buffer size. Was that
> intended?

Yes, it was intended. Change only GetAmpleVideoFrames() to 1 is not enough to fix the problem. mLastVideoFrame is one additional buffer. By remove the mLastVideoFrame, we can remove one more buffer. Total reduced buffer count becomes 2. Then I confirmed the problem was addressed on unagi.

Comment 24

[Inder]

I could verify that the patch does fix the seek issue we were seeing.
Chris/Sotaro: If there are no side effects of this change then please review and commit it as soon as we can.

Comment 25

[Sotaro Ikeda [:sotaro]]

(In reply to Inder from comment #24)
> I could verify that the patch does fix the seek issue we were seeing.
> Chris/Sotaro: If there are no side effects of this change then please review
> and commit it as soon as we can.

doublec, can you review the patch soon?

Comment 26

[zhang.yonglin]

we need this patch immediately, pls confirm when this patch OK? Thanks

Comment 27

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

Does not sounds like a Gaia bug. Moving to General to reflect this.

Comment 28

[Sotaro Ikeda [:sotaro]]

Attachment: parch v2 - store less buffers for video playback at MediaOmxReader

commitable pach. Carry "chris.double: review+"

Comment 29

[Sotaro Ikeda [:sotaro]]

Comment on attachment 734640 [details] [diff] [review]
parch v2 - store less buffers for video playback at MediaOmxReader

Carry "chris.double: review+"

Comment 30

[Sotaro Ikeda [:sotaro]]

Attachment: parch v2 for b2g18 - store less buffers for video playback at MediaOmxReader

commitable pach for b2g18. Carry "chris.double: review+"

Comment 31

[Sotaro Ikeda [:sotaro]]

https://tbpl.mozilla.org/?tree=Try&rev=fef6d0a068bf

Comment 32

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a65e8af95d39

Comment 33

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/a65e8af95d39

Comment 34

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-b2g18/rev/6deb867802b9
https://hg.mozilla.org/releases/mozilla-b2g18_v1_0_1/rev/f56b48e52980

Comment 35

[Kai-Chih Hu [:khu]]

Reporter said this issue could be reproduced with latest codes. Re-open this case.

Comment 36

[Sotaro Ikeda [:sotaro]]

(In reply to khu from comment #35)
> Reporter said this issue could be reproduced with latest codes. Re-open this
> case.

Which hardware is used for it? I use only unagi.

Comment 37

[mlevin]

Issue reproduces:
During normal playback at the start of the video the audio appears to be only in sync with the video for a while, then it is not in sync. In addition if the User slides the bar ahead then lets go, the audio is heard, but the video is static.

Tested on the following Leo environment:

Leo Build ID: 20130412070204
Kernel Date: Mar 15
Gecko: http://hg.mozilla.org/releases/mozilla-b2g18/rev/950db80b82cb
Gaia: 54ab4ce3f9107d1624c18bed2a09ace6bbc6601f

Comment 38

[Sotaro Ikeda [:sotaro]]

(In reply to mlevin from comment #37)
> Issue reproduces:
> During normal playback at the start of the video the audio appears to be
> only in sync with the video for a while, then it is not in sync. In addition
> if the User slides the bar ahead then lets go, the audio is heard, but the
> video is static.
> 
> Tested on the following Leo environment:
> 
so, is it leo device?

Comment 39

[mlevin]

I tested on Leo as per the build ID in comment 37:

"Leo Build ID: 20130412070204"

Comment 40

[Sotaro Ikeda [:sotaro]]

Can you confirm on a device for tef? Because it is a tef+ bug. I assume it happens because of buffer starvation on the leo device. Each device uses different number of buffer for video decoding. It is a device dependent thing.

Comment 41

[mlevin]

(In reply to Sotaro Ikeda [:sotaro] from comment #40)
> Can you confirm on a device for tef? Because it is a tef+ bug. I assume it
> happens because of buffer starvation on the leo device. Each device uses
> different number of buffer for video decoding. It is a device dependent
> thing.

I took the same memory card out of the Leo device and placed it into the Unagi device.

Issue reproduces: The audio eventually gets out of sync with the video. In addition when I slide the bar to advance the video, the video app crashes then closes and then the home screen appears.

File tested both here and in comment 37: 40 MB MP4 video

Comment 42

[mlevin]

(In reply to mlevin from comment #41)
> (In reply to Sotaro Ikeda [:sotaro] from comment #40)
> > Can you confirm on a device for tef? Because it is a tef+ bug. I assume it
> > happens because of buffer starvation on the leo device. Each device uses
> > different number of buffer for video decoding. It is a device dependent
> > thing.
> 
> I took the same memory card out of the Leo device and placed it into the
> Unagi device.
> 
> Issue reproduces: The audio eventually gets out of sync with the video. In
> addition when I slide the bar to advance the video, the video app crashes
> then closes and then the home screen appears.
> 
> File tested both here and in comment 37: 40 MB MP4 video

Unagi build for this test:
Environmental  Variables:
Unagi Build ID: 20130411070205
Kernel Date: Dec 5
Gecko: http://hg.mozilla.org/releases/mozilla-b2g18/rev/f671fa539473
Gaia: e7e338a765e22334b40ced41489a785941382c66

Comment 43

[Sotaro Ikeda [:sotaro]]

(In reply to mlevin from comment #41)
> (In reply to Sotaro Ikeda [:sotaro] from comment #40)
> > Can you confirm on a device for tef? Because it is a tef+ bug. I assume it
> > happens because of buffer starvation on the leo device. Each device uses
> > different number of buffer for video decoding. It is a device dependent
> > thing.
> 
> I took the same memory card out of the Leo device and placed it into the
> Unagi device.
> 
> Issue reproduces: The audio eventually gets out of sync with the video. In
> addition when I slide the bar to advance the video, the video app crashes
> then closes and then the home screen appears.
> 
> File tested both here and in comment 37: 40 MB MP4 video

Thanks for the confirmation. I never see the crash on my phone. Can you attach the crash log?

Comment 44

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #43)
> (In reply to mlevin from comment #41)
> > (In reply to Sotaro Ikeda [:sotaro] from comment #40)
> > > Can you confirm on a device for tef? Because it is a tef+ bug. I assume it
> > > happens because of buffer starvation on the leo device. Each device uses
> > > different number of buffer for video decoding. It is a device dependent
> > > thing.
> > 
> > I took the same memory card out of the Leo device and placed it into the
> > Unagi device.
> > 
> > Issue reproduces: The audio eventually gets out of sync with the video. In
> > addition when I slide the bar to advance the video, the video app crashes
> > then closes and then the home screen appears.
> > 
> > File tested both here and in comment 37: 40 MB MP4 video
> 
> Thanks for the confirmation. I never see the crash on my phone. Can you
> attach the crash log?

and adb shell log when the crash happens. Thanks.

Comment 45

[mlevin]

(In reply to Sotaro Ikeda [:sotaro] from comment #44)
> (In reply to Sotaro Ikeda [:sotaro] from comment #43)
> > (In reply to mlevin from comment #41)
> > > (In reply to Sotaro Ikeda [:sotaro] from comment #40)
> > > > Can you confirm on a device for tef? Because it is a tef+ bug. I assume it
> > > > happens because of buffer starvation on the leo device. Each device uses
> > > > different number of buffer for video decoding. It is a device dependent
> > > > thing.
> > > 
> > > I took the same memory card out of the Leo device and placed it into the
> > > Unagi device.
> > > 
> > > Issue reproduces: The audio eventually gets out of sync with the video. In
> > > addition when I slide the bar to advance the video, the video app crashes
> > > then closes and then the home screen appears.
> > > 
> > > File tested both here and in comment 37: 40 MB MP4 video
> > 
> > Thanks for the confirmation. I never see the crash on my phone. Can you
> > attach the crash log?
> 
> and adb shell log when the crash happens. Thanks.

Crash Log Link: 
https://crash-stats.mozilla.com/report/index/fe9e4d6d-3d41-487e-a6a7-f4c452130412

Comment 46

[Sotaro Ikeda [:sotaro]]

Thanks for the crash log. The log does not have function information. It is not clear why it happens...

Comment 47

[Sotaro Ikeda [:sotaro]]

take the bug. I am actually analyzing.

Comment 48

[mlevin]

Attachment: Video crash logcat

Perhaps this attached logcat can shed some light.

Comment 49

[Sotaro Ikeda [:sotaro]]

(In reply to mlevin from comment #48)
> Created attachment 736968 [details]
> Video crash logcat
> 
> Perhaps this attached logcat can shed some light.

Thanks, did you use attachment 708965 [details] for playback?

Comment 50

[Sotaro Ikeda [:sotaro]]

(In reply to mlevin from comment #48)
> Created attachment 736968 [details]
> Video crash logcat
> 
> Perhaps this attached logcat can shed some light.

There no information why the app process crashed. From the log, there are following possibility.
- file is corrupted
- problem to read data from SDCARD

-------------------------------------------related log.

04-12 13:57:15.772: E/MPEG4Extractor(473): Video is malformed
04-12 13:57:15.772: W/OMXCodec(473): Ignore Corrupt NAL
04-12 13:57:15.772: E/MPEG4Extractor(473): Video is malformed
04-12 13:57:15.772: W/OMXCodec(473): Ignore Corrupt NAL
04-12 13:57:15.772: E/MPEG4Extractor(473): Video is malformed
04-12 13:57:15.772: W/OMXCodec(473): Ignore Corrupt NAL
04-12 13:57:15.772: E/MPEG4Extractor(473): Video is malformed
04-12 13:57:15.772: W/OMXCodec(473): Ignore Corrupt NAL

Comment 51

[Sotaro Ikeda [:sotaro]]

When I playback the attachment 708965 [details], I see no "Video is malformed" log.

Comment 52

[Tony Chung [:tchung]]

(In reply to Sotaro Ikeda [:sotaro] from comment #51)
> When I playback the attachment 708965 [details], I see no "Video is
> malformed" log.

sotaro, mlevin: can we also try the "smurf video" attachment example on the Leo device as well; and see if it's reproducible?   attach logs and crash reports as usual.  

Thanks

Comment 53

[Sotaro Ikeda [:sotaro]]

Attachment: logcat of Smurf video playback on sotaro's unagi phone

Comment 54

[Sotaro Ikeda [:sotaro]]

Attachment: logcat of attachment708965 on sotaro's unagi phone

Comment 55

[Sotaro Ikeda [:sotaro]]

(In reply to Tony Chung [:tchung] from comment #52)
> (In reply to Sotaro Ikeda [:sotaro] from comment #51)
> > When I playback the attachment 708965 [details], I see no "Video is
> > malformed" log.
> 
> sotaro, mlevin: can we also try the "smurf video" attachment example on the
> Leo device as well; and see if it's reproducible?   attach logs and crash
> reports as usual.
> 
> Thanks

I can play "smurf video" and attachment708965 [details] normally. attachment 737397 [details], attachment 737396 [details] are logcat of each playback.

Comment 56

[mlevin]

(In reply to Tony Chung [:tchung] from comment #52)
> (In reply to Sotaro Ikeda [:sotaro] from comment #51)
> > When I playback the attachment 708965 [details], I see no "Video is
> > malformed" log.
> 
> sotaro, mlevin: can we also try the "smurf video" attachment example on the
> Leo device as well; and see if it's reproducible?   attach logs and crash
> reports as usual.  
> 
> Thanks

sotaro, tchung:

Tested on same Leo environment as before:
Leo Build ID: 20130412070204
Kernel Date: Mar 15
Gecko: http://hg.mozilla.org/releases/mozilla-b2g18/rev/950db80b82cb
Gaia: 54ab4ce3f9107d1624c18bed2a09ace6bbc6601f

Result: 
1. No crashes for this Smurf video (filename: VID-20121226-WA0000)
2. Audio is ahead of video by a small amount, thus they are not in sync.


Logcat attached titled "Leo_4_12_13". 

Note: logcat cannot capture everything since video cannot be played while tethered to laptop. When video ended Leo was immediately tethered to laptop and logcat was then captured.

Comment 57

[mlevin]

Attachment: Logcat of Leo test with Smurf video

Comment 58

[mlevin]

Correction to Comment 56: 
Logcat titled: "Logcat of Leo test with Smurf video".

Comment 59

[Sotaro Ikeda [:sotaro]]

(In reply to mlevin from comment #57)
> Created attachment 737544 [details]
> Logcat of Leo test with Smurf video

From the log, it seems that there is no problem.

Comment 60

[Sotaro Ikeda [:sotaro]]

(In reply to mlevin from comment #56)
> (In reply to Tony Chung [:tchung] from comment #52)
> > (In reply to Sotaro Ikeda [:sotaro] from comment #51)
> > > When I playback the attachment 708965 [details], I see no "Video is
> > > malformed" log.
> > 
> > sotaro, mlevin: can we also try the "smurf video" attachment example on the
> > Leo device as well; and see if it's reproducible?   attach logs and crash
> > reports as usual.  
> > 
> > Thanks
> 
> sotaro, tchung:
> 
> Tested on same Leo environment as before:
> Leo Build ID: 20130412070204
> Kernel Date: Mar 15
> Gecko: http://hg.mozilla.org/releases/mozilla-b2g18/rev/950db80b82cb
> Gaia: 54ab4ce3f9107d1624c18bed2a09ace6bbc6601f
> 
> Result: 
> 1. No crashes for this Smurf video (filename: VID-20121226-WA0000)
> 2. Audio is ahead of video by a small amount, thus they are not in sync.
> 

[2] seems different problem from the bug. Audio output for Firefox OS v1.01 does not handle audio out latency in /gecko/media/libsydneyaudio/src/sydney_audio_gonk.cpp. It might affect to [2].

I think this bug could be closed.

Comment 61

[Daniel Coloma:dcoloma]

closing it, if we need a follow-up bug, let's open it separately

Comment 62

[leo.bugzilla.gecko]

leo device still has a problem after adapting attached patch.
Do I need to create another issue? or any one can follow up here?

Comment 63

[leo.bugzilla.gecko]

Sorry for confusing..
It seems like fixed in leo device, also.

Comment 64

[Sotaro Ikeda [:sotaro]]

FYI, attachment 735058 [details] in Bug 832653 could provide two more buffers to gecko. I am thinking to apply it after v1.1.

Comment 65

[Al Tsai [:atsai]]

verified according to comment 63

Comment 66

[leo.bugzilla.gecko]

When the video application retrieves thumbnail from the contents, same problem occur.
(At the time video application seek to total duration /4 position)

Thumbnail is show up becuase first 2~3 frames are rendered.
But it delays 3 seconds for every MPEG4 encoded files to be retrieved thumbnail.

Comment 67

[leo.bugzilla.gecko]

I want to reopen this issue because I think comment 66 is related to this issue.
If it's not, please close the issue then I will create another one.

Comment 68

[Jason Smith [:jsmith]]

(In reply to leo.bugzilla.gecko from comment #67)
> I want to reopen this issue because I think comment 66 is related to this
> issue.
> If it's not, please close the issue then I will create another one.

Please open another issue.

Comment 69

[Sotaro Ikeda [:sotaro]]

(In reply to leo.bugzilla.gecko from comment #67)
> I want to reopen this issue because I think comment 66 is related to this
> issue.
> If it's not, please close the issue then I will create another one.

This bug is not related to comment 66. It is a different problem.

Comment 70

[Sotaro Ikeda [:sotaro]]

GetAmpleVideoFrames() of 1 in attachment 734653 [details] [diff] [review] regressed video playback especially in Bug 863441. GetAmpleVideoFrames() needs to be more than 3. Just increase the number regress this problem. Instead Bug 864230 will fix the problem caused by increase of GetAmpleVideoFrames().