871482 - [A/V] Multiple request for load or thumbnail extraction

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::Video, defect, P1)

Description

[leo.bugzilla.gecko]

When video application load video file and seek to get thumbnail, sometimes it requests more than one, simutaneously.
H/W codec doesn't support multi-instance.
So, several request for load and seek, is failed becuase of H/W codec limitation.

Failed video files are never listed up on video application

Comment 1

[Lukas Blakk [:lsblakk] use ?needinfo]

Blocking on the fact that we're not showing the user all their video files.  Also adding qawanted to look into the "sometimes it's requesting more than one" - what are the circumstances when this occurs?

Comment 2

[mlevin]

Issue reproduces on the following Leo:

Environmental  Variables:
Leo Build ID: 20130513070206
Gecko: http://hg.mozilla.org/releases/mozilla-b2g18/rev/50726fbdc25e
Gaia: a503d9a1d0a588f3689243c1ddc0f016ede51b9d

3 out of 13 video files did not appear on the video thumbs when opening the video app on the Leo.

2 of the 3 were .3gp (.5Mb and 1.5Mb) files and 1 was a .mp4 (52.6Mb)file. The other files that did appear were also .3gp files with varying sizes and an .mp4 files (40.8Mb).

Logcat attached titled: leo_video.txt

Comment 3

[mlevin]

Attachment: Leo Video App Not Showing All Thumbs

Comment 4

[leo.bugzilla.gecko]

Video application try to load another file before previous one is cleaned up.
I think it comes from mediaDB.

Comment 5

[David Flanagan [:djf]]

(In reply to leo.bugzilla.gecko from comment #0)
> When video application load video file and seek to get thumbnail, sometimes
> it requests more than one, simutaneously.

How do you know that this is what is happening? If you've done some diagnosis of the bug to determine this, please share your findings.  What line numbers, for example?

In order to debug this, I think we need the set of videos that you are testing with. Could you attach a link to the videos?  (On dropbox, maybe?)

There are many messages in the logcat output from gecko about a corrupt video file.  Were you intentionally testing with unplayable video files?

There are no JavaScript errors or warnings in the logcat, so it is not obvious to me that this is a Video app or Gaia bug, but it could be.

Mark: if this bug turns out to be related to apps/video/js/metadata.js or to shared/js/mediadb.js, I'd like to take it from you.  Feel free to do some preliminary investigation yourself if you'd like, or just reassign it to me.

Comment 6

[David Flanagan [:djf]]

(In reply to leo.bugzilla.gecko from comment #4)
> Video application try to load another file before previous one is cleaned up.
> I think it comes from mediaDB.

Its unlikely to be related to MediaDB. A relatively recent change (on v1-train) to the video app makes the video app responsible for doing metadata parsing and for correctly serializing it so that only one video is being scanned at a time. If the reporter is correct that the app is trying to get metadata for two videos at once, the failure is almost certainly happening in apps/video/js/metadata.js

But we really need the test videos so we can reproduce. Or, if reproducing this bug requires things like zero-length video files, then we need to know that.

Comment 7

[David Flanagan [:djf]]

mlevin: how can we get copies of the videos you used to reproduce this bug?

Comment 8

[mlevin]

(In reply to David Flanagan [:djf] from comment #7)
> mlevin: how can we get copies of the videos you used to reproduce this bug?

I can zip them and e-mail you the zipped file. It is 192Mb in size. Please let me know.

Comment 9

[David Flanagan [:djf]]

How about sending just the 3 video files that fail to appear in the Video app instead of the entire batch.

Comment 10

[mlevin]

(In reply to David Flanagan [:djf] from comment #9)
> How about sending just the 3 video files that fail to appear in the Video
> app instead of the entire batch.

Done via e-mail to you. If you have issues receiving please let me know.

-Matt.

Comment 11

[David Flanagan [:djf]]

Mark,

I now have copies of the test videos from comment 2, so I'm taking this bug for myself, at least to investigate whether it is Gaia-related.

Comment 12

[leo.bugzilla.gecko]

(In reply to David Flanagan [:djf] from comment #5)
> How do you know that this is what is happening? If you've done some
> diagnosis of the bug to determine this, please share your findings.  What
> line numbers, for example?

I found that some video files are missing on video list.
And I also found that OMXCodec cannot create H/W codec instance because someone other already held H/W codec.
So it fails to load and show error like below.
01-06 01:41:30.649 E       863      GeckoConsole                                 Content JS WARN at app://video.gaiamobile.org/gaia_build_defer_index.js:212 in metadataError: MediaDB: error parsing metadata for _ExternalSD/M007_3GP_MPEG4(SP@L1)_AAC-LC.3gp : M007 3GP MPEG4(SP@L1) AAC-LC 3gp
Then I saw multiple request from video application.

> In order to debug this, I think we need the set of videos that you are
> testing with. Could you attach a link to the videos?  (On dropbox, maybe?)
Test files that I tested is more than 4GB that include unsupported files also.
But i'm sure that this is not for unsupported file.

> There are many messages in the logcat output from gecko about a corrupt
> video file.  Were you intentionally testing with unplayable video files?
Missing files are list up well if SDcard has only one of those files.
(All video files are encoded by one of MPEG4, H.264, H.263, AAC, AMR, MP3 and extention is 3GP/MP4)

> There are no JavaScript errors or warnings in the logcat, so it is not
> obvious to me that this is a Video app or Gaia bug, but it could be.
I add some log in video.js
In normal case, sequence is like below in MetadataParser() in video.js.
load file -> createThumbnail -> doneSeeking -> captureFrame -> unload file
But in error case,
load file -> createThumbnail -> unload file 2 -> .......
Then second request always fails.

> Mark: if this bug turns out to be related to apps/video/js/metadata.js or to
> shared/js/mediadb.js, I'd like to take it from you.  Feel free to do some
> preliminary investigation yourself if you'd like, or just reassign it to me.

Comment 13

[leo.bugzilla.gecko]

> Its unlikely to be related to MediaDB. A relatively recent change (on
> v1-train) to the video app makes the video app responsible for doing
> metadata parsing and for correctly serializing it so that only one video is
> being scanned at a time. If the reporter is correct that the app is trying
> to get metadata for two videos at once, the failure is almost certainly
> happening in apps/video/js/metadata.js

Version which I checked this issue, doesn't have metadata.js in video app.
I will check with latest version and inform you

Comment 14

[leo.bugzilla.gecko]

(In reply to David Flanagan [:djf] from comment #11)
> Mark,
> 
> I now have copies of the test videos from comment 2, so I'm taking this bug
> for myself, at least to investigate whether it is Gaia-related.

I found two issues.

[1] multiple reqeust from video application, that I mentioned already.
In this case, 
load file -> createThumbnail -> unload file 2 -> ....

[2] After video unload file, it request another one.
load file -> createThumbnail -> doneSeeking -> captureFrame -> unload file -> load file2
At the time of load file2, somtimes, first one is not unloaded yet.
MediaElement takes a time to release OMXcodec resources.

Comment 15

[David Flanagan [:djf]]

(In reply to leo.bugzilla.gecko from comment #13)
> Version which I checked this issue, doesn't have metadata.js in video app.
> I will check with latest version and inform you

Thank you. Let me know if you can still reproduce it with the latest code.

Comment 16

[David Flanagan [:djf]]

(In reply to mlevin from comment #10)
> (In reply to David Flanagan [:djf] from comment #9)
> > How about sending just the 3 video files that fail to appear in the Video
> > app instead of the entire batch.
> 
> Done via e-mail to you. If you have issues receiving please let me know.
> 
> -Matt.

Matt: 

Of the three files you sent, the large mp4 file appears to just not be playable, at least on on my unagi. It does not load, even when it is the only video on the device (which means that its failure to play is not due to hardware contention as this bug report suggests.)

The other two files work fine for me.

I've just set the qawanted flag.  I guess that flag is for you. Could you check whether you can still reproduce this?  Are you testing on a Leo device?  Given that the reporter of this bug had out-of-date code in his Leo build is it possible that you have the same out-of-date code?

What I want to know is whether you're running with bug 856542 fixed. One way you can tell is to look at the title of a .3gp video, like the H.264 Sample.3gp file you sent me.  If the video app displays the ".3gp" extension as part of the video title, then you have out of date code.  If you it does not display that extension, then you're running with current code.

Comment 17

[mlevin]

(In reply to David Flanagan [:djf] from comment #16)
> (In reply to mlevin from comment #10)
> > (In reply to David Flanagan [:djf] from comment #9)
> > > How about sending just the 3 video files that fail to appear in the Video
> > > app instead of the entire batch.
> > 
> > Done via e-mail to you. If you have issues receiving please let me know.
> > 
> > -Matt.
> 
> Matt: 
> 
> Of the three files you sent, the large mp4 file appears to just not be
> playable, at least on on my unagi. It does not load, even when it is the
> only video on the device (which means that its failure to play is not due to
> hardware contention as this bug report suggests.)
> 
> The other two files work fine for me.
> 
> I've just set the qawanted flag.  I guess that flag is for you. Could you
> check whether you can still reproduce this?  Are you testing on a Leo
> device?

As per comment 2, yes a Leo device.

  Given that the reporter of this bug had out-of-date code in his Leo
> build is it possible that you have the same out-of-date code?
> 
> What I want to know is whether you're running with bug 856542 fixed. One way
> you can tell is to look at the title of a .3gp video, like the H.264
> Sample.3gp file you sent me.  If the video app displays the ".3gp" extension
> as part of the video title, then you have out of date code.  If you it does
> not display that extension, then you're running with current code.

David,

I re-tested on Unagi and Leo for you.

Unagi:
Mozilla RIL
Channel: Nightly
Unagi Build ID: 20130515070208
Gecko: http://hg.mozilla.org/releases/mozilla-b2g18/rev/d06cfe7d67c2
Gaia: 0ddb515f15cbc6b74fc2742b7599d6ae74c6413f

Result: 
All 3 video thumbnails appeared without the extensions. The video app continually shows the thin horizontal load app bar and never goes away. None of the 3 thumbnails open the video upon tapping them, it just stays at the thumbnail display.

Leo:
Mozilla RIL
Channel: Nightly
Leo Build ID: 20130515070208
Gecko: http://hg.mozilla.org/releases/mozilla-b2g18/rev/d06cfe7d67c2
Gaia: 0ddb515f15cbc6b74fc2742b7599d6ae74c6413f

Result: 
The only video thumbnail showing is the one titled "Arun" (the .mp4 file), the other 2 (the "sample" .3gp videos) do not show up. All extensions are not part of the thumbnail titles. The video app continually shows the thin horizontal load app bar and never goes away. Upon tapping the thumbnail to open the video, it just stays at the thumbnail display and never opens.

Comment 18

[David Flanagan [:djf]]

Matt,

Since I couldn't duplicate with just the 3 files, I suppose you ought to share all of your videos with me.  Emailing a link to google docs worked well last time. Thanks!

In particular, the mp4 file you shared last time didn't have "Arun" in the filename, so maybe I don't have that one.

I'd also be interested to know if you have any messages in your logcat when you reproduce this bug.

The symptom you're seeing (the scanning indicator never goes away and videos don't open) are not things that were reported by the original poster, but they might be clues to the same underlying problem...

Comment 19

[David Flanagan [:djf]]

Thanks Matt. I've got Arun.mp4 now. If I still can't reproduce, I'll ask for the full set of 13 movies.

Comment 20

[mlevin]

David,

Please see attached logcat of the issue, titled "Leo_Video_app_issue_log.txt".

In addition I have e-mailed you today the shared zip files which I used for this test.

The Leo I used was the same one as in comment 17, same issue as well.

Matt

Comment 21

[mlevin]

Attachment: Leo_Video_app_issue_log.txt

Comment 22

[David Flanagan [:djf]]

Matt,

I'm using exactly the same 5/15 build as you were, on a unagi, using the three video files you sent, and it work fine for me. I cannot reproduce any of the issues you're seeing.

Since our hardware and software are the same, I have to assume that there is something different about the files on our sdcards. Please create another zip file of all the videos you've got on the sdcard and email me a link. The video app also has to scan any music files with .mp4 and .ogg extensions, so if you have any of those on the card, please include them as well.

It would also be helpful if you did this:

  adb shell ls -laR /sdcard/ > /tmp/list

And then sent me the /tmp/list file it creates.  That might help me see if you have other media files lurking anywhere in other directories of the sdcard.  (For example, if you've been testing bluetooth transfer of videos, there might be files in sdcard/downloads/bluetooth).

Comment 23

[mlevin]

(In reply to David Flanagan [:djf] from comment #22)
> Matt,
> 
> I'm using exactly the same 5/15 build as you were, on a unagi, using the
> three video files you sent, and it work fine for me. I cannot reproduce any
> of the issues you're seeing.
> 
> Since our hardware and software are the same, I have to assume that there is
> something different about the files on our sdcards. Please create another
> zip file of all the videos you've got on the sdcard and email me a link. The
> video app also has to scan any music files with .mp4 and .ogg extensions, so
> if you have any of those on the card, please include them as well.
> 
> It would also be helpful if you did this:
> 
>   adb shell ls -laR /sdcard/ > /tmp/list
> 
> And then sent me the /tmp/list file it creates.  That might help me see if
> you have other media files lurking anywhere in other directories of the
> sdcard.  (For example, if you've been testing bluetooth transfer of videos,
> there might be files in sdcard/downloads/bluetooth).

David,

Go to https://drive.google.com/a/qanalydocs.com/?tab=mo#my-drive and see the file All_Media_On_SD.zip. That has all the video media on my sd card.

Also see attached document called list generated from the adb command in your comment 22.

Unagi used for this test was same as in comment 17. Issue reproduced.

Comment 24

[mlevin]

Attachment: List

Comment 25

[David Flanagan [:djf]]

Thanks Matt.  I can reproduce now!

Comment 26

[David Flanagan [:djf]]

Matt,

The VID_0008.3gp file you sent is the one that is causing the problems. When I play it in desktop firefox or quicktime, ti is just solid green with gurgling sounds.

When the video app tries to scan it on the phone, gecko just can't process it, but doesn't report an error.  I'll report a gecko bug on that and try to create a workaround in gaia.  It might help, however to know more about where the file came from.  Is this a file provided by Leo, or is it a video you recorded yourself like the other videos you sent?

Comment 27

[mlevin]

(In reply to David Flanagan [:djf] from comment #26)
> Matt,
> 
> The VID_0008.3gp file you sent is the one that is causing the problems. When
> I play it in desktop firefox or quicktime, ti is just solid green with
> gurgling sounds.
> 
> When the video app tries to scan it on the phone, gecko just can't process
> it, but doesn't report an error.  I'll report a gecko bug on that and try to
> create a workaround in gaia.  It might help, however to know more about
> where the file came from.  Is this a file provided by Leo, or is it a video
> you recorded yourself like the other videos you sent?

David,
Good find. I took out the sd card and copied that file to my laptop and tried playing it, but it would not play. It must be a corrupted file. From the file name it might be a video file recorded on one of 3 devices I have: Inari, Leo and Unagi. Also it was in the "bluetooth" folder on the sd card, so it might've been an issue when downloaded via bluetooth transfer. I deleted the file using file explorer with the sdcard attached to my laptop via USB. Issue does not reproduce now.

Comment 28

[David Flanagan [:djf]]

Matt,

I'm glad to hear that you can not reproduce the issue when you remove the corrupt file.  We still need to defend against this kind of metadata hang for corrupt files, so there is still something to fix here.

But I'm now also left not knowing whether this is the same issue that Leo originally reported.

Leo: would you attach a logcat output? Would you also try to give more precise steps to reproduce?  Does this happen often?  Do I need many videos on the phone to see it, or does it occur all the time? I should be getting a Leo device to try this on. But I also need to know what build you are using when you see the bug.

Comment 29

[cajbir (:cajbir)]

(In reply to leo.bugzilla.gecko from comment #0)
> 
> Failed video files are never listed up on video application

I've also seen this and raised bug 869289 about it.

Comment 30

[leo.bugzilla.gecko]

Attachment: logs by leo device

Comment 31

[leo.bugzilla.gecko]

(In reply to David Flanagan [:djf] from comment #28)
> Leo: would you attach a logcat output? Would you also try to give more
> precise steps to reproduce?  Does this happen often?  Do I need many videos
> on the phone to see it, or does it occur all the time? I should be getting a
> Leo device to try this on. But I also need to know what build you are using
> when you see the bug.
I attached logcat output when this problem occurred. (There was many files in sdcard)

But...last week, Patch for Bug 856542 is merged on LG side.
And I could not see this problem after the patch. (Attached logcat is by previous version)

Could you please confirm that this problem is solved by that patch?

And one more question...
After the patch, video application seems to have performace issue.
I hardly scroll down/up the video list.
There was more than 45 video files in the list.

Comment 32

[David Flanagan [:djf]]

(In reply to leo.bugzilla.gecko from comment #31)

> But...last week, Patch for Bug 856542 is merged on LG side.
> And I could not see this problem after the patch. (Attached logcat is by
> previous version)
> 
> Could you please confirm that this problem is solved by that patch?
>

Yes, that's the patch I was talking about back in comment #6.
 
> And one more question...
> After the patch, video application seems to have performace issue.
> I hardly scroll down/up the video list.
> There was more than 45 video files in the list.

If you have a lot of videos, the app is busy generating thumbnails for them and this is expected to impact performance.  Maybe there is something we can do to make it more responsive, but please file a new bug about that.

I'm going to repurpose this bug to work around the hanging on corrupt files issue
accidentally discovered by Matt.

Comment 33

[David Flanagan [:djf]]

Per comment 31, this bug is was actually invalid, and was reported using a build that did not have the patch from bug 856542 applied.  I'm only keeping it open because in the process of diagnosing it we found another bug.  (The other bug only occurs with strangely corrupt video files that are playable but not seekable.) The fix for this should be easy.

Comment 34

[David Flanagan [:djf]]

Attachment: link to patch on github

This patch fixes the hang that we get when we have a corrupt file that we get a metadataloaded event for, but that we can not seek.

I'll attach a corrupt video so you can reproduce the bug.

Comment 35

[David Flanagan [:djf]]

Attachment: corrupt video file for reproducing the bug

Comment 36

[David Flanagan [:djf]]

Comment on attachment 755006 [details]
link to patch on github

Leo,

Would you check that the attached patch fixes the bug?  The bug I see is that if the attached video file is on the sdcard, the video app will get stuck on that video and will scan forever.  The patch fixes this for me.

Dominic has looked at the code, but could not reproduce the bug on his hardware (so it is possible that this is something that was fixed recently in gecko)

Comment 37

[Dominic Kuo [:dkuo]]

David,

I was not able to reproduce this issue with the corrupt video you attached on unagi, with latest b2g18 gecko and gaia master. As we discussed offline, maybe gecko has fixed it recently. But in case there are some other corrupt videos that gecko cannot process, this patch can prevent the Video app from hanging forever.

Comment 38

[David Flanagan [:djf]]

I find the same thing as Dominic does in comment 37: with a current gecko build (on Unagi and Leo devices) the hang while seeking in the corrupt file does not occur anymore.  Instead, we get an exception when trying to draw the frame from the corrupt video, and we already have code to handle that.

We're still left with code in gaia currently that can hang if a video does not seek. There is no current known bug that causes that, but it might be safer to land the patch.

Comment 39

[David Flanagan [:djf]]

Landed on master (with review by dkuo): https://github.com/mozilla-b2g/gaia/commit/ea49d484fa0f7dbb263fb3ae24a4befa232d2474

Comment 40

[Dominic Kuo [:dkuo]]

Comment on attachment 755006 [details]
link to patch on github

David,

As I mentioned in comment 37, this patch can prevent the Video app from hanging forever, so I am r+ ing this patch.

Comment 41

[John Ford [:jhford] CET/CEST Berlin Time]

Uplifted ea49d484fa0f7dbb263fb3ae24a4befa232d2474 to:
v1-train: a8ba47d6e44fbf4ce1b4219aece3fdc8284082dc

Comment 42

[John Hammink]

Removing NI as there is now a Moztrap case for this.