1147241 - [Video] Seek functionality via dragging progress bar does not update preview when seeking and is slow to resume after seeking

Status: RESOLVED FIXED

(Core :: Audio/Video, defect)

Description

[Pi Wei Cheng [:piwei] (inactive)]

Issue found while verifying bug 1133955.

STR:
1) On Flame 3.0, record a video of ~30 seconds
2) In Video app, start playing the video created at step 1
3) Drag the progress bar around, and let go of finger. Repeat this a few times.

Actual: While dragging the progress bar, the image preview does NOT update as dragging occurs. And after let go of finger, sometimes the video is slow to resume playing (takes ~3 seconds to resume).

Expected: The image preview updates as dragging occurs (so user knows where in the video they're seeking); and after finger leaves screen, the app continues to play the video without delay.

Device: Flame 3.0
BuildID: 20150324010202
Gaia: efebbafd12fc42ddcd378948b683a51106517660
Gecko: 840cfd5bc971
Gonk: b83fc73de7b64594cd74b33e498bf08332b5d87b
Version: 39.0a1 (3.0) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:39.0) Gecko/39.0 Firefox/39.0

Video and branch check will be added tomorrow.

Comment 1

[Pi Wei Cheng [:piwei] (inactive)]

Video:
https://www.youtube.com/watch?v=yRBYg9_2-oo

This issue occurs on 2.2.

Device: Flame 2.2
BuildID: 20150324002504
Gaia: 014d38f7ad3912b8b33cb08ce7535a5dc5aced59
Gecko: 7a9f2a248e57
Gonk: ebad7da532429a6f5efadc00bf6ad8a41288a429
Version: 37.0 (2.2) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0

--------

This issue does not occur on 2.1, however on 2.1 bug 1133955 occurs.

Device: Flame 2.1
BuildID: 20150325001202
Gaia: b8ae0df34362420fe4a9c90effa5247a1f5c844d
Gecko: 2a05cd42088b
Gonk: b83fc73de7b64594cd74b33e498bf08332b5d87b
Version: 34.0 (2.1) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0

------

Also this issue is affecting Gallery and Camera Preview app. It won't let me change Product/Component to Core/Video Audio for some reason.

Comment 2

[Pi Wei Cheng [:piwei] (inactive)]

Attachment: logcat of issue

Comment 3

[KTucker [:KTucker][Inactive 3/4/2016]]

This was written because of a failed verification. It was a blocker so nominating this one 2.2?

Comment 4

[Bobby Chien]

Can't be reproduced in this Flame-v2.2.

-----

Gaia-Rev        2d42a7c1d99472780e3aeb2e16d7cb0a4adbd222
Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/ebb742bfe1a4
Build-ID        20150325162500
Version         37.0
Device-Name     flame
FW-Release      4.4.2
FW-Incremental  39
FW-Date         Thu Oct 16 18:19:14 CST 2014
Bootloader      L1TC00011880

Comment 5

[Pi Wei Cheng [:piwei] (inactive)]

Issue is still reproducible on latest 2.2. The preview seems to update faster now when dragging the progress bar, but the slow-to-resume-playing issue still reliably reproduces.

This is tested with 319MB memory Flame.

Device: Flame 2.2 (KK, full flash, 319MB memory)
BuildID: 20150330002503
Gaia: 473cd63f53c855299b719285d9b95e3f2910782f
Gecko: 4b13c4254e2f
Gonk: ebad7da532429a6f5efadc00bf6ad8a41288a429
Version: 37.0 (2.2) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0

Comment 6

[Milan Sreckovic [:milan] (needinfo for best results)]

Can we reproduce with more than one device, configured to 319mb, and can we reproduce on the same device when not set to 319mb?

Comment 7

[Pi Wei Cheng [:piwei] (inactive)]

I did test with 2 devices when writing up the original bug, so yes it reproduces on multiple Flame's.

Re-tested with same build on comment 5 in 512MB memory and it behaves about the same. Video is slow to resume after dragging the progress bar. Repro rate is about 50%.

Comment 8

[Milan Sreckovic [:milan] (needinfo for best results)]

Thanks.  I'm assuming the "slow to resume" is still in the ~3 second timeframe; and just to be explicit, the image preview is also not updating?

Comment 9

[Pi Wei Cheng [:piwei] (inactive)]

Image preview is now updating, though it's not updating very quickly, it's like it's updating 2 frames per second of dragging. And yes the slow to resume is ~3 seconds.

Comment 10

[Blake Wu [:bwu][:blakewu]]

I can repro this problem on my v2.2 Flame and start to check this problem.

Comment 11

[Blake Wu [:bwu][:blakewu]]

After some investigations, sought frame is decoded fast enough, but video frame is not rendered in time.  
The rootcause is remainingTime[1] in AdvanceFrame() is not correct and large which makes next AdvanceFrame() call late and consequently video frames are rendered late. 
For seek forward case, 
The patch in bug 1112438 causes this regression since remainingTime is partly calculated from mAudioStartTime. By reverting that patch, seek forward becomes fast again. 
For seek backward case, 
Continue to check why remainingTime is large. 

[1]https://dxr.mozilla.org/mozilla-central/source/dom/media/MediaDecoderStateMachine.cpp?from=MediaDecoderStateMachine.cpp&case=true#2931

Comment 12

[Blake Wu [:bwu][:blakewu]]

(In reply to Blake Wu [:bwu][:blakewu] from comment #11)
> After some investigations, sought frame is decoded fast enough, but video
> frame is not rendered in time.  
> The rootcause is remainingTime[1] in AdvanceFrame() is not correct and large
> which makes next AdvanceFrame() call late and consequently video frames are
> rendered late. 
> For seek forward case, 
> The patch in bug 1112438 causes this regression since remainingTime is
> partly calculated from mAudioStartTime. By reverting that patch, seek
> forward becomes fast again. 
> For seek backward case, 
> Continue to check why remainingTime is large. 
The reason is frame->mTime used for the calculation of remainingTime is large because skipToNextKeyFrame is triggered wrongly and many video frames are skipped. After forcing disabling skipToNextKeyFrame, seek is fast.

Comment 13

[Blake Wu [:bwu][:blakewu]]

(In reply to Blake Wu [:bwu][:blakewu] from comment #12)
> (In reply to Blake Wu [:bwu][:blakewu] from comment #11)
> > After some investigations, sought frame is decoded fast enough, but video
> > frame is not rendered in time.  
> > The rootcause is remainingTime[1] in AdvanceFrame() is not correct and large
> > which makes next AdvanceFrame() call late and consequently video frames are
> > rendered late. 
> > For seek forward case, 
> > The patch in bug 1112438 causes this regression since remainingTime is
> > partly calculated from mAudioStartTime. By reverting that patch, seek
> > forward becomes fast again. 
Bug 1146304 has fixed this and the patch is landed yesterday. 
> > For seek backward case, 
> > Continue to check why remainingTime is large. 
> The reason is frame->mTime used for the calculation of remainingTime is
> large because skipToNextKeyFrame is triggered wrongly and many video frames
> are skipped. After forcing disabling skipToNextKeyFrame, seek is fast.

Comment 14

[Blake Wu [:bwu][:blakewu]]

(In reply to Blake Wu [:bwu][:blakewu] from comment #13)
> (In reply to Blake Wu [:bwu][:blakewu] from comment #12)
> > (In reply to Blake Wu [:bwu][:blakewu] from comment #11)
> > > After some investigations, sought frame is decoded fast enough, but video
> > > frame is not rendered in time.  
> > > The rootcause is remainingTime[1] in AdvanceFrame() is not correct and large
> > > which makes next AdvanceFrame() call late and consequently video frames are
> > > rendered late. 
> > > For seek forward case, 
> > > The patch in bug 1112438 causes this regression since remainingTime is
> > > partly calculated from mAudioStartTime. By reverting that patch, seek
> > > forward becomes fast again. 
> Bug 1146304 has fixed this and the patch is landed yesterday. 
> > > For seek backward case, 
> > > Continue to check why remainingTime is large. 
> > The reason is frame->mTime used for the calculation of remainingTime is
> > large because skipToNextKeyFrame is triggered wrongly and many video frames
> > are skipped. After forcing disabling skipToNextKeyFrame, seek is fast.
Currently the decision to trigger skipToNextKeyFrame seems too aggressive. 
Bug 1152218 is created to fix it.

Comment 15

[Blake Wu [:bwu][:blakewu]]

Under waiting the approval-mozilla-b2g37 for the 2.2 patch in bug 1152218.

Comment 16

[Blake Wu [:bwu][:blakewu]]

The 2.2 patch in bug 1152218 has been landed as well. Both problems described in comment 14 are fixed, so is this bug.

Comment 17

[Pi Wei Cheng [:piwei] (inactive)]

I'll be verifying this issue. Setting tracking flags to reflect current state.

Comment 18

[Pi Wei Cheng [:piwei] (inactive)]

This issue is NOT fixed on Flame 3.0, but it is fixed on Flame 2.2. Behavior on v3.0 is basically the same as original bug writeup: Preview doesn't update when dragging progress bar, and video is slow to resume after seeking.

Bug reproduces on:
Device: Flame 3.0 Master (full flashed 319MB KK)
BuildID: 20150427010202
Gaia: b4c949cdc780893897c9b45c1adea46e2eb694ff
Gecko: 37d60e3b8be6
Gonk: b83fc73de7b64594cd74b33e498bf08332b5d87b
Version: 40.0a1 (3.0 Master) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:40.0) Gecko/40.0 Firefox/40.0

Bug does not reproduce on:
Device: Flame 2.2 (full flashed 319MB KK)
BuildID: 20150427002504
Gaia: 265ca0bc9408c21fc4b25a259fcee7fb642cd06b
Gecko: 1908685d798d
Gonk: ebad7da532429a6f5efadc00bf6ad8a41288a429
Version: 37.0 (2.2) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0


Needinfo'ing the assignee to gain attention. I can file a separate bug if needed.

Comment 19

[Blake Wu [:bwu][:blakewu]]

(In reply to Pi Wei Cheng [:piwei] from comment #18)
> This issue is NOT fixed on Flame 3.0, but it is fixed on Flame 2.2. Behavior
> on v3.0 is basically the same as original bug writeup: Preview doesn't
> update when dragging progress bar, and video is slow to resume after seeking.
> 
> Bug reproduces on:
> Device: Flame 3.0 Master (full flashed 319MB KK)
> BuildID: 20150427010202
> Gaia: b4c949cdc780893897c9b45c1adea46e2eb694ff
> Gecko: 37d60e3b8be6
> Gonk: b83fc73de7b64594cd74b33e498bf08332b5d87b
> Version: 40.0a1 (3.0 Master) 
> Firmware Version: v18D-1
> User Agent: Mozilla/5.0 (Mobile; rv:40.0) Gecko/40.0 Firefox/40.0
> 
> Bug does not reproduce on:
> Device: Flame 2.2 (full flashed 319MB KK)
> BuildID: 20150427002504
> Gaia: 265ca0bc9408c21fc4b25a259fcee7fb642cd06b
> Gecko: 1908685d798d
> Gonk: ebad7da532429a6f5efadc00bf6ad8a41288a429
> Version: 37.0 (2.2) 
> Firmware Version: v18D-1
> User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0
> 
> 
> Needinfo'ing the assignee to gain attention. I can file a separate bug if
> needed.
Thanks for your info! 
Please file a bug and also attach a recorded video to show us how you tested it.

Comment 20

[Pi Wei Cheng [:piwei] (inactive)]

I have filed bug 1159343 for follow up.

Comment 21

[Blake Wu [:bwu][:blakewu]]

(In reply to Pi Wei Cheng [:piwei] from comment #20)
> I have filed bug 1159343 for follow up.
Thanks!