[Video] Seek functionality via dragging progress bar does not update preview when seeking and is slow to resume after seeking

RESOLVED FIXED

Status

()

Core
Audio/Video
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: piwei, Assigned: bwu)

Tracking

unspecified
ARM
Gonk (Firefox OS)
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(blocking-b2g:2.2+, b2g-v2.1 unaffected, b2g-v2.2 verified, b2g-master fixed)

Details

(Whiteboard: [3.0-Daily-Testing], URL)

Attachments

(1 attachment)

(Reporter)

Description

3 years ago
Issue found while verifying bug 1133955.

STR:
1) On Flame 3.0, record a video of ~30 seconds
2) In Video app, start playing the video created at step 1
3) Drag the progress bar around, and let go of finger. Repeat this a few times.

Actual: While dragging the progress bar, the image preview does NOT update as dragging occurs. And after let go of finger, sometimes the video is slow to resume playing (takes ~3 seconds to resume).

Expected: The image preview updates as dragging occurs (so user knows where in the video they're seeking); and after finger leaves screen, the app continues to play the video without delay.

Device: Flame 3.0
BuildID: 20150324010202
Gaia: efebbafd12fc42ddcd378948b683a51106517660
Gecko: 840cfd5bc971
Gonk: b83fc73de7b64594cd74b33e498bf08332b5d87b
Version: 39.0a1 (3.0) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:39.0) Gecko/39.0 Firefox/39.0

Video and branch check will be added tomorrow.
(Reporter)

Comment 1

3 years ago
Video:
https://www.youtube.com/watch?v=yRBYg9_2-oo

This issue occurs on 2.2.

Device: Flame 2.2
BuildID: 20150324002504
Gaia: 014d38f7ad3912b8b33cb08ce7535a5dc5aced59
Gecko: 7a9f2a248e57
Gonk: ebad7da532429a6f5efadc00bf6ad8a41288a429
Version: 37.0 (2.2) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0

--------

This issue does not occur on 2.1, however on 2.1 bug 1133955 occurs.

Device: Flame 2.1
BuildID: 20150325001202
Gaia: b8ae0df34362420fe4a9c90effa5247a1f5c844d
Gecko: 2a05cd42088b
Gonk: b83fc73de7b64594cd74b33e498bf08332b5d87b
Version: 34.0 (2.1) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0

------

Also this issue is affecting Gallery and Camera Preview app. It won't let me change Product/Component to Core/Video Audio for some reason.
status-b2g-v2.1: --- → unaffected
status-b2g-v2.2: --- → affected
status-b2g-master: --- → affected
Summary: [Video] Seek functionality via dragging progress bar is slow to update and does not update preview when seeking → [Video] Seek functionality via dragging progress bar does not update preview when seeking and is slow to resume after seeking
Whiteboard: [3.0-Daily-Testing]
(Reporter)

Comment 2

3 years ago
Created attachment 8583176 [details]
logcat of issue
(Reporter)

Updated

3 years ago
QA Whiteboard: [QAnalyst-Triage?]
Flags: needinfo?(ktucker)
This was written because of a failed verification. It was a blocker so nominating this one 2.2?
Blocks: 1133955
blocking-b2g: --- → 2.2?
QA Whiteboard: [QAnalyst-Triage?] → [QAnalyst-Triage+]
Flags: needinfo?(ktucker)

Updated

3 years ago
Component: Gaia::Video → Video/Audio
Product: Firefox OS → Core

Comment 4

3 years ago
Can't be reproduced in this Flame-v2.2.

-----

Gaia-Rev        2d42a7c1d99472780e3aeb2e16d7cb0a4adbd222
Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/ebb742bfe1a4
Build-ID        20150325162500
Version         37.0
Device-Name     flame
FW-Release      4.4.2
FW-Incremental  39
FW-Date         Thu Oct 16 18:19:14 CST 2014
Bootloader      L1TC00011880
(Reporter)

Comment 5

3 years ago
Issue is still reproducible on latest 2.2. The preview seems to update faster now when dragging the progress bar, but the slow-to-resume-playing issue still reliably reproduces.

This is tested with 319MB memory Flame.

Device: Flame 2.2 (KK, full flash, 319MB memory)
BuildID: 20150330002503
Gaia: 473cd63f53c855299b719285d9b95e3f2910782f
Gecko: 4b13c4254e2f
Gonk: ebad7da532429a6f5efadc00bf6ad8a41288a429
Version: 37.0 (2.2) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0
Can we reproduce with more than one device, configured to 319mb, and can we reproduce on the same device when not set to 319mb?
(Reporter)

Comment 7

3 years ago
I did test with 2 devices when writing up the original bug, so yes it reproduces on multiple Flame's.

Re-tested with same build on comment 5 in 512MB memory and it behaves about the same. Video is slow to resume after dragging the progress bar. Repro rate is about 50%.
Thanks.  I'm assuming the "slow to resume" is still in the ~3 second timeframe; and just to be explicit, the image preview is also not updating?
(Reporter)

Comment 9

3 years ago
Image preview is now updating, though it's not updating very quickly, it's like it's updating 2 frames per second of dragging. And yes the slow to resume is ~3 seconds.

Updated

3 years ago
blocking-b2g: 2.2? → 2.2+
(Assignee)

Comment 10

3 years ago
I can repro this problem on my v2.2 Flame and start to check this problem.
Assignee: nobody → bwu
(Assignee)

Updated

3 years ago
Keywords: regressionwindow-wanted
(Assignee)

Updated

3 years ago
Keywords: regressionwindow-wanted
(Assignee)

Comment 11

3 years ago
After some investigations, sought frame is decoded fast enough, but video frame is not rendered in time.  
The rootcause is remainingTime[1] in AdvanceFrame() is not correct and large which makes next AdvanceFrame() call late and consequently video frames are rendered late. 
For seek forward case, 
The patch in bug 1112438 causes this regression since remainingTime is partly calculated from mAudioStartTime. By reverting that patch, seek forward becomes fast again. 
For seek backward case, 
Continue to check why remainingTime is large. 

[1]https://dxr.mozilla.org/mozilla-central/source/dom/media/MediaDecoderStateMachine.cpp?from=MediaDecoderStateMachine.cpp&case=true#2931
(Assignee)

Comment 12

3 years ago
(In reply to Blake Wu [:bwu][:blakewu] from comment #11)
> After some investigations, sought frame is decoded fast enough, but video
> frame is not rendered in time.  
> The rootcause is remainingTime[1] in AdvanceFrame() is not correct and large
> which makes next AdvanceFrame() call late and consequently video frames are
> rendered late. 
> For seek forward case, 
> The patch in bug 1112438 causes this regression since remainingTime is
> partly calculated from mAudioStartTime. By reverting that patch, seek
> forward becomes fast again. 
> For seek backward case, 
> Continue to check why remainingTime is large. 
The reason is frame->mTime used for the calculation of remainingTime is large because skipToNextKeyFrame is triggered wrongly and many video frames are skipped. After forcing disabling skipToNextKeyFrame, seek is fast.
(Assignee)

Comment 13

3 years ago
(In reply to Blake Wu [:bwu][:blakewu] from comment #12)
> (In reply to Blake Wu [:bwu][:blakewu] from comment #11)
> > After some investigations, sought frame is decoded fast enough, but video
> > frame is not rendered in time.  
> > The rootcause is remainingTime[1] in AdvanceFrame() is not correct and large
> > which makes next AdvanceFrame() call late and consequently video frames are
> > rendered late. 
> > For seek forward case, 
> > The patch in bug 1112438 causes this regression since remainingTime is
> > partly calculated from mAudioStartTime. By reverting that patch, seek
> > forward becomes fast again. 
Bug 1146304 has fixed this and the patch is landed yesterday. 
> > For seek backward case, 
> > Continue to check why remainingTime is large. 
> The reason is frame->mTime used for the calculation of remainingTime is
> large because skipToNextKeyFrame is triggered wrongly and many video frames
> are skipped. After forcing disabling skipToNextKeyFrame, seek is fast.
See Also: → bug 1146304
(Assignee)

Comment 14

3 years ago
(In reply to Blake Wu [:bwu][:blakewu] from comment #13)
> (In reply to Blake Wu [:bwu][:blakewu] from comment #12)
> > (In reply to Blake Wu [:bwu][:blakewu] from comment #11)
> > > After some investigations, sought frame is decoded fast enough, but video
> > > frame is not rendered in time.  
> > > The rootcause is remainingTime[1] in AdvanceFrame() is not correct and large
> > > which makes next AdvanceFrame() call late and consequently video frames are
> > > rendered late. 
> > > For seek forward case, 
> > > The patch in bug 1112438 causes this regression since remainingTime is
> > > partly calculated from mAudioStartTime. By reverting that patch, seek
> > > forward becomes fast again. 
> Bug 1146304 has fixed this and the patch is landed yesterday. 
> > > For seek backward case, 
> > > Continue to check why remainingTime is large. 
> > The reason is frame->mTime used for the calculation of remainingTime is
> > large because skipToNextKeyFrame is triggered wrongly and many video frames
> > are skipped. After forcing disabling skipToNextKeyFrame, seek is fast.
Currently the decision to trigger skipToNextKeyFrame seems too aggressive. 
Bug 1152218 is created to fix it.
Depends on: 1152218
(Assignee)

Comment 15

3 years ago
Under waiting the approval-mozilla-b2g37 for the 2.2 patch in bug 1152218.
(Assignee)

Comment 16

3 years ago
The 2.2 patch in bug 1152218 has been landed as well. Both problems described in comment 14 are fixed, so is this bug.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Reporter)

Comment 17

3 years ago
I'll be verifying this issue. Setting tracking flags to reflect current state.
status-b2g-v2.2: affected → fixed
status-b2g-master: affected → fixed
(Reporter)

Comment 18

3 years ago
This issue is NOT fixed on Flame 3.0, but it is fixed on Flame 2.2. Behavior on v3.0 is basically the same as original bug writeup: Preview doesn't update when dragging progress bar, and video is slow to resume after seeking.

Bug reproduces on:
Device: Flame 3.0 Master (full flashed 319MB KK)
BuildID: 20150427010202
Gaia: b4c949cdc780893897c9b45c1adea46e2eb694ff
Gecko: 37d60e3b8be6
Gonk: b83fc73de7b64594cd74b33e498bf08332b5d87b
Version: 40.0a1 (3.0 Master) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:40.0) Gecko/40.0 Firefox/40.0

Bug does not reproduce on:
Device: Flame 2.2 (full flashed 319MB KK)
BuildID: 20150427002504
Gaia: 265ca0bc9408c21fc4b25a259fcee7fb642cd06b
Gecko: 1908685d798d
Gonk: ebad7da532429a6f5efadc00bf6ad8a41288a429
Version: 37.0 (2.2) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0


Needinfo'ing the assignee to gain attention. I can file a separate bug if needed.
QA Whiteboard: [QAnalyst-Triage+] → [failed-verification][QAnalyst-Triage?]
status-b2g-v2.2: fixed → verified
Flags: needinfo?(ktucker)
Flags: needinfo?(bwu)
QA Whiteboard: [failed-verification][QAnalyst-Triage?] → [failed-verification][QAnalyst-Triage+]
Flags: needinfo?(ktucker)
(Assignee)

Comment 19

3 years ago
(In reply to Pi Wei Cheng [:piwei] from comment #18)
> This issue is NOT fixed on Flame 3.0, but it is fixed on Flame 2.2. Behavior
> on v3.0 is basically the same as original bug writeup: Preview doesn't
> update when dragging progress bar, and video is slow to resume after seeking.
> 
> Bug reproduces on:
> Device: Flame 3.0 Master (full flashed 319MB KK)
> BuildID: 20150427010202
> Gaia: b4c949cdc780893897c9b45c1adea46e2eb694ff
> Gecko: 37d60e3b8be6
> Gonk: b83fc73de7b64594cd74b33e498bf08332b5d87b
> Version: 40.0a1 (3.0 Master) 
> Firmware Version: v18D-1
> User Agent: Mozilla/5.0 (Mobile; rv:40.0) Gecko/40.0 Firefox/40.0
> 
> Bug does not reproduce on:
> Device: Flame 2.2 (full flashed 319MB KK)
> BuildID: 20150427002504
> Gaia: 265ca0bc9408c21fc4b25a259fcee7fb642cd06b
> Gecko: 1908685d798d
> Gonk: ebad7da532429a6f5efadc00bf6ad8a41288a429
> Version: 37.0 (2.2) 
> Firmware Version: v18D-1
> User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0
> 
> 
> Needinfo'ing the assignee to gain attention. I can file a separate bug if
> needed.
Thanks for your info! 
Please file a bug and also attach a recorded video to show us how you tested it.
Flags: needinfo?(bwu)
(Reporter)

Updated

3 years ago
Depends on: 1159343
(Reporter)

Comment 20

3 years ago
I have filed bug 1159343 for follow up.
Flags: needinfo?(bwu)
(Assignee)

Comment 21

3 years ago
(In reply to Pi Wei Cheng [:piwei] from comment #20)
> I have filed bug 1159343 for follow up.
Thanks!
Flags: needinfo?(bwu)

Updated

3 years ago
See Also: → bug 1159343
You need to log in before you can comment on or make changes to this bug.