1161768 - [Video][Scrubber TIme] When scrubbing througha video, the "Time Elapsed" does not update until the user stops scrubbing

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::Video, enhancement, P3)

Description

[Derek Harris [:DerekH] - Dont need info. Last day 6/5/15]

Attachment: Video Scrubbing Logcat

Description:
When the user is scrubbing back and forth on the video playback bar, the time elapsed located on the left will not update until the user stops scrubbing. This is inconsistent with the music app. The music app will update the time elpased as the user is scrubbing back and forth, instead of afterwards. This makes finding a specific time of the video frusturating because the user has to constantly stop to wait for the time to update, and then try to correct themselvs 


Repro Steps:
1) Update a Flame to 20150505010204
2) Open Video app
3) Select a video
4) Scrub backa nd fourth through the video> Observe the "Time Elapsed" on the left


Actual:
The elapsed time does not update until the user stops scrubbing

Expected:
The elapsed time updates as the user is scrubbing


Environmental Variables:
Device: Flame 3.0 (319mb)(Kitkat)(Full Flash)
Build ID: 20150505010204
Gaia: 70077825aab2c7a79611befb40a5fe7e610d5443
Gecko: 102d0e9aa9e1
Gonk: a9f3f8fb8b0844724de32426b7bcc4e6dc4fa2ed
Version: 40.0a1 (3.0)
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:40.0) Gecko/40.0 Firefox/40.0


Repro frequency: 10/10
See attached: Logcat, Video - https://youtu.be/lkSvbuuk8gs

Comment 1

[Derek Harris [:DerekH] - Dont need info. Last day 6/5/15]

This issue DOES occur on Flame 2.2, 2.1, and 2.0

The elapsed time does not update until the user stops scrubbing


Environmental Variables:
Device: Flame 2.2 (319mb)(Kitkat)(Full Flash)
Build ID: 20150505002501
Gaia: 772a9491909abd02dc67278dd453746e2dd358a8
Gecko: 2df83538ae20
Gonk: ab265fb203390c70b8f2a054f38cf4b2f2dad70a
Version: 37.0 (2.2)
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0


Environmental Variables:
Device: Flame 2.1 (319mb)(Kitkat)(Full Flash)
Build ID: 20150504001200
Gaia: b4a03b7ee61de5a479b3cf0916f47e91a43b0f50
Gecko: 03bfe12e50d0
Gonk: a9f3f8fb8b0844724de32426b7bcc4e6dc4fa2ed
Version: 34.0 (2.1)
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0


Environmental Variables:
Device: Flame 2.0 (319mb)(Kitkat)(Full Flash)
Build ID: 20150427000202
Gaia: 84898cadf28b1a1fcd03b726cff658de470282f0
Gecko: f700ee0a507d
Gonk: ebad7da532429a6f5efadc00bf6ad8a41288a429
Version: 32.0 (2.0)
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:32.0) Gecko/32.0 Firefox/32.0

Comment 2

[Peter Bylenga [:PBylenga]]

NI on component owner for nomination decision and assignment.

Comment 3

[No-Jun Park [:njpark]]

Hi Blake, this bug sounds like https://bugzilla.mozilla.org/show_bug.cgi?id=1069806, which does not have any status updates for a while.  can you confirm?

Comment 4

[Blake Wu [:bwu][:blakewu]]

I think this bug is to address the problem that frame is not displayed immediately when users drag the progress bar unless users stop. And bug 1069806 is to address the timestamp is not currently shown when user drags the bar. It is a little different.

Comment 5

[Blake Wu [:bwu][:blakewu]]

John, 
May I know how often video APP calls fastSeek when users drag the process bar?

Comment 6

[Derek Harris [:DerekH] - Dont need info. Last day 6/5/15]

Blake(In reply to Blake Wu [:bwu][:blakewu] from comment #4)
> I think this bug is to address the problem that frame is not displayed
> immediately when users drag the progress bar unless users stop. And bug
> 1069806 is to address the timestamp is not currently shown when user drags
> the bar. It is a little different.

Blake, this bug was originally created to show that the timestamps were not updating as the user scrubbed through the playback bar. But I did also see that the video frame was not updating in realtime either.

Comment 7

[Blake Wu [:bwu][:blakewu]]

(In reply to Derek Harris [:DerekH] from comment #6)
> Blake(In reply to Blake Wu [:bwu][:blakewu] from comment #4)
> > I think this bug is to address the problem that frame is not displayed
> > immediately when users drag the progress bar unless users stop. And bug
> > 1069806 is to address the timestamp is not currently shown when user drags
> > the bar. It is a little different.
> 
> Blake, this bug was originally created to show that the timestamps were not
> updating as the user scrubbed through the playback bar. But I did also see
> that the video frame was not updating in realtime either.
Got it. Thanks for your information!

Comment 8

[John Hu [:johnhu][:johu][:醬糊小弟]]

(In reply to Blake Wu [:bwu][:blakewu] from comment #5)
> John, 
> May I know how often video APP calls fastSeek when users drag the process
> bar?

It is called on every touch move event received.

Comment 9

[Blake Wu [:bwu][:blakewu]]

(In reply to John Hu [:johnhu][:johu][:醬糊小弟] from comment #8)
> (In reply to Blake Wu [:bwu][:blakewu] from comment #5)
> > John, 
> > May I know how often video APP calls fastSeek when users drag the process
> > bar?
> 
> It is called on every touch move event received.
Thanks for this info. 
How frequent touch move event is triggered if user's finger doesn't leave screen? 
If it is very high, there would be many seeking operations in a short time. That may cause bad user experience since too many seeks could cause gecko busy in doing unnecessary seeks. It would better to do seek when a period of time or space is passed. We can split the video duration to 40 slices (for exmaple), each slice is duration/40. If the difference between last seek to the next seek is larger than one slice, we do seek. For example, if a video's duration is 20 second (slice = 20/40 = 0.5 sec), video app only do next seek if it is far away from the previous seek by over o.5 second.

Comment 10

[Blake Wu [:bwu][:blakewu]]

John, 
How do you think?

Comment 11

[John Hu [:johnhu][:johu][:醬糊小弟]]

(In reply to Blake Wu [:bwu][:blakewu] from comment #9)
> Thanks for this info. 
> How frequent touch move event is triggered if user's finger doesn't leave
> screen? 
> If it is very high, there would be many seeking operations in a short time.
> That may cause bad user experience since too many seeks could cause gecko
> busy in doing unnecessary seeks. It would better to do seek when a period of
> time or space is passed. We can split the video duration to 40 slices (for
> exmaple), each slice is duration/40. If the difference between last seek to
> the next seek is larger than one slice, we do seek. For example, if a
> video's duration is 20 second (slice = 20/40 = 0.5 sec), video app only do
> next seek if it is far away from the previous seek by over o.5 second.

The frequency is very high, I think. I don't think to split a video into multiple slices is a good idea because we cannot expected the length of a video is large enough to slice it. Like, if we have a video only have 3sec and we slice 40 slices, each slice may have 75ms. This doesn't make sense.

To wait for a small piece of time before seeking may be a good idea. But it would be nice to explore if we have other better idea, like seeking the last dragging position after onseeked event is fired and do the final seeking while user release the finger.

Comment 12

[Blake Wu [:bwu][:blakewu]]

(In reply to John Hu [:johnhu][:johu][:醬糊小弟] from comment #11)
> (In reply to Blake Wu [:bwu][:blakewu] from comment #9)
> > Thanks for this info. 
> > How frequent touch move event is triggered if user's finger doesn't leave
> > screen? 
> > If it is very high, there would be many seeking operations in a short time.
> > That may cause bad user experience since too many seeks could cause gecko
> > busy in doing unnecessary seeks. It would better to do seek when a period of
> > time or space is passed. We can split the video duration to 40 slices (for
> > exmaple), each slice is duration/40. If the difference between last seek to
> > the next seek is larger than one slice, we do seek. For example, if a
> > video's duration is 20 second (slice = 20/40 = 0.5 sec), video app only do
> > next seek if it is far away from the previous seek by over o.5 second.
> 
> The frequency is very high, I think. I don't think to split a video into
> multiple slices is a good idea because we cannot expected the length of a
> video is large enough to slice it. Like, if we have a video only have 3sec
> and we slice 40 slices, each slice may have 75ms. This doesn't make sense.
We can set a minimum value, like 100ms or 200ms to avoid this case. 
> 
> To wait for a small piece of time before seeking may be a good idea. But it
> would be nice to explore if we have other better idea, like seeking the last
> dragging position after onseeked event is fired and do the final seeking
Waiting onseeked to trigger the next seek might be not good for UX since there will be still many seeks waiting and queued to be done. It would better to avoid some many seeks from app side. 
> while user release the finger.

Comment 13

[John Hu [:johnhu][:johu][:醬糊小弟]]

(In reply to Blake Wu [:bwu][:blakewu] from comment #12)
> > To wait for a small piece of time before seeking may be a good idea. But it
> > would be nice to explore if we have other better idea, like seeking the last
> > dragging position after onseeked event is fired and do the final seeking
> Waiting onseeked to trigger the next seek might be not good for UX since
> there will be still many seeks waiting and queued to be done. It would
> better to avoid some many seeks from app side. 

I mean to sync the last position and issue the seek which should be made in gaia. If the seeked is not received, the video frame will be updated. So, what we can do is to move the scrubber to correct position. At this moment, we don't issue seek request. Once the seeked event is got, we calculate the current position and use it to issue the seek request. If this can be made by gecko, it would be good. But this should be able to be implemented in gaia.

> > while user release the finger.

Comment 14

[John Hu [:johnhu][:johu][:醬糊小弟]]

(In reply to John Hu [:johnhu][:johu][:醬糊小弟] from comment #13)
> (In reply to Blake Wu [:bwu][:blakewu] from comment #12)
> > > To wait for a small piece of time before seeking may be a good idea. But it
> > > would be nice to explore if we have other better idea, like seeking the last
> > > dragging position after onseeked event is fired and do the final seeking
> > Waiting onseeked to trigger the next seek might be not good for UX since
> > there will be still many seeks waiting and queued to be done. It would
> > better to avoid some many seeks from app side. 
> 
> I mean to sync the last position and issue the seek which should be made in
> gaia. If the seeked is not received, the video frame will be updated. 
                                                       ^^^^^^^^^^^  <== typo here. the video frame won't be updated.

Comment 15

[No-Jun Park [:njpark]]

Hi Blake, do you think it's worth marking this bug as 3.0 blocker?

Comment 16

[Blake Wu [:bwu][:blakewu]]

(In reply to No-Jun Park [:njpark] from comment #15)
> Hi Blake, do you think it's worth marking this bug as 3.0 blocker?
Sure. I think it's worth since the current user experience is not good.

Comment 17

[Blake Wu [:bwu][:blakewu]]

[Blocking Requested - why for this release]:
The user experience of seek is not good enough and there is some room to improve.

Comment 18

[Hema Koka [:hema]]

Removing it from blocker and marking it enhancement. 

Russ, once the telemetry work for 2.5 is done, please look into this. Also if you can provide an estimate for this and the risk in the changes, it will be great. Based on that, we can slot this into the appropriate sprint. 

(For now, slotting this into one of the september sprints, so it stays in the radar for planning)

Thanks
Hema

Comment 19

[Russ Nicoletti [:russn]]

John and Blake, I'm not certain which issue or issues this bug is focused on. Is it to update the elapsed time while the user is moving the scrubber or is it to update the frame while the user is moving the scrubber. Comment 4 seems to indicate it is about updating the frame and that bug 1069806 is about updating the elapsed time. I'm asking because they way I understand comment 13, John's suggestion will ensure the elapsed time is updated but will result in the frame not being updated. So if this bug is about updating the frame while the user is moving the scrubber, John's suggestion will not resolve this bug. Am I correct?

Comment 20

[Russ Nicoletti [:russn]]

Btw, I'm wondering if we can use a variation of Blake's suggestion (comment 9). That is, given the elapsed time has a granularity of 1 second, we can have a fixed "seek slice" of 1 second for all videos. If the scrubber is moved more than 1 second past of forward the last seek position, we seek (and explicitly update the elapsed time -- don't wait for onseeked). I have implemented a POC of this implementation and it seems to work well: [1] (pardon the unprofessional video)

[1] http://www.youtube.com/watch?v=dfXIhQcOOVU

Comment 21

[Blake Wu [:bwu][:blakewu]]

(In reply to Russ Nicoletti [:russn] (PTO until 8/10/15) from comment #19)
> John and Blake, I'm not certain which issue or issues this bug is focused
> on. Is it to update the elapsed time while the user is moving the scrubber
> or is it to update the frame while the user is moving the scrubber. Comment
> 4 seems to indicate it is about updating the frame and that bug 1069806 is
> about updating the elapsed time. I'm asking because they way I understand
> comment 13, John's suggestion will ensure the elapsed time is updated but
> will result in the frame not being updated. So if this bug is about updating
> the frame while the user is moving the scrubber, John's suggestion will not
> resolve this bug. Am I correct?
IMHO, what we should do is to update the frame while the user is moving the scrubber and the elapsed time needs to be updated as the same as the timestamp of the displayed frame.

Comment 22

[John Hu [:johnhu][:johu][:醬糊小弟]]

(In reply to Russ Nicoletti [:russn] (PTO until 8/10/15) from comment #19)
> So if this bug is about updating
> the frame while the user is moving the scrubber, John's suggestion will not
> resolve this bug. Am I correct?

Yes, you are correct.

IIRC, this should be an issue that UI doesn't update while user drags scrubber.

(In reply to Russ Nicoletti [:russn] (PTO until 8/10/15) from comment #20)
> Btw, I'm wondering if we can use a variation of Blake's suggestion (comment
> 9). That is, given the elapsed time has a granularity of 1 second, we can
> have a fixed "seek slice" of 1 second for all videos. If the scrubber is
> moved more than 1 second past of forward the last seek position, we seek
> (and explicitly update the elapsed time -- don't wait for onseeked). I have
> implemented a POC of this implementation and it seems to work well: [1]
> (pardon the unprofessional video)
> 
> [1] http://www.youtube.com/watch?v=dfXIhQcOOVU

This looks pretty good. Please also test with very short video file. Thanks.

Comment 23

[Russ Nicoletti [:russn]]

With my POC (comment 20), the behavior for short videos (less than 3 seconds) is not great since it will only seek at most twice while dragging. Perhaps we can use a shorter seek slice (250ms?) when the video duration is less than a specified duration (3 seconds?).

Comment 24

[Russ Nicoletti [:russn]]

Hema, I believe the effort to address this issue is small (a couple of days) and the risk is also small. Putting it in a September sprint should be fine.

Comment 25

[Russ Nicoletti [:russn]]

After discussing with djf, I think a variation on johu's suggestion is best: when the user is moving the play head, record the seek position but only seek when the video element is not in the middle of seeking. When we get the 'seeked' event, if the current position of the media is not the latest that was recorded, seek to the latest recorded position.

Comment 26

[BMO Automation]

Firefox OS is not being worked on