1398028 - HTMLVideoElement.currentTime does not update concurrent with MediaSource.duration

Status: RESOLVED INVALID

(Core :: General, defect)

Description

[guest271314]

User Agent: Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/60.0.3112.113 Chrome/60.0.3112.113 Safari/537.36

Steps to reproduce:

1. Set MediaSource SourceBuffer.mode to "sequence"
2. At waiting event within sourceopen event handler of MediaSource instance call .endOfStream() on MediaSource 
3. Within sourceended event of MediaSource log HTML.videoElement.currentTime and MediaSource.duration


Actual results:

1. HTMLVideoElement.currentTime is incorrect as to actual playback
2. timeupdate event of HTMLVideoElement is not dispatched


Expected results:

1. HTMLVideoElement.currentTime should be within one second of MediaSource.duration
2. HTMLVideoElement timeupdate event should be dispatched

which each occurs at Chromium 60

Comment 1

[guest271314]

To reproduce https://github.com/guest271314/recordMediaFragments/blob/master/demos/recordMediaFragments.html

Comment 2

[Jean-Yves Avenard [:jya]]

I see no link between the steps you describe and the page you link to. Please make a page that narrowed on what you think is an issue.

currentTime will be of the value of duration, only once playback has ended. So you must wait for the "ended" event.

sourceended event indicate that the source buffer has been ended and that the endOfStream() operation has completed.

It has nothing to do with the progress of currentTime, nor playback.

The currentTime value you will read in your sourceended event handler will be the value of where playback is up to, there's ZERO guarantee that it will have reached the end.

Comment 3

[guest271314]

(In reply to Jean-Yves Avenard [:jya] from comment #2)
> I see no link between the steps you describe and the page you link to.
> Please make a page that narrowed on what you think is an issue.
> 
> currentTime will be of the value of duration, only once playback has ended.
> So you must wait for the "ended" event.
> 
> sourceended event indicate that the source buffer has been ended and that
> the endOfStream() operation has completed.
> 
> It has nothing to do with the progress of currentTime, nor playback.
> 
> The currentTime value you will read in your sourceended event handler will
> be the value of where playback is up to, there's ZERO guarantee that it will
> have reached the end.

Chromium 60 updates the .duration of <video> element following each appended ArrayBuffer. The issue is that Firefox Trunk (57) does not render playback of all of the appended array buffers, nor update the .duration property of the <video> element. 

Another issue is that audio of concatenated Blobs is not rendered at Firefox 55 or 57.

Comment 4

[guest271314]

(In reply to Jean-Yves Avenard [:jya] from comment #2)
> I see no link between the steps you describe and the page you link to.
> Please make a page that narrowed on what you think is an issue.
> 
> currentTime will be of the value of duration, only once playback has ended.
> So you must wait for the "ended" event.
> 
> sourceended event indicate that the source buffer has been ended and that
> the endOfStream() operation has completed.
> 
> It has nothing to do with the progress of currentTime, nor playback.
> 
> The currentTime value you will read in your sourceended event handler will
> be the value of where playback is up to, there's ZERO guarantee that it will
> have reached the end.

Have tried to implement recording media fragments, then concatenating media fragments to a single Blob using MediaSource with SourceBuffer set to both "segments" and "sequence" at Firefox 55 and 57. Neither approach renders media playback the same as Chromium 60. How to achieve this?

plnkr https://plnkr.co/edit/79Fbu8oQuWDKgoDENFHm?p=preview

Comment 5

[Jean-Yves Avenard [:jya]]

you're understanding is incorrect. you should read the MSE spec more closely. All your answers are in the spec. Yet you continue to lodge issues over and over again on the MSE github that clearly show you haven't read the spec carefully enough.

How the duration behaves is as per spec.
when you append a buffer, only when the updateend event has been fired would the duration algorithm guaranteed to have been called.
appendBuffer is an asynchronous operation.

the durationchange event would also have been fired (This is the only thing telling you that the duration has been updated).

Do not assume, because by luck this is what chrome happen to do one thing at a particular time, that the duration would be updated at other times.

If you read the duration right after calling appendBuffer, you don't know for sure that the operation has actually been completed, again it's an asynchronous operation as per spec.

You need to rely on specific events to know if an attribute has been updated. Do not assume that because you've observed a particular behaviour that this is how all other browser will behave.

If you want to know if the duration has been updated -> you need to do that work when "durationchange" has been fired.
If you want to know if playback has reach the end, you need to listen to the "ended" event.

calling endOfStream() will not magically make all your content entirely played. It only says to the MSE engine to not expect more data to come.

Firefox isn't Chrome.

Comment 6

[guest271314]

(In reply to Jean-Yves Avenard [:jya] from comment #5)
> you're understanding is incorrect. you should read the MSE spec more
> closely. All your answers are in the spec. Yet you continue to lodge issues
> over and over again on the MSE github that clearly show you haven't read the
> spec carefully enough.
> 
> How the duration behaves is as per spec.
> when you append a buffer, only when the updateend event has been fired would
> the duration algorithm guaranteed to have been called.
> appendBuffer is an asynchronous operation.
> 
> the durationchange event would also have been fired (This is the only thing
> telling you that the duration has been updated).
> 
> Do not assume, because by luck this is what chrome happen to do one thing at
> a particular time, that the duration would be updated at other times.
> 
> If you read the duration right after calling appendBuffer, you don't know
> for sure that the operation has actually been completed, again it's an
> asynchronous operation as per spec.
> 
> You need to rely on specific events to know if an attribute has been
> updated. Do not assume that because you've observed a particular behaviour
> that this is how all other browser will behave.
> 
> If you want to know if the duration has been updated -> you need to do that
> work when "durationchange" has been fired.
> If you want to know if playback has reach the end, you need to listen to the
> "ended" event.
> 
> calling endOfStream() will not magically make all your content entirely
> played. It only says to the MSE engine to not expect more data to come.
> 
> Firefox isn't Chrome.

Yes, understanding may be incorrect, here, that is reason for asking questions. Called .endOfStream() at waiting event, then attached timeupdate event to try to get all parts of media played back at MediaSource, timeupdate event is not called.

updateend event is used at both trials of different approaches.

The purpose of the questions is attempt to get better understanding. 

Not sure why there is an apparent aversion to users of the implementations asking questions. 

Do you suggest that we stop trying to implement the application at Firefox?

Comment 7

[Jean-Yves Avenard [:jya]]

The W3c github repository, nor Bugzilla are there to provide consulting services, nor write your code for you. You have lodged countless "bugs" on github, asking details that are fully explained in the MSE or HTML5 spec.
Here you report bugs, if there is a problem in our implementation. If you want to ask question or how to do something, as mentioned multiple times there are better platforms to do that: forums, stack overflow etc.
This is my last answer to any of your posts

Comment 8

[guest271314]

(In reply to Jean-Yves Avenard [:jya] from comment #7)
> The W3c github repository, nor Bugzilla are there to provide consulting
> services, nor write your code for you. You have lodged countless "bugs" on
> github, asking details that are fully explained in the MSE or HTML5 spec.
> Here you report bugs, if there is a problem in our implementation. If you
> want to ask question or how to do something, as mentioned multiple times
> there are better platforms to do that: forums, stack overflow etc.
> This is my last answer to any of your posts

Did not ask you or anyone else to write code. No, the questions are not fully explained, else would not ask the questions seeking clarification. 

You exaggerate with your estimation of "countless" bugs being filed by this user to the point of being inherently false. You can count the bugs that this user has filed.

Ok, that is your last answer to any of this users' posts.Perhaps you should not have replied in the first instance if you are not inclined to help, and are only willing to refer to a specification when specific questions about the specification and implementations are asked.

Comment 9

[guest271314]

(In reply to Jean-Yves Avenard [:jya] from comment #7)
> You have lodged countless "bugs" on github

That is a lie.

Comment 10

[Jean-Yves Avenard [:jya]]

(In reply to guest271314 from comment #9)
> (In reply to Jean-Yves Avenard [:jya] from comment #7)
> > You have lodged countless "bugs" on github
> 
> That is a lie.


It certainly feels like it.

If you want help on how to program, have questions on how to use MSE, here is *NOT* the right place.
Use Stack Overflow, can try the mozillazine as well; http://forums.mozillazine.org/viewforum.php?f=25&sid=c873b41ebd82f0e1a6965ffc01109d67

Again, Bugzilla (or github issues) is there to report bugs, not to ask questions for help or clarification on some point in the spec.

If you're 100%"sure you have found a bug, compared with what the spec states and found a discrepancy, then please lodge it. Otherwise you're just hogging our limited resources and taking valuable time.

Comment 11

[guest271314]

(In reply to Jean-Yves Avenard [:jya] from comment #10)
> (In reply to guest271314 from comment #9)
> > (In reply to Jean-Yves Avenard [:jya] from comment #7)
> > > You have lodged countless "bugs" on github
> > 
> > That is a lie.
> 
> 
> It certainly feels like it.
> 
> If you want help on how to program, have questions on how to use MSE, here
> is *NOT* the right place.
> Use Stack Overflow, can try the mozillazine as well;
> http://forums.mozillazine.org/viewforum.
> php?f=25&sid=c873b41ebd82f0e1a6965ffc01109d67
> 
> Again, Bugzilla (or github issues) is there to report bugs, not to ask
> questions for help or clarification on some point in the spec.
> 
> If you're 100%"sure you have found a bug, compared with what the spec states
> and found a discrepancy, then please lodge it. Otherwise you're just hogging
> our limited resources and taking valuable time.

fwiw, finally composed a version which meets requirement at both Firefox and Chromium. There were several issues which faced. Created a pattern which substitutes HTMLMediaElement.mozCaptureStream() for AudioContext() and HTMLCanvasElement.captureStream(); using fetch() to get a Blob of media resource as a workaround for cross-origin request; waiting, canplay and timeupdate events of HTMLMediaElement to continue playing media element having MediaSource as .src while recording media. The MediaSource specification did not offer guidance as to how to achieve the procedure, other than to use "sequence" as .mode of SourceBuffer. Was not entirely trivial, at least for this user, took about 27 different versions to achieve.

Comment 12

[guest271314]

(In reply to Jean-Yves Avenard [:jya] from comment #10)
> (In reply to guest271314 from comment #9)
> > (In reply to Jean-Yves Avenard [:jya] from comment #7)
> > > You have lodged countless "bugs" on github
> > 
> > That is a lie.
> 
> 
> It certainly feels like it.
> 
> If you want help on how to program, have questions on how to use MSE, here
> is *NOT* the right place.
> Use Stack Overflow, can try the mozillazine as well;
> http://forums.mozillazine.org/viewforum.
> php?f=25&sid=c873b41ebd82f0e1a6965ffc01109d67
> 
> Again, Bugzilla (or github issues) is there to report bugs, not to ask
> questions for help or clarification on some point in the spec.
> 
> If you're 100%"sure you have found a bug, compared with what the spec states
> and found a discrepancy, then please lodge it. Otherwise you're just hogging
> our limited resources and taking valuable time.

One issue that have noticed with approach which renders expected result at Firefox is that the HTMLVideoElement does not consistently play to the end of the media resource. Not sure if the issue is MediaSource or HTMLMediaElement implementation. 

In order to play the media to the end of the HTMLMediaElement.duration or MediaSource.duration it is necessary to manually seek on timeline of controls to a time beyond the current time when the playback stalls.

Comment 13

[guest271314]

(In reply to Jean-Yves Avenard [:jya] from comment #10)
> If you're 100%"sure you have found a bug, compared with what the spec states
> and found a discrepancy, then please lodge it. Otherwise you're just hogging
> our limited resources and taking valuable time.

This is the bug https://bugzilla.mozilla.org/show_bug.cgi?id=1400587

Comment 14

[Firefox Bug Husbandry Bot]

Moving from Core::Untriaged to Core::General https://bugzilla.mozilla.org/show_bug.cgi?id=1407598