Same timestamps in audio recorded with media recorder api (in audio+video mode)(with vorbis)

RESOLVED FIXED in Firefox 45

Status

()

P2
normal
Rank:
25
RESOLVED FIXED
4 years ago
3 years ago

People

(Reporter: stephane.schoorens, Assigned: fs.in.nccu)

Tracking

37 Branch
mozilla45
x86_64
Linux
Points:
---

Firefox Tracking Flags

(firefox45 fixed)

Details

Attachments

(1 attachment, 1 obsolete attachment)

(Reporter)

Description

4 years ago
User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:37.0) Gecko/20100101 Firefox/37.0
Build ID: 20150403141925

Steps to reproduce:

Recording a stream (audio + video) with the media recorder api
var session = {
        audio: true,
        video: {
            mozMediaSource: source || "screen",
            mediaSource: source || "screen"
        }
    };

    navigator.getUserMedia(session, 
        function (stream) {
            var mediaRecorder = new MediaRecorder(stream);
            mediaRecorder.start();
        });


Actual results:

timestamps in the audio track are the same each seconds.

Cmd to analyse it:
gst-launch-0.10 filesrc location=record.webm ! decodebin ! audioconvert ! identity ! alsasink -v -m
Result:
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:00.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef00130e0"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:00.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef00738f0"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:00.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef00738f0"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:00.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef0096040"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:00.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef0085210"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:00.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef00855d0"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:00.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef0085210"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:00.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef0085210"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:01.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef0096540"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:01.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef0096180"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:01.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef0085490"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:01.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef00b7190"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:01.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef00130e0"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = "chain   ******* (identity0:sink) (4096 bytes, timestamp: 0:00:01.000000000, duration: 0:00:00.021333333, offset: -1, offset_end: -1, flags: 0 ) 0x7efef0096680"


Expected results:

timestamps need to be set on each audio frame. this didn't happen with an audio only record.
(Reporter)

Updated

4 years ago
Summary: Smae timestamp in audio recorded with media recorder (in audio+video mode)(with vorbis) → Same timestamps in audio recorded with media recorder (in audio+video mode)(with vorbis)
(Reporter)

Comment 1

4 years ago
This also happen while recording camera+audio instead of screen+audio
(Reporter)

Updated

4 years ago
Component: Untriaged → Web Apps
(Reporter)

Updated

4 years ago
Summary: Same timestamps in audio recorded with media recorder (in audio+video mode)(with vorbis) → Same timestamps in audio recorded with media recorder api (in audio+video mode)(with vorbis)

Comment 2

3 years ago
Andrew should this be filed under Web API?
Flags: needinfo?(overholt)
(In reply to vkrishnamoorthy@mozilla.com [:Vishy] from comment #2)
> Andrew should this be filed under Web API?

Let's try Media.
Component: Web Apps → Video/Audio: Recording
Flags: needinfo?(overholt)
Product: Firefox → Core
Status: UNCONFIRMED → NEW
Rank: 25
Ever confirmed: true
Priority: -- → P2
(Assignee)

Comment 4

3 years ago
Created attachment 8688817 [details] [diff] [review]
Handle timestamps of video/webm vorbis track encoding.

Audio-only recording is audio/ogg with opus encoding.
Audio frame duration will be treated correctly in OpusTrackEncoder/OggWriter.

video/webm recording is vp8/vorbis while audio track timecode is not handled.
My patch tries to write timestamp in VorbisTrackEncoder,
and insert timecode of audio frames in EbmlComposer.

With my patch the recorded webm files can be played normally with sound by VLC.
Without it the recorded files will be played by VLC silently. 

Hello Maire,
I don't know exactly who could be the reviewer.
Do you have any idea? Thanks.
Flags: needinfo?(mreavy)
Comment on attachment 8688817 [details] [diff] [review]
Handle timestamps of video/webm vorbis track encoding.

Review of attachment 8688817 [details] [diff] [review]:
-----------------------------------------------------------------

This looks good to me.  It make sense to have a second review from any of the following: jesup, roc, or jya.
Attachment #8688817 - Flags: review+
Randell -- If you have time today, can you review this?
Flags: needinfo?(mreavy) → needinfo?(rjesup)

Updated

3 years ago
Attachment #8688817 - Flags: review+

Updated

3 years ago
Flags: needinfo?(rjesup)
From my perspective, we're good to land this.
Flags: needinfo?(fs.in.nccu)
(Assignee)

Comment 8

3 years ago
Created attachment 8689298 [details] [diff] [review]
Handle timestamps of video/webm vorbis track encoding. r=mreavy, r=rjesup

carry r+ from mreavy and rjesup.

try submission:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=6662c7085516
Assignee: nobody → fs.in.nccu
Attachment #8688817 - Attachment is obsolete: true
Flags: needinfo?(fs.in.nccu)
Attachment #8689298 - Flags: review+
(Assignee)

Updated

3 years ago
Keywords: checkin-needed

Comment 10

3 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/ac214b95a784
Status: NEW → RESOLVED
Last Resolved: 3 years ago
status-firefox45: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla45
(Reporter)

Comment 11

3 years ago
Amazing =p. Thanks all
You need to log in before you can comment on or make changes to this bug.