836814 - Cannot seek in media resources accessed via JAR/APP protocol URIs

Status: NEW

(Core :: Networking, defect, P5)

Description

[Pawel Wojciechowski]

Attachment: package.zip

User Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.56 Safari/537.17

Steps to reproduce:

1. Download attached zip file with "Audio test" packaged application (application.zip + package.manifest)
2. Unzip, install and execute the app
3. Tap big red rectangle to start playing OGG audio file from 60 seconds playback position


Actual results:

Audio file is played from the very beginning


Expected results:

Audio file should be played from 60 seconds position (file is 1 minute 54 seconds long)

Comment 1

[Pawel Wojciechowski]

Tested on b2g device (unagi).

Comment 2

[cajbir (:cajbir)]

This happens on all platforms. I'm pretty sure the result is expected due to races in the way media elements is spec'd due to the following:

document.getElementById("play").onclick = function() {
    audio.play();
    audio.currentTime = 60;
}


I'll follow up with details later to confirm if that's the case.

The usual way to set a starting time is (and I've tested these work):

1) Media fragments (note the URL):

   <audio src="audio.ogg#t=60" id="test" controls="controls"></audio>

2) Set the current time after metadata is loaded:

    audio.addEventListener("loadedmetadata", function() {
        console.log("EVENT loadedmetadata");
        audio.currentTime = 60;
    });

Comment 3

[cajbir (:cajbir)]

Missed one. In the case of wanting to play from 60 seconds after pressing play:

document.getElementById("play").onclick = function() {
    audio.currentTime = 60;
}

audio.addEventListener("seeked", function() {
    console.log("EVENT seeked");
    audio.play();
});

Comment 4

[ZieL]

Hi Chris,

thank you for your response. But I'm afraid using media fragments for navigation is not an option for us. Because we depending on the current instruction we jump to different positions in the file. Do media fragment work in this case?
I've tried the following solution without media fragments.

document.getElementById("play").onclick = function() {
    audio.currentTime = 60;
}

audio.addEventListener("seeked", function() {
    console.log("EVENT seeked");
    audio.play();
});


But I ended up with the same issue. Works perfectly fine in the browser and installed app (not packaged) but not in the packaged app. 


Do you have any other hint we can try?

BTW, regarding your assumption "This happens on all platforms." It's not true. The issue happens only on FFOS and only if the app is packaged.


Thx,
Leo

Comment 5

[cajbir (:cajbir)]

(In reply to ZieL from comment #4)
> BTW, regarding your assumption "This happens on all platforms." It's not
> true. The issue happens only on FFOS and only if the app is packaged.

It wasn't an assumption, I tested your app on desktop and it produced the same issue.

Comment 6

[ZieL]

Hi Chris,

I can't follow you. If I open index.html in FF 18.0.2 everything works as expected.
We can probably take a video if it helps you. Please let me know.

Leo

Comment 7

[Dylan Oliver [:doliver]]

Nominating for v1 on behalf on Nokia -- this is the cause of bug 836643 and for them is happening only when using Maps as a packaged app.

Comment 8

[cajbir (:cajbir)]

If you log the value of "currentTime" in the "seeked" function, what does it show?

document.getElementById("play").onclick = function() {
    audio.currentTime = 60;
}

audio.addEventListener("seeked", function() {
    console.log("EVENT seeked: " + audio.currentTime);
    audio.play();
});

Does seeking work in a package app? If you have a packaged app with the audio file with the standard controls, can you seek? Maybe seeking is broken on whatever channels packaged apps use.

Comment 9

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Minusing for now since it sounds like it probably is something that can be worked around using different application code.

Please renominate if there's something B2G specific, or something that can't be worked around, here.

Comment 10

[Dariusz Siedlecki]

No workaround works for seeking - it's just that seeking is completely broken in the packaged app. The file duration is reported as Infinity and seeking to 60 seconds won't work until the file will physically be played at the 60 second mark.

If we try to seek before, we start playing again at 0 seconds.

The only workaround is to play through the whole file, which is impossible for us to do in this case. Therefore it's still a blocker for us, please set the blocking status back to tef.

We also tried both ways to set the starting time mentioned by Chris in comment 2, they don't work in the packaged app either.

Comment 11

[Jason Smith [:jsmith]]

Renom by comment 10.

Comment 12

[Andrew Overholt [:overholt]]

Since it seems that voice navigation for walking directions (bug 836643) isn't a v1.0.1 feature, it doesn't make sense to hold v1.0.1 for fixing this.

Let's make sure we fix this for v1.1 (I'm setting leo?).

Chris Double, please unassign yourself if you're not the best owner here.

Comment 13

[cajbir (:cajbir)]

Has seeking in files in packaged apps ever worked? Are these files accessed via blob url's (I'm not familiar with how packaged apps work). If so, seeking support in those seemed to get added in bug 730765. Did seeking work then?

Comment 14

[cajbir (:cajbir)]

Possibly bug 785999.

Comment 15

[cajbir (:cajbir)]

Given the package.zip in attachment 708666 [details], how does one install it to test?

Comment 16

[cajbir (:cajbir)]

Oops, the flags got reset somehow when I posted that comment and I don't have permission to change them back.

Comment 17

[Jason Smith [:jsmith]]

(In reply to Chris Double (:doublec) from comment #15)
> Given the package.zip in attachment 708666 [details], how does one install
> it to test?

You would need to install it as a packaged app using the installPackage function. There's also a remote debugging option, but I'm not knowledgeable on that one.

let me quickly throw this up on my server..

Comment 18

[Jason Smith [:jsmith]]

Okay, you should be able to install this here - http://mozqa.com/webapi-permissions-tests/ under "Packaged App Test Case 19"

Comment 19

[cajbir (:cajbir)]

Thanks for that Jason. Packaged apps appear to be stored as zip files and resources accessed via app:// uri's. The underlying JAR channel isn't seekable though which is why Ogg resources in it aren't seeking AFAICT.

Comment 20

[Jason Smith [:jsmith]]

If we aren't getting this in for v1.01, we should probably document this as a known limitation for v1.01 for media content in packaged apps.

Comment 21

[Jason Smith [:jsmith]]

Or should we push back to get this into v1.01? Chris - What do you think? Can we live with this for v1.01 or not?

Comment 22

[cajbir (:cajbir)]

(In reply to Jason Smith [:jsmith] from comment #21)
> Or should we push back to get this into v1.01? Chris - What do you think?
> Can we live with this for v1.01 or not?

I'm not sure if you're asking me or Chris Jones...I'm not really positioned to tell if it's a requirement for V1.01 or not.

Comment 23

[cajbir (:cajbir)]

If you add an Ogg Skeleton Index to the file in the package using OggIndex [1] then you'll be able to seek in the file with the fix in bug 848639 applied. That bug fixes an issue where the duration wasn't being computed from indexed ogg files correctly. By having an index the file duration can be computed and seeking works. 

This doesn't fix the general problem of seeking in JAR resources but may be a suitable workaround until that is done.

[1] https://github.com/cpearce/OggIndex

Comment 24

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Jason might be able to help out here. He's worked a lot on the app protocol handling lately.

Comment 25

[Jason Duell]

> The underlying JAR channel isn't seekable though which is why Ogg resources 
> in it aren't seeking AFAICT.

I'm not clear on how the regular browser is different here than B2G.  nsIChannels in general (and nsIJARChannel in particular) don't have any API for seeking (well, HTTP has range requests, but it's not the same as nsISeekableStream).  How exactly is the seek request getting transmitted to the JARChannel level?

When we open a JARChannel, the JAR cache mmaps the whole thing, which gives us random access.  So we should be able to add seeking to JARChannels if needed. But I'm wondering if the issue isn't something else (are we not using media cache when we should be?)

Comment 26

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

We definitely shouldn't use the media cache for app: URLs. Though that's probably a separate bug.

Comment 27

[cajbir (:cajbir)]

(In reply to Jason Duell (:jduell) from comment #25)
> I'm not clear on how the regular browser is different here than B2G. 
> nsIChannels in general (and nsIJARChannel in particular) don't have any API
> for seeking (well, HTTP has range requests, but it's not the same as
> nsISeekableStream).  How exactly is the seek request getting transmitted to
> the JARChannel level?

It's not getting transmitted. We have implementations of MediaResource (in content/media) for the things that require different support (eg, File Channels vs HTTP Channels). We have nothing for JAR. 

> When we open a JARChannel, the JAR cache mmaps the whole thing, which gives
> us random access.  So we should be able to add seeking to JARChannels if
> needed. But I'm wondering if the issue isn't something else (are we not
> using media cache when we should be?)

So this sounds like we just need a MediaResource implementation that knows about and can access the JAR cache stuff.

Comment 28

[Jason Duell]

> the JAR cache mmaps the whole thing, which gives us random access.

We don't have client API access for random access right now, but if it makes the MediaResource impl easier you could probably implement a new nsSeekableChannel.idl and make nsJARChannel implement it.  But perhaps you can just do whatever you're doing now for file channels w/o needing random access at the channel level.

Comment 29

[cajbir (:cajbir)]

(In reply to Jason Duell (:jduell) from comment #28)
> nsSeekableChannel.idl and make nsJARChannel implement it.  But perhaps you
> can just do whatever you're doing now for file channels w/o needing random
> access at the channel level.

For File Channels we get the ISeekableStream from the InputStream. I'm assuming this won't work with Jar Channels as they are currently.

Comment 30

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Yeah, I don't think SeekableChannel is what we want. I think we want to leverage nsISeekableStream here somehow.

Chris: Are loads currently going through the media cache? If so, why isn't that causing us to be able to rewind here?

If we're not going through the media cache (which would be great if that's the case), then is the solution here to make sure that we call nsIChannel->Open() and make sure that that returns an nsISeekableStream from the JAR channel?

Comment 31

[Jason Duell]

> solution here to make sure that we call nsIChannel->Open() and make sure that 
> that returns an nsISeekableStream from the JAR channel?

Yeah, that's bug 849049, already filed as blocking this bug :)

> Are loads currently going through the media cache? If so, why isn't that 
> causing us to be able to rewind here?

Sounds like we are using media cache, from chatting on IRC with Chris.  We seem to need the duration field early on in the process, which is why we need seek.  Perhaps Chris can explain more.

There's a possible workaround to all of this work: see bug 836643 comment 14

Comment 32

[cajbir (:cajbir)]

(In reply to Jonas Sicking (:sicking) from comment #30)
> Chris: Are loads currently going through the media cache? If so, why isn't
> that causing us to be able to rewind here?

The stream is being identified as 'infinite' or 'live', making seeking not possible. If we get the duration then we can seek. We usually need to seek to get the duration with Ogg files (Unless they have an index or provide a special HTTP header).

> If we're not going through the media cache (which would be great if that's
> the case), then is the solution here to make sure that we call
> nsIChannel->Open() and make sure that that returns an nsISeekableStream from
> the JAR channel?

Whether or not we go through the media cache, the nsISeekableStream approach seems to be the best idea. We can then fix up MediaResource to make use of this.

Comment 33

[AndyT]

(we already added the indices for the next release of the Maps app, so that would probably also solve it for the Maps app)

Comment 34

[Jason Duell]

doublec: soo many questions :)

1) Am I right that this no longer blocks bug 836643 (Here Maps audio), since it will now provide OGG indexes, and once bug 848639 lands we won't need to seek in JAR files (at least for the maps app case)?

2) If #1 is true, can we unmark this as a leo+ blocker? I.e. is there anything else we know of that requires JAR seeking in the leo timeframe?  That will give me a sense of how soon we need bug 849049 fixed too.

3) Why does bug 848639 (OGG indexes) block this bug? I thought reading indexes was an alternative to seeking to get the duration.

Comment 35

[Jason Duell]

And also, does this bug still need to block b2g-maps meta-bug?

Comment 36

[cajbir (:cajbir)]

(In reply to Jason Duell (:jduell) from comment #34)
> 1) Am I right that this no longer blocks bug 836643 (Here Maps audio), since
> it will now provide OGG indexes, and once bug 848639 lands we won't need to
> seek in JAR files (at least for the maps app case)?

The original submitted will need to confirm if this is the case. I've added a 'needsinfo' to find out.

> 
> 2) If #1 is true, can we unmark this as a leo+ blocker? I.e. is there
> anything else we know of that requires JAR seeking in the leo timeframe? 
> That will give me a sense of how soon we need bug 849049 fixed too.

I do not know of anything.

> 
> 3) Why does bug 848639 (OGG indexes) block this bug? I thought reading
> indexes was an alternative to seeking to get the duration.

Because the bug was originally "Can't see in Ogg files in application packages" and that bug fixes part of it.

Comment 37

[Dariusz Siedlecki]

We tried this on build 20130315070220 and it's still blocking bug 836643. We'll verify this on the next OTA update and provide the necessary info.

Comment 38

[Jason Duell]

Dariusz: now that bug 848639 has landed on the b2g18 tree we may start to see bug 836643 fixed (if the next version of here maps is being used).  See bug 836643 comment 29.  If/when we see that working we should probably leo- this bug.

Comment 39

[Dariusz Siedlecki]

I can confirm that bug 848639 is fixed in the latest OTA, so we'll be pushing an update to Here Maps with the indexed OGG file to fix bug 836643.

From our side you can leo- this bug.

Comment 40

[Jason Smith [:jsmith]]

Renom per comment 39

Comment 41

[Lukas Blakk [:lsblakk] use ?needinfo]

Removing leo? as per comment 39.

Comment 42

[Pawel Wojciechowski]

This issue is already resolved.

Comment 43

[Jason Duell]

Not fixed--we still can't seek in JAR files--but it no longer blocks maps, so it doesn't need to be fixed any time soon AFAICT.

Comment 44

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

This isn't a video issue.

Comment 45

[Firefox Bug Husbandry Bot]

Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258