812098 - Audio tag does not support blob which passed by web activity

Status: RESOLVED WORKSFORME

(Core :: Audio/Video, defect)

Description

[Dominic Kuo [:dkuo]]

In B2G, after we received an audio file, bluetooth app will pass the blob to Music app by using web activity.
Music will use URL.createObjectURL(blob) to create url then assign to an audio tag, but this resulted in below.

Error log:

[JavaScript Warning: "HTTP "Content-Type" of "application/octet-stream" is not supported. Load of media resource blob:b15c8f5c-0af8-4eb7-bd0e-f41a887247bf failed."

Comment 1

[David Flanagan [:djf]]

Dominic: have you tested the blob.type property before passing it to the activity and after receiving it?  Maybe it is changing when passed.

Are you using DeviceStorage('sdcard') to get the blob from the filesystem? I wonder if using "sdcard" instead of "music" changes the types of the returned files...

One possible workaround would be to read the contents of the untyped blob into an array buffer (with FileReader) and then create a new blob with the correct type (with something like new Blob([arraybuffer], "audio/mp3") and see that will play.

Of course that doesn't address the underlying bug in activities or device storage.

Comment 2

[[:fabrice] Fabrice Desré]

Paul, can you take a look at this?

Comment 3

[Paul Adenot (:padenot)]

Sure. Can I have proper STR and/or a test case? 

The problem here is that the media code does not know which decoder to create for this media, since the decoder decision process is based solely on the mime type.

I guess we can fix this by either:

- Make |createBlobUrl| sniff for a mimetype when the |contentType| is empty or "application/octet" stream or fix the sniffing if it does sniff (I would guess it does not)
- Assign the mimetype as comment 1 mentions so the media code can choose which decoder to create

Comment 4

[Dominic Kuo [:dkuo]]

(In reply to David Flanagan [:djf] from comment #1)
> Dominic: have you tested the blob.type property before passing it to the
> activity and after receiving it?  Maybe it is changing when passed.

First of all, I used a mp3 to test

before passing the blob: blob.type is audio/mpeg
after receiving the blob: blob.type is unknown (empty)
so yes, it was changed after it passed by web activity.

> Are you using DeviceStorage('sdcard') to get the blob from the filesystem? I
> wonder if using "sdcard" instead of "music" changes the types of the
> returned files...

I am using DeviceStorage('sdcard') and DeviceStorage('music'),
and both of them return the correct mime type: audio/mpeg

> One possible workaround would be to read the contents of the untyped blob
> into an array buffer (with FileReader) and then create a new blob with the
> correct type (with something like new Blob([arraybuffer], "audio/mp3") and
> see that will play.

> Of course that doesn't address the underlying bug in activities or device
> storage.

I believe the blob(file) object which return by DeviceStorage maybe changed 
after passed by web activity, so that's why I cannot get blob.type,
and also why audio tag doesn't support it?

Comment 5

[Dominic Kuo [:dkuo]]

Paul, Here is a sample code that I tested myself:

var storage = navigator.getDeviceStorage('sdcard');
var getRequest = storage.get(filePath);

getRequest.onsuccess = function() {
  var a = new MozActivity({
    name: 'open',
    data: {
      blob: getRequest.result,
      filename: filePath
    }
  });
};

And the app that receives the web activity will have:

window.onload = function() {
  navigator.mozSetMessageHandler('activity', handleOpenActivity);
};

function handleOpenActivity(request) {
  var blob = request.source.data.blob;
  ...
}

(In reply to Paul ADENOT (:padenot) from comment #3)
> Sure. Can I have proper STR and/or a test case? 
> 
> The problem here is that the media code does not know which decoder to
> create for this media, since the decoder decision process is based solely on
> the mime type.
> 
> I guess we can fix this by either:
> 
> - Make |createBlobUrl| sniff for a mimetype when the |contentType| is empty
> or "application/octet" stream or fix the sniffing if it does sniff (I would
> guess it does not)
> - Assign the mimetype as comment 1 mentions so the media code can choose
> which decoder to create

Comment 6

[Paul Adenot (:padenot)]

So, this is a web activity problem, I'm not sure I'm competent to patch this. Also, I don't have a device to test yet (I have ordered one, though). We could hack our way around that, though.

Comment 7

[Paul Adenot (:padenot)]

Alright, after a fair bit of debugging, we loose the properties we want exactly here [1]. No idea why, though, this is pretty far from the media code.

[1]: http://mxr.mozilla.org/mozilla-central/source/content/base/src/nsFrameMessageManager.cpp#588

Comment 8

[[:fabrice] Fabrice Desré]

Can this be tested again now that bug 811615 landed with fixes to the way we wrap blobs when sending them from chrome to content?

Comment 9

[Paul Adenot (:padenot)]

Fabrice, thanks, my testcase now works.

dkuo, you might want to try again after either applying the patches in bug 811615 on a local gecko tree, or after those patches land on the tree you usually use.

Comment 10

[Dominic Kuo [:dkuo]]

Hey Paul, sorry for the late reply, I have tested my testcase again and it works now!
Thanks for clarifying this issue.