1007223 - AudioContext::DecodeAudioData passes ArrayBuffer data across threads without transferring it

Status: VERIFIED FIXED

(Core :: Web Audio, defect)

Description

[Jeff Walden [:Waldo]]

void
AudioContext::DecodeAudioData(const ArrayBuffer& aBuffer,
                              DecodeSuccessCallback& aSuccessCallback,
                              const Optional<OwningNonNull<DecodeErrorCallback> >& aFailureCallback)
{
  // Sniff the content of the media.
  // Failed type sniffing will be handled by AsyncDecodeMedia.
  nsAutoCString contentType;
  NS_SniffContent(NS_DATA_SNIFFER_CATEGORY, nullptr,
                  aBuffer.Data(), aBuffer.Length(),
                  contentType);

  nsRefPtr<DecodeErrorCallback> failureCallback;
  if (aFailureCallback.WasPassed()) {
    failureCallback = &aFailureCallback.Value();
  }
  nsRefPtr<WebAudioDecodeJob> job(
    new WebAudioDecodeJob(contentType, this, aBuffer,
                          &aSuccessCallback, failureCallback));
  mDecoder.AsyncDecodeMedia(contentType.get(),
                            aBuffer.Data(), aBuffer.Length(), *job);
  // Transfer the ownership to mDecodeJobs
  mDecodeJobs.AppendElement(job);
}

When |aBuffer| is passed to |WebAudioDecodeJob|, the underlying object is cached in the decode job.  We pass in the length and the raw data from the ArrayBuffer into AsyncDecodeMedia.  In there we create a |new MediaDecodeTask(aContentType, aBuffer, aLength, aDecodeJob, mThreadPool);|, then |CreateReader()| on it.  That creates a |BufferMediaResource| passing in the raw buffer pointer.  And |BufferMediaResource| does all sorts of stuff to what's in the pointer/length pair.  Boo-urns.  Or am I misreading this somehow?

(Of course, this is all problematic because the buffer passed into decodeAudioData is user-controlled and can be neutered at any point after the call completes [but before the decoding task stuff completes].  Or the data could be moved around there's sufficiently little of it and someone does something that requests a *stable* address for the data, causing the data to be copied into heap-allocated permanent [modulo neutering] storage.  Just to absolutely clear to any readers not fully on top of this.)

Comment 1

[Boris Zbarsky [:bzbarsky]]

Looking at the spec at https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#dfn-decodeAudioData it looks like we're supposed to neuter the arraybuffer, no?

We're also supposed to un-neuter.... but that seems lossy, since neutering will affect all the views, no?  I think this spec is bunk.  :(

Comment 2

[Boris Zbarsky [:bzbarsky]]

I guess http://webaudio.github.io/web-audio-api/#widl-AudioContext-decodeAudioData-Promise-AudioBuffer--ArrayBuffer-audioData-DecodeSuccessCallback-successCallback-DecodeErrorCallback-errorCallback is the actual current version, but it has the same issue.

Comment 3

[(no longer active)]

Ah, yes...  We should neuter the buffer here indeed.  Paul, can you take this please?

Comment 4

[Boris Zbarsky [:bzbarsky]]

The point is, the spec says the buffer should be "un-neutered" in some cases, but there is no such thing.  Once you neuter a buffer, it stays neutered forever.  So the spec needs to either copy, or neuter and move on.

Comment 5

[(no longer active)]

Oh, sorry, I read the bug too fast.

So I tried to remember how we got to the status quo.  This goes back to October 2012 when I brought up the original issue: <http://lists.w3.org/Archives/Public/public-audio/2012OctDec/0419.html>.  There we convinced Chris Rogers to change the spec, but the only thing being discussed in that thread was neutering the typed array of the AudioBuffer that gets passed to decodeAudioData.  We finally agreed and I filed this spec bug <https://www.w3.org/Bugs/Public/show_bug.cgi?id=20161> where Chris changed the spec, but it seemed like he perhaps mistakenly added the prose about restoring.  But then I filed bug 832098 (which this bug is a dupe of) and apparently didn't pay too much attention to the actual text that went into the spec, and I apologize for that. :(

I think what we need to do here is comment 3, and change the spec to remove the prose about restoring the typed array.

Comment 7

[Paul Adenot (:padenot)]

Attachment: neuter

Ehsan, here is the patch with a test. To me, it seems to work, but I'm not super confident I got it right, because it's kind of my first time doing this kind of stuff.

In particular, I'm not sure if neutering the ArrayBuffer affects the lifetime of the underlying memory: should we free it ourself, or is there some gc magic that'll do it for us?

Also, the spec bug is file here: https://github.com/WebAudio/web-audio-api/issues/319

Comment 8

[(no longer active)]

Comment on attachment 8421013 [details] [diff] [review]
neuter

Review of attachment 8421013 [details] [diff] [review]:
-----------------------------------------------------------------

::: content/media/webaudio/AudioContext.cpp
@@ +446,5 @@
> +  AutoPushJSContext cx(GetJSContext());
> +
> +  size_t length = aBuffer.Length();
> +  JS::RootedObject obj(cx, aBuffer.Obj());
> +  uint8_t* data = static_cast<uint8_t*>(JS_StealArrayBufferContents(cx, obj));

This takes ownership over the underlying buffer, you need to free it using JS_Free() once the WebAudioDecodeJob object is done with this.

::: content/media/webaudio/MediaBufferDecoder.h
@@ +31,5 @@
>    // You may omit both the success and failure callback, or you must pass both.
>    // The callbacks are only necessary for asynchronous operation.
>    WebAudioDecodeJob(const nsACString& aContentType,
>                      dom::AudioContext* aContext,
> +                    JSObject* aBuffer,

I _think_ not using a HandleObject here is fine because you're sticking the value inside a JS::Heap<> immediately, but please make sure that the rooting analysis bot on the try server agrees with me.  ;-)

Comment 9

[Jeff Walden [:Waldo]]

Comment on attachment 8421013 [details] [diff] [review]
neuter

Review of attachment 8421013 [details] [diff] [review]:
-----------------------------------------------------------------

::: content/media/webaudio/MediaBufferDecoder.cpp
@@ +492,3 @@
>    MOZ_COUNT_CTOR(WebAudioDecodeJob);
>  
> +  mArrayBuffer = aBuffer;

Erm.  Why should, or does, this class even accept and store a JSObject* in the first place?  Why can't we just remove it entirely?

There's something fairly scary, seems to me, about storing pointers to JS heap stuff in objects that are accessed on different threads, as here (if memory serves).  Just remove the argument and member field entirely.

::: content/media/webaudio/test/test_decodeAudioDataNeuter.html
@@ +24,5 @@
> +      ok(xhr.response.byteLength == 0, "The source ArrayBuffer is neutered after decoding.");
> +      SimpleTest.finish();
> +    }, function() {
> +      ok(false, "We could not decode the file.");
> +    });

Given the neutering is "supposed" to happen sync, you should test byteLength immediately after the decodeAudioData call as well as in the success/fail continuations.

Note that because of this being a security bug, this test shouldn't land with the patch.  I typically post patch and test as separate attachments with separate review requests, so make clear that sec-approvals and landing requests and such don't conflate the two.

Comment 10

[Paul Adenot (:padenot)]

Attachment: Test that we neuter the ArrayBuffer when using decodeAudioData. r=

The test, split off.

Comment 11

[Paul Adenot (:padenot)]

Attachment: interdiff

I went with Waldo's suggestion, since we don't need for JS to know what we do
with the array anymore, and we free it by hand when we are done.

I could not find JS_Free. js_free and JS_free both exist, I went with the latter
because JS_* are more commonly used in Gecko, passing a nullptr as a first arg,
because that's allowed.

Comment 12

[Paul Adenot (:padenot)]

Attachment: rollup

Rollup of the initial patch + the interdiff.

Comment 13

[Paul Adenot (:padenot)]

Attachment: Test that we neuter the ArrayBuffer when using decodeAudioData. r=

forgot to qref

Comment 14

[(no longer active)]

(In reply to Paul Adenot (:padenot) from comment #11)
> I could not find JS_Free. js_free and JS_free both exist, I went with the
> latter
> because JS_* are more commonly used in Gecko, passing a nullptr as a first
> arg,
> because that's allowed.

Sorry, JS_Free was a typo.  Blame it on the Shift key.

(FWIW, JS_ prefix means external, js_ means internal.)

Comment 15

[Paul Adenot (:padenot)]

Comment on attachment 8421654 [details] [diff] [review]
rollup

[Security approval request comment]
How easily could an exploit be constructed based on the patch?
No idea. Waldo, do you mind to quickly elaborate on that?

Do comments in the patch, the check-in comment, or tests included in the patch paint a bulls-eye on the security problem?
I split off the tests from the patch. There is a one line comment that explains what the new code is doing, we can probably remove it.

Which older supported branches are affected by this flaw?
All of them, I guess, back to Firefox 25

If not all supported branches, which bug introduced the flaw?
N/A

Do you have backports for the affected branches? If not, how different, hard to create, and risky will they be?
This code did not move much, I would expect that the patch applies on all branches.

How likely is this patch to cause regressions; how much testing does it need?
There is a change in behavior, we now aligns with the spec. I would not expect regressions.

Comment 16

[Jeff Walden [:Waldo]]

(In reply to Paul Adenot (:padenot) from comment #15)
> [Security approval request comment]
> How easily could an exploit be constructed based on the patch?
> No idea. Waldo, do you mind to quickly elaborate on that?

No idea.  Depends on exactly what audio decoding entails, and that I haven't the slightest idea about.  Once I determine something is memory-unsafe, I tend not to investigate much beyond that point.  If I just want to fix bugs, there's just not much reason to deeply consider weaponization feasibility, except for a new intellectual challenge.

> How likely is this patch to cause regressions; how much testing does it need?
> There is a change in behavior, we now aligns with the spec. I would not
> expect regressions.

There could be pages expecting the passed-in buffer to not be neutered, given no implementation neuters now (per a skim of WebKit/Blink code).  Pages that expect non-neutering are Doing It Wrong.  I have no idea how likely it is any would be doing this.  If we're super-super-paranoid, we could decode a copy of the data in the ArrayBuffer and not neuter, leaving the neutering fix to trunk.  Such a hackaround would double memory consumption; it's probably not worth the extra effort, given Web Audio is still bleeding-edge tech.

Comment 17

[Paul Adenot (:padenot)]

(In reply to Jeff Walden [:Waldo] (remove +bmo to email) from comment #16)
> > How likely is this patch to cause regressions; how much testing does it need?
> > There is a change in behavior, we now aligns with the spec. I would not
> > expect regressions.
> 
> There could be pages expecting the passed-in buffer to not be neutered,
> given no implementation neuters now (per a skim of WebKit/Blink code). 
> Pages that expect non-neutering are Doing It Wrong.  I have no idea how
> likely it is any would be doing this.  If we're super-super-paranoid, we
> could decode a copy of the data in the ArrayBuffer and not neuter, leaving
> the neutering fix to trunk.  Such a hackaround would double memory
> consumption; it's probably not worth the extra effort, given Web Audio is
> still bleeding-edge tech.

Note that Chrome people agreed that neutering should be done (It was a google person merging my patch to the spec). They just haven't written the code yet.

I don't think we need to implement the work around you mention. I can't really think of a use case for that.

Comment 18

[Al Billings [:abillings - ex-MoCo]]

Comment on attachment 8421654 [details] [diff] [review]
rollup

sec-approval+ for trunk. We should get beta and aurora patches made and nominated once it is in trunk.

Comment 19

[Paul Adenot (:padenot)]

Attachment: imported patch fix-test

I think the only thing that relied on this behaviour was our test :-)

Comment 20

[(no longer active)]

Comment on attachment 8423985 [details] [diff] [review]
imported patch fix-test

Review of attachment 8423985 [details] [diff] [review]:
-----------------------------------------------------------------

Haha, this is embarrassing.  Sorry!  ;-)

Comment 21

[Paul Adenot (:padenot)]

Attachment: patch

JSAPI changes bitrot the patch :-(

Comment 22

[Paul Adenot (:padenot)]

To be specific, this just changes the way to get the context and to choose the compartment.

Comment 23

[(no longer active)]

Comment on attachment 8426168 [details] [diff] [review]
patch

Review of attachment 8426168 [details] [diff] [review]:
-----------------------------------------------------------------

::: content/media/webaudio/AudioContext.cpp
@@ +431,5 @@
>                                const Optional<OwningNonNull<DecodeErrorCallback> >& aFailureCallback)
>  {
> +  AutoJSAPI jsapi;
> +  JSContext* cx = jsapi.cx();
> +  JSAutoCompartment comp(cx, aBuffer.Obj());

Boris can you please double check this?

Comment 24

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 8426168 [details] [diff] [review]
patch

I _think_ it's right, but this stuff is pretty new... Bobby, is this how it's meant to be used?

Comment 25

[Bobby Holley (:bholley)]

Comment on attachment 8426168 [details] [diff] [review]
patch

Review of attachment 8426168 [details] [diff] [review]:
-----------------------------------------------------------------

::: content/media/webaudio/AudioContext.cpp
@@ +431,5 @@
>                                const Optional<OwningNonNull<DecodeErrorCallback> >& aFailureCallback)
>  {
> +  AutoJSAPI jsapi;
> +  JSContext* cx = jsapi.cx();
> +  JSAutoCompartment comp(cx, aBuffer.Obj());

Please call the JSAutoCompartment |ac|. AutoJSAPI usage looks good otherwise.

Comment 26

[Paul Adenot (:padenot)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e08a482fcead

Comment 27

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/e08a482fcead

Comment 28

[Lukas Blakk [:lsblakk] use ?needinfo]

Let's please get an uplift nomination asap, this needs to get into next week's Beta.

Comment 29

[Paul Adenot (:padenot)]

Comment on attachment 8426168 [details] [diff] [review]
patch

[Approval Request Comment]
Bug caused by (feature/regressing bug #): Landing of web audio (or maybe shortly after)
User impact if declined: not sure 
Testing completed (on m-c, etc.): m-c, locally, has test to be landed separately, but green everywhere 
Risk to taking this patch (and alternatives if risky): little behavioral change, other implementers have agreed to change their behavior as well.  
String or IDL/UUID changes made by this patch: none

Comment 30

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/15ec20e99da2
https://hg.mozilla.org/releases/mozilla-beta/rev/5ecccdf09f33

Comment 31

[Ryan VanderMeulen [:RyanVM]]

Backed out from beta for bustage.
https://hg.mozilla.org/releases/mozilla-beta/rev/91e40e5e4d26

https://tbpl.mozilla.org/php/getParsedLog.php?id=40470548&tree=Mozilla-Beta

Comment 32

[Paul Adenot (:padenot)]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #31)
> Backed out from beta for bustage.
> https://hg.mozilla.org/releases/mozilla-beta/rev/91e40e5e4d26
> 
> https://tbpl.mozilla.org/php/getParsedLog.php?id=40470548&tree=Mozilla-Beta

Can you land the patch name "rollup" on beta? The patch got bitrotted while I was writing it, but "rollup" has the right code for beta, I believe.

Comment 33

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/3ca7edf8fb5b

Comment 34

[Ryan VanderMeulen [:RyanVM]]

Still busted.
https://hg.mozilla.org/releases/mozilla-beta/rev/3512c7ff3a7a

https://tbpl.mozilla.org/php/getParsedLog.php?id=40543146&tree=Mozilla-Beta

Comment 35

[Paul Adenot (:padenot)]

Well, that's sad. I'll give you a patch that builds, and push it if you want. Sorry to waste your time with that.

Comment 36

[Ryan VanderMeulen [:RyanVM]]

No problem, it happens :). Feel free to push yourself if you want.

Comment 37

[Paul Adenot (:padenot)]

Attachment: Neuter the ArrayBuffer passed to decodeAudioData. r=

This is the backport for beta. It changes enough that I think it needs another
review.

Comment 38

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/73da1997e6da

Comment 39

[Ryan VanderMeulen [:RyanVM]]

Bustage follow-up:
https://hg.mozilla.org/releases/mozilla-beta/rev/8260d1fb1c34

Comment 40

[(no longer active)]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #39)
> Bustage follow-up:
> https://hg.mozilla.org/releases/mozilla-beta/rev/8260d1fb1c34

post-landing r?padenot

Comment 41

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-b2g28_v1_3/rev/5dea846ef683

This is getting to be a hairy enough backport that I vote wontfix for b2g26. It's EOL in less than 2 weeks anyway.

Comment 42

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/73da1997e6da
https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/8260d1fb1c34

Comment 43

[Paul Adenot (:padenot)]

Looks good, thanks.

Comment 44

[Matt Wobensmith [:mwobensmith][:matt:]]

Confirmed problem on Fx29 using Paul's test.
Verified fixed using builds of Fx30, Fx31 and Fx32 from 2014-06-03.

Comment 45

[Justin Wood (:Callek)]

not able to easily transplant, wontfix on SeaMonkey 2.26.1 (Gecko 29 based)