1185106 - Implement async functions (ES 2017 proposal)

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[mkierski]

Proposal: https://tc39.github.io/ecmascript-asyncawait/

Comment 1

[Till Schneidereit [:till]]

ni? for myself because I promised to provide a bare-bones self-hosted constructor for Promise, to be used as the implementation vehicle here.

Comment 2

[Till Schneidereit [:till]]

Attachment: Part 0: Stub implementation of Promise

This implements the Promise object as an empty stub. It also has stubs for all the (static) methods, but doesn't store the callbacks anywhere. I guess converting it to an extended function and using the reserved slots might just work.

Comment 3

[mkierski]

Attachment: Basic implementation of promises

Comment 4

[Eric Faust [:efaust]]

Comment on attachment 8636818 [details] [diff] [review]
Basic implementation of promises

Punting to till, the selfhosting guru.

Comment 5

[Till Schneidereit [:till]]

Comment on attachment 8636818 [details] [diff] [review]
Basic implementation of promises

Review of attachment 8636818 [details] [diff] [review]:
-----------------------------------------------------------------

For future reference, it's best to keep changes in separate patches, at least while reviewing. Much easier to separate out what you changed on top of my changes. As it is now, I'm effectively reviewing my own changes, too.

Please make sure that all indentation is using spaces, not tabs. Promise.js contains a mix of both.

I added lots of additional comments below that need to be addressed. In general, you should read the self-hosting documentation once again to see what the restrictions are compared to normal JS code. I didn't note all issues in Promise.js because most are repeats of the same issues already mentioned.

Also, the implementation isn't spec-compliant in various ways. I don't know how important that is for your project, you should discuss that with efaust. At the very least, I'd like to see comments about deviations from the spec, so we're aware of them.

::: js/src/builtin/Promise.cpp
@@ +20,5 @@
> +static const JSFunctionSpec promise_static_methods[] = {
> +    JS_SELF_HOSTED_FN("all", "Promise_all", 1, 0),
> +    JS_SELF_HOSTED_FN("race", "Promise_race", 1, 0),
> +    JS_SELF_HOSTED_FN("reject", "Promise_reject", 1, 0),
> +    JS_SELF_HOSTED_FN("init", "Promise_init", 2, 0),

You only need to install functions on an object if they're exposed to content and should be called on that object. That's not the case for Promise_init: it's neither supposed to be exposed to content (but is, by this line), nor should you ever call it via Promise.init. (But see below for more issues with this function.)

@@ +34,5 @@
> +    HandleValue fn = args.get(0);
> +
> +    if (!IsCallable(fn)) {
> +        JS_ReportErrorNumber(cx, GetErrorMessage, nullptr, JSMSG_NOT_FUNCTION,
> +                             "Promise constructor argument");

Please test which errors the current implementation in Gecko throws here and match those. (Hint, test for too few arguments in addition to IsCallable, and fix the error message.)

@@ +43,5 @@
> +    JSObject* obj = NewBuiltinClassInstance(cx, &PromiseObject::class_);
> +    if (!obj)
> +        return false;
> +
> +    Rooted<JSObject*> rt(cx, obj);

This can be RootedObject. Also, you can get rid of the JSObject* above and just use
RootedObject obj(cx, NewBuiltinClassInstance(cx, &PromiseObject::class_));
if (!obj)
    return false;

(and change all usages of rt to obj below.)

@@ +45,5 @@
> +        return false;
> +
> +    Rooted<JSObject*> rt(cx, obj);
> +    JSObject* promiseConstructor = JS_GetConstructor(cx, rt);
> +    Rooted<JSObject*> rtPromiseConstructor(cx, promiseConstructor);

Same here: RootedObject, and no intermediary JSObject*.

@@ +56,5 @@
> +    JS::AutoValueVector argValues(cx);
> +    argValues.append(promise);
> +    argValues.append(fn);
> +    HandleValueArray arr(argValues);
> +    JS_CallFunctionName(cx, rtPromiseConstructor, "init",

There are some problems with this: the most important one is that content could just change the way promises are initialized by doing `Promise.init = myEvilFunction`. However, it's also quite a bit slower than it'd need to be.

Instead, the instance should be fully initialized in C++ code. The way things are set up right now, that'd be quite annoying: defining properties on JSObjects from C++ sucks a bit. However, we shouldn't do that anyway: right now, the internal state of a promise could just be changed by content code by doing assigning to the _state, _value, or _deferreds properties. (Or changing their internal values.)

So how should we do this instead, you ask. By using reserved slots. A good example for how to use those and initialize instances with enough of them is the ArrayIterator in builtin/Array.js. Take a look at CreateArrayIteratorAt in that file: it calls the intrinsic NewArrayIterator (defined and installed in SelfHosting.cpp). That returns an object with 3 reserved slots. Check ArrayIteratorObject::class_ in jsiter.cpp for how that works. Then observe how the UnsafeSetReservedSlot intrinsic is used to set the slots. Reserved slots can also be set from C++, which is what you want to do in the constructor. Try to understand how the ArrayIterator implementation works: it'll help you understand how to use the reserved slots from both C++ and self-hosted JS code.

::: js/src/builtin/Promise.js
@@ +2,5 @@
> + * License, v. 2.0. If a copy of the MPL was not distributed with this
> + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
> +
> +function Promise_all(promises) {
> +		return new Promise(function (resolve, reject) {

As described above, this needs to use an intrinsic function for calling the C++-implemented constructor, similar to how things are done for ArrayIterator.

@@ +5,5 @@
> +function Promise_all(promises) {
> +		return new Promise(function (resolve, reject) {
> +			if (promises.length === 0) return resolve([]);
> +			var remaining = promises.length;
> +			function res(i, val) {

Please give all functions and variables speaking names. If they're a bit longer, that's ok.

@@ +7,5 @@
> +			if (promises.length === 0) return resolve([]);
> +			var remaining = promises.length;
> +			function res(i, val) {
> +				try {
> +					if (val && (typeof val === 'object' || typeof val === 'function')) {

There's an `IsCallable` intrinsic which should be used here. Somewhat irritatingly, not all callables are functions.

@@ +10,5 @@
> +				try {
> +					if (val && (typeof val === 'object' || typeof val === 'function')) {
> +						var then = val.then;
> +						if (typeof then === 'function') {
> +							then.call(val, function (val) { res(i, val) }, reject);

In self-hosted code, using .call and .apply (and pretty much all other method calls) is forbidden in 99.9% of all cases. This code would break if content code overrode Function.prototype.call. Use `callFunction` instead.

@@ +23,5 @@
> +					reject(ex);
> +				}
> +			}
> +			for (var i = 0; i < promises.length; i++) {
> +				res(i, promises[i]);

Can't you just inline the contents of `res` into this loop?

@@ +28,5 @@
> +			}
> +		});
> +}
> +
> +function Promise_race(values) {

This doesn't really implement Promise.prototype.race, right? It's somewhat ok if the implementation is just enough to get async/await working, but if so, please add a comment explaining why it's done this way. If race isn't needed to implement async/await at all, I'd much rather see this replaced with a stub that just throws an exception saying that it's not really implemented.

@@ +31,5 @@
> +
> +function Promise_race(values) {
> +    return new Promise(function (resolve, reject) {
> +        for (var i = 0, len = values.length; i < len; i++) {
> +            values[i].then(resolve, reject);

Same here: no method calls without callFunction.

@@ +43,5 @@
> +    });
> +}
> +
> +function Promise_resolve(value) {
> +    if (value && typeof value === 'object' && value.constructor === Promise) {

`value.constructor === Promise` probably isn't the right check: it can be fooled by changing value.__proto__, Promise.prototype.constructor, or value.constructor. Check the specification for how this check is supposed to work.

@@ +57,5 @@
> +  return this.then(null, onRejected);
> +}
> +
> +function asap(cb) {
> +  cb();

This implementation of asap will definitely not be sufficient for your needs. It's going to be very important to delay running `cb` until the next turn of the event loop. That's probably the most annoying part of doing all this in SpiderMonkey. (Though it might also be easier than I think.) Talk to efaust or Waldo about how to approach this.

@@ +93,5 @@
> +
> +MakeConstructible(Promise, {
> +  then: Promise_then,
> +  catch: Promise_catch
> +});

As explained above, you can't use this constructor but have to use an intrinsic as is done for ArrayIterator.

There's another reason for this that I didn't explain before: a promise created this way, using `new Promise` in self-hosted code will not have the same constructor as one created using `new Promise` in content code. That's because the `Promise` constructor available to content code isn't the one you have here. Instead it's the one created from PromiseObject::class_. One consequence here is that the `constructor` properties would be different, als would be __proto__.

::: js/src/gc/StoreBuffer.cpp
@@ +105,5 @@
>  
> +// template struct StoreBuffer::MonoTypeBuffer<StoreBuffer::ValueEdge>;
> +// template struct StoreBuffer::MonoTypeBuffer<StoreBuffer::CellPtrEdge>;
> +// template struct StoreBuffer::MonoTypeBuffer<StoreBuffer::SlotsEdge>;
> +// template struct StoreBuffer::MonoTypeBuffer<StoreBuffer::WholeCellEdges>;

This needs review from someone else. I commented out these lines to get things to compile, but really, that's not the right solution. My guess is that adding a new source file changed which cpp files get compiled together, causing issues here. Talk to efaust or waldo about how best to deal with this.

Comment 6

[Terrence Cole [:terrence]]

Attachment: fix_unified_bustage-v0.diff

This patch fixes the unified bustage.

The issue here is that we may or may not need the template instantiations depending one what code the compiler sees in the translation unit containing the instantiations.

Comment 7

[Eric Faust [:efaust]]

Comment on attachment 8640005 [details] [diff] [review]
fix_unified_bustage-v0.diff

Review of attachment 8640005 [details] [diff] [review]:
-----------------------------------------------------------------

Seems like it should fix the problem. There's no way commenting out those specializations can be correct, so I'm willing to believe this should be the way forward.

Comment 8

[mkierski]

Attachment: Implemented a polyfill for promises

Comment 9

[Eric Faust [:efaust]]

Comment on attachment 8640126 [details] [diff] [review]
Implemented a polyfill for promises

Review of attachment 8640126 [details] [diff] [review]:
-----------------------------------------------------------------

::: js/src/moz.build
@@ +343,4 @@
>  SOURCES += [
>      'builtin/RegExp.cpp',
>      'frontend/Parser.cpp',
> +    'gc/StoreBuffer.cpp',

At the very least, the patch that terrence wrote should land seperately. This fix has nothing to do with the patch for the polyfill.

Comment 10

[mkierski]

Attachment: Fixed (rebased) implementation of promises

Note: It is dependent on terrence's patch (does not include it)

Comment 11

[Till Schneidereit [:till]]

Comment on attachment 8640212 [details] [diff] [review]
Fixed (rebased) implementation of promises

Review of attachment 8640212 [details] [diff] [review]:
-----------------------------------------------------------------

I've been thinking about this some more and would really prefer for this to not include my original patch, either. The reason is that I'm deeply uncomfortable with reviewing my own code.

The way this works is that you first apply terrence's patch, then mine, then yours. Change the descriptions to "Bug 1185106 - Part N: description". That way, it's clear which order they have to be applied in without additional comments.

Clearing review for now, please re-request for the new patch.

Comment 12

[mkierski]

Attachment: Part 1: (terrence) fixes for the StoreBuffer

Comment 13

[mkierski]

Attachment: Part 2: (till) Promises boilerplate

Comment 14

[mkierski]

Attachment: Part 3: Promises implementation

Comment 15

[Eric Faust [:efaust]]

Comment on attachment 8640670 [details] [diff] [review]
Part 2: (till) Promises boilerplate

Review of attachment 8640670 [details] [diff] [review]:
-----------------------------------------------------------------

I believe this.

::: js/src/gc/StoreBuffer.cpp
@@ +105,5 @@
>  
> +// template struct StoreBuffer::MonoTypeBuffer<StoreBuffer::ValueEdge>;
> +// template struct StoreBuffer::MonoTypeBuffer<StoreBuffer::CellPtrEdge>;
> +// template struct StoreBuffer::MonoTypeBuffer<StoreBuffer::SlotsEdge>;
> +// template struct StoreBuffer::MonoTypeBuffer<StoreBuffer::WholeCellEdges>;

We should not still do this. We even have a patch in this series so that it's not necessary.

Comment 16

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/48a35f84fb9d

Comment 17

[Till Schneidereit [:till]]

Attachment: Part 1: Exclude StoreBuffer.cpp from unified build to prevent build bustage

Fixed up the patch format for part 1 and landed:
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/48a35f84fb9d

Comment 18

[Till Schneidereit [:till]]

Attachment: Part 2: Stub implementation of Promise

Removed the StoreBuffer.cpp changes from part 2, carrying review.

I didn't land this because it doesn't make any sense whatsoever. We might fold it in with part 3 in the end, not sure yet.

Comment 19

[Till Schneidereit [:till]]

Comment on attachment 8640672 [details] [diff] [review]
Part 3: Promises implementation

Review of attachment 8640672 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks for the new patch. This is getting closer, but still has quite a few issues. Please read through the comments carefully and try to address them all. I'm still particularly concerned with the asap stuff and don't fully understand how it works. For the parts of the code that aren't straight-forward, comments would be really important.

::: js/src/builtin/Promise.cpp
@@ +7,5 @@
>  #include "builtin/Promise.h"
>  
>  #include "jscntxt.h"
>  
> +#define PROMISE_STATE_SLOT 0

You can add these defines to builtin/SelfHostingDefines.h, include that in this cpp file, and remove them from here and from Promise.js. That way they're not duplicated and can't accidentally change in only one place.

@@ +33,5 @@
>  PromiseConstructor(JSContext* cx, unsigned argc, Value* vp)
>  {
>      CallArgs args = CallArgsFromVp(argc, vp);
>  
> +    HandleValue fn = args.get(0);

Move this to below the args length check.

@@ +37,5 @@
> +    HandleValue fn = args.get(0);
> +
> +    if (args.length() == 0) {
> +      JS_ReportErrorNumber(cx, GetErrorMessage, nullptr, JSMSG_MORE_ARGS_NEEDED,
> +          "Promise.constructor", "0", "s");

Nit: check the indentations and general formatting in this file, too.

@@ +53,5 @@
>          return false;
> +
> +    RootedObject promiseConstructor(cx, JS_GetConstructor(cx, promise));
> +
> +    JS_SetReservedSlot(promise, PROMISE_STATE_SLOT, NullValue()); // state

See the comment for reject in Promise.js.

@@ +55,5 @@
> +    RootedObject promiseConstructor(cx, JS_GetConstructor(cx, promise));
> +
> +    JS_SetReservedSlot(promise, PROMISE_STATE_SLOT, NullValue()); // state
> +    JS_SetReservedSlot(promise, PROMISE_VALUE_SLOT, NullValue()); // value
> +    JS_SetReservedSlot(promise, PROMISE_DEFERREDS_SLOT, ObjectValue(*JS_NewArrayObject(cx, 0))); // deferreds

The result of calling JS_NewArrayObject needs to be checked: it could fail, just as NewBuiltinClassInstance above could.

However, we really don't want to call JS_NewArrayObject here: it creates an array in the content compartment, with the behavior

@@ +56,5 @@
> +
> +    JS_SetReservedSlot(promise, PROMISE_STATE_SLOT, NullValue()); // state
> +    JS_SetReservedSlot(promise, PROMISE_VALUE_SLOT, NullValue()); // value
> +    JS_SetReservedSlot(promise, PROMISE_DEFERREDS_SLOT, ObjectValue(*JS_NewArrayObject(cx, 0))); // deferreds
> +    JS_SetReservedSlot(promise, PROMISE_CONSTRUCTOR_SLOT, ObjectValue(*promiseConstructor)); // constructor

As explained in Promise.js, I don't think you need this slot. If that's correct, please remove it and change the slot count to 3.

@@ +64,5 @@
> +    argValues.append(ObjectValue(*promise));
> +    argValues.append(fn);
> +    HandleValueArray arr(argValues);
> +
> +    RootedPropertyName name(cx, JS_AtomizeAndPinString(cx, "Promise_init")->asAtom().asPropertyName());

Use Atomize(cx, "Promise_init", 12) here. You also need to check the result because the allocation might fail.

@@ +70,5 @@
> +
> +    if (!GlobalObject::getIntrinsicValue(cx, cx->global(), name, &selfHostedFun))
> +        return false;
> +
> +    JS_CallFunctionValue(cx, promise, selfHostedFun, arr, &initRval);

I *think* this should check for the result of calling the function and return false without setting the return value. Then you can also just return true in the last line.

::: js/src/builtin/Promise.h
@@ +23,5 @@
>  
>  } // namespace js
>  
> +bool
> +PromiseConstructor(JSContext* cx, unsigned argc, Value* vp);

Move this into the js namespace.

::: js/src/builtin/Promise.js
@@ +1,2 @@
>  /* This Source Code Form is subject to the terms of the Mozilla Public
> +* License, v. 2.0. If a copy of the MPL was not distributed with this

Nit: undo this change.

@@ +8,2 @@
>  
> +/* This implementation is by no means complete and is using a polyfill.

Nit: please change the comment format to the following:
/**
 * Text
 * more text
 */

@@ +24,3 @@
>  }
>  
> +MakeConstructible(Handler, {});

Use std_Object_create(null) instead of {} here.

@@ +24,5 @@
>  }
>  
> +MakeConstructible(Handler, {});
> +
> +function Promise_setState(state) {

For all these internal utility functions (the Promise_getFoo, etc ones), I think it'd be much better to explicitly pass in the promise and then just call them: `Promise_getState(promise)`

Also, this "state" is whether the promise was resolved or not, right? In that case, change it to "Promise_setIsResolved", and the getter to "Promise_getIsResolved". (Note: I think I misunderstood this, so the name is probably ok as-is. See the comment for Promise_reject below.)

@@ +30,4 @@
>  }
>  
> +function Promise_getState() {
> +  return UnsafeGetReservedSlot(this, PROMISE_STATE_SLOT);

Use UnsafeGetObjectFromReservedSlot for all object slots and UnsafeGetBooleanFromReservedSlot for bools.

@@ +53,5 @@
> +  return UnsafeGetReservedSlot(this, PROMISE_CONSTRUCTOR_SLOT);
> +}
> +
> +function Promise_init(promise, fn) {
> +  doResolve(fn, resolve.bind(promise), reject.bind(promise));

I don't think using "bind" is ok here. If content changes Function.prototype.bind, that'll change what happens here. Wrap the functions into closures instead: `function() {callFunction(resolve, promise}`. That's better for performance, too.

@@ +56,5 @@
> +function Promise_init(promise, fn) {
> +  doResolve(fn, resolve.bind(promise), reject.bind(promise));
> +}
> +
> +function Promise_all(promises) {

This is supposed to take an iterable, not just arrays. I think it's ok for now to only deal with arrays (and array-likes), but please add a comment mentioning this.

@@ +57,5 @@
> +  doResolve(fn, resolve.bind(promise), reject.bind(promise));
> +}
> +
> +function Promise_all(promises) {
> +  var results = [];

This code is unsafe: it creates an Array with the __proto__ being the content compartment's Array.prototype. If there's a setter defined for an index, that will be called instead of the value being stored in the index.

Also, I think it's best to only call promises.length once, so use something like
var length = promises.length;
var results = NewDenseArray(length);

And change all usages of promises.length below to use length.

@@ +60,5 @@
> +function Promise_all(promises) {
> +  var results = [];
> +  return NewPromise(function (resolve, reject) {
> +    if (promises.length === 0)
> +			return resolve([]);

Change the indentation to spaces, here and everywhere else. In general, check the formatting of all code once again: there are various alignment issues, which I didn't explicitly call out below.

@@ +68,5 @@
> +        if (valueOrPromise && (typeof valueOrPromise === 'object' || IsCallable(valueOrPromise))) {
> +        	var then = valueOrPromise.then;
> +          if (IsCallable(then)) {
> +            callFunction(then, valueOrPromise,
> +              function (valueOrPromise) { resolveChain(index, valueOrPromise); }, reject);

Nit: horizontally align this argument with the beginning of the first argument.

@@ +72,5 @@
> +              function (valueOrPromise) { resolveChain(index, valueOrPromise); }, reject);
> +            return;
> +          }
> +        }
> +        results[index] = valueOrPromise;

Is it ok to not store the result if an exception was thrown? It probably is because it looks like we're not doing anything with the results in that case, but I'm not sure.

@@ +73,5 @@
> +            return;
> +          }
> +        }
> +        results[index] = valueOrPromise;
> +        if (--remaining === 0) {

Can't you just use `index === length - 1` here and get rid of `remaining`?

@@ +80,5 @@
> +      } catch (ex) {
> +        reject(ex);
> +      }
> +    }
> +    for (var i = 0; i < promises.length; i) {

Nit: no braces needed for single-line bodies.

@@ +87,5 @@
> +  });
> +}
> +
> +function Promise_race(values) {
> +  throw new Error("Promise.race is not implemented");

I'm pretty sure this doesn't actually work: the "Error" constructor almost certainly isn't available. Use ThrowTypeError for now, I would say.

@@ +100,5 @@
> +function Promise_resolve(value) {
> +  if (value && typeof value === 'object' &&
> +    callFunction(Promise_getConstructor, value) === this) {
> +    return value;
> +  }

This doesn't do what (IIUC) it should be doing: Promise_getConstructor would only work on actual instances of Promise. For everything else, it would either crash if no reserved slot with that index exists, or return some bogus value. Both are bad.

Also, if you do something like `MyObject.resolve = Promise.resolve` and then call `MyObject.resolve(someValue)`, you're checking if the result of `Promise_getConstructor(someValue)` equals `MyObject`.

I think it'd be best to implement the IsPromise spec function as an intrinsic. The IsTypedArray intrinsic is a good example for how that'd work. Check the definition of that in SelfHosting.cpp and usages in builtin/TypedArray.js.

We don't currently support subclassing of builtins (but efaust is on the case!), so it's probably ok to just return the value of the IsPromise check returns true.

@@ +109,4 @@
>  }
>  
>  function Promise_catch(onRejected) {
> +  return this.then(null, onRejected);

Per spec, this should use `undefined`, not `null`.

@@ +112,5 @@
> +  return this.then(null, onRejected);
> +}
> +
> +function asap(cb) {
> +  cb();

I still don't understand how this works and actually defers execution. Can you add an explanation? Also, it still executes in the same event loop turn instead of in the next one, right? If that's the case, it definitely needs to change, otherwise the semantics of async/await won't be what we want them to be at all.

@@ +116,5 @@
> +  cb();
> +}
> +
> +function handle(deferred) {
> +  var me = this;

Change this to `promise`, and use it everywhere below instead of `this`.

@@ +117,5 @@
> +}
> +
> +function handle(deferred) {
> +  var me = this;
> +  if (callFunction(Promise_getState, this) === null) {

This check will never succeed because the state is always a boolean. Change `null` to `false`.

@@ +140,5 @@
> +    deferred.resolve(returnValue);
> +  });
> +}
> +
> +function doResolve(fn, onFulfilled, onRejected) {

It's not ideal that we have Promise_resolve, doResolve, and resolve. I don't have good suggestions, but names that make it more clear what each of these does would be good. If that turns out to be hard, at the very least we need comments explaining their roles.

@@ +172,5 @@
> +        return;
> +      }
> +    }
> +    callFunction(Promise_setState, this, true);
> +    callFunction(Promise_setValue, this, newValue);

Can't these two calls be moved into "finale", with the values passed in?

@@ +179,5 @@
> +		callFunction(reject, this, ex);
> +	}
> +}
> +
> +function reject(newValue) {

Same issue was with resolve: multiple functions with very similar names. Please either change the names or add comments.

@@ +180,5 @@
> +	}
> +}
> +
> +function reject(newValue) {
> +  callFunction(Promise_setState, this, false);

Hmm, maybe I'm misunderstanding what the state stores. It looks like it starts out with `null`, indicating that it's unresolved, and can change to `true` for resolved and `false` for rejected? If that's the case, please change it to use enum values. Well, the poor-man's version of them, because we don't have enums in JS. Define values in SelfHostingDefines.h like this and use them here and in Promise.cpp:
#define PROMISE_STATE_UNRESOLVED 0
#define PROMISE_STATE_RESOLVED   1
#define PROMISE_STATE_REJECTED   2

@@ +185,5 @@
> +  callFunction(Promise_setValue, this, newValue);
> +  callFunction(finale, this);
> +}
> +
> +function finale() {

I'm not sure I understand what "finale" means. Perhaps "finalize"? That would still not be an ideal name, as it sounds like some destructor thing, which it isn't, really. I don't have a good suggestion, but this name certainly isn't right.

@@ +187,5 @@
> +}
> +
> +function finale() {
> +  var deferreds = callFunction(Promise_getDeferreds, this);
> +  for (var i = 0, len = deferreds.length; i < len; i) {

Nit: no braces needed.

@@ +188,5 @@
> +
> +function finale() {
> +  var deferreds = callFunction(Promise_getDeferreds, this);
> +  for (var i = 0, len = deferreds.length; i < len; i) {
> +    callFunction(handle, this, deferreds[i]);

Is this guaranteed to not throw? I think so, but am not sure. If it is, add a comment noting that so readers can immediately tell that nothing bad can happen here.

@@ +195,4 @@
>  }
>  
>  function Promise_then(onFulfilled, onRejected) {
> +  var me = this;

Change this to "promise".

::: js/src/vm/SelfHosting.cpp
@@ +1387,5 @@
>            CallNonGenericSelfhostedMethod<Is<LegacyGeneratorObject>>, 2, 0),
>      JS_FN("CallStarGeneratorMethodIfWrapped",
>            CallNonGenericSelfhostedMethod<Is<StarGeneratorObject>>, 2, 0),
>  
>      JS_FN("IsWeakSet",               intrinsic_IsWeakSet,               1,0),

Nit: undo this change.

@@ +1392,2 @@
>      JS_FN("NewDenseArray",           intrinsic_NewDenseArray,           1,0),
> +    JS_FN("NewPromise",              intrinsic_NewPromise, 1, 0),

Nit: please align the arguments as in the lines above.

::: js/src/vm/SelfHosting.h
@@ -17,5 @@
>   * Check whether the given JSFunction is a self-hosted function whose
>   * self-hosted name is the given name.
>   */
>  bool IsSelfHostedFunctionWithName(JSFunction* fun, JSAtom* name);
> -

Nit: undo this change.

Comment 20

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/48a35f84fb9d

Comment 21

[mkierski]

Till, I agree with most of the comments and am about to send a new, fixed patch. I also did a few things to refactor the code. It is actually based on a polyfill taken from GitHub, and it has a few of wrong design patterns (like using null/true/false as tri-state).

I also introduced a quasi-eventloop for deferring calls to setTimeout. I will also want to make setTimeout available to the user as soon as this promise implementation gets accepted.

Moreover, I added some basic test cases for promises.

> Can't you just use `index === length - 1` here and get rid of `remaining`?

I don't think so - there is no guarantee that the promise with the last index in the array will resolve as last. Or maybe I don't understand something?

> This doesn't do what (IIUC) it should be doing: Promise_getConstructor would only work
> on actual instances of Promise. For everything else, it would either crash
> if no reserved slot with that index exists, or return some bogus value. Both are bad.

As far as I read, accessing an unregistered slot just returns an `undefined` value. My implementation of `isPromise` closely follows the one provided in the spec, and it works... are we sure we want to change this?
For now I'm leaving this one as-is.

Comment 22

[mkierski]

Attachment: Part 3: Promises implementation

This fixes all of mentioned issues, except for those which I don't understand/fully agree with. Also I did some work to make code more readable, hopefully enough :)

Comment 23

[mkierski]

Attachment: Part 4: Updated some parser grammar rules

Currently only a part of async/await grammar is supported:
- Async function statements are supported.
- Await expressions are supported (as regular unary expressions).
All other parts of proposal are probably not supported.
Even the supported parts of implementation may not be 100% compliant with
the grammar. This is to be considered a proof-of-concept implementation.

Comment 24

[mkierski]

Attachment: Part 3: Promises implementation

I found a bug, fixed that and added a test case.

Comment 25

[Eric Faust [:efaust]]

Attachment: Refactor DEFFUN in jits

Comment 26

[mkierski]

Comment on attachment 8640988 [details] [diff] [review]
Part 1: Exclude StoreBuffer.cpp from unified build to prevent build bustage

Already landed separately.

Comment 27

[Eric Faust [:efaust]]

Attachment: Refactor DEFFUN in jits

This should fix the baseline stack problems.

Comment 28

[mkierski]

Attachment: Part 4: Parser grammar rules

Comment 29

[mkierski]

Attachment: Part 5: Bytecode and self-hosted wrapper code

Comment 30

[mkierski]

Attachment: Part 6: Refactored JIT DEFFUN (by efaust)

Comment 31

[Eric Faust [:efaust]]

Comment on attachment 8651237 [details] [diff] [review]
Part 6: Refactored JIT DEFFUN (by efaust)

This is not yet r+, if it needs review, we should get it from Jan.

Comment 32

[Eric Faust [:efaust]]

Comment on attachment 8651235 [details] [diff] [review]
Part 5: Bytecode and self-hosted wrapper code

Review of attachment 8651235 [details] [diff] [review]:
-----------------------------------------------------------------

Cancelling review for now. This patch needs to get the right hunks into the right other patches where it should live. Please repost when that happens.

The parser stuff should go to the parser. The promise stuff should go to the promise patch. Please make a new patch for the DEFFUN refactor, as well.

The emitter sections, however, look pretty good! The self-hosted stuff also seems fine. So the async/await parts of this look fine, but it is parts of several patches sewn together.

::: js/src/builtin/AsyncFunctions.js
@@ +3,5 @@
> + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
> +
> +function AsyncFunction_wrap(genFunction) {
> +  var fun = function() {
> +    return AsyncFunction_start(callFunction(std_Function_apply, genFunction, null, arguments));

we talked about adding the try catch here. We should do that.

::: js/src/builtin/Promise.js
@@ +135,2 @@
>      return value;
> +  } // FIXME this does not properly check if 'value' is a Promise, which it should do.

These changes should split out and go with the ones that till is supposed to review, right?

::: js/src/frontend/BytecodeEmitter.cpp
@@ -998,5 @@
>  bool
>  BytecodeEmitter::emitIndex32(JSOp op, uint32_t index)
>  {
>      MOZ_ASSERT(checkStrictOrSloppy(op));
> -

This blank line and the one above where there intentionally. Let's leave them.

@@ +5887,5 @@
>  bool
> +BytecodeEmitter::emitAsyncWrapper(unsigned index) {
> +    JSAtom* atom = Atomize(cx, "AsyncFunction_wrap", 18);
> +    return (atom != nullptr &&
> +        emitAtomOp(atom, JSOP_GETINTRINSIC) &&

This formatting can't stand. We want more whitespace.

if (!atom)
    return false;

return emitAtomOp(...) &&
       ...;

@@ -7995,5 @@
> -      // so currently we just return "true" as a placeholder.
> -      case PNK_AWAIT:
> -        if(!emit1(JSOP_TRUE))
> -            return false;
> -        break;

This seems like it should not have been in a previous patch, but it's fine.

::: js/src/frontend/Parser.cpp
@@ +2290,5 @@
>  {
>      MOZ_ASSERT_IF(kind == Statement, funName);
>  
> +    if (asyncKind == AsyncFunction)
> +      generatorKind = StarGenerator;

4 spaces indenture, please.

@@ +2785,5 @@
>          !report(ParseStrictError, pc->sc->strict(), null(), JSMSG_STRICT_FUNCTION_STATEMENT))
>          return null();
>  
> +    if (asyncKind == AsyncFunction)
> +      generatorKind = StarGenerator;

and here.

@@ +6308,5 @@
>        case TOK_NAME: {
>          TokenKind next;
> +        TokenKind nextSameLine;
> +
> +        #ifdef NIGHTLY_BUILD

These changes should be in the parser patch, and are stale, right?

::: js/src/vm/Interpreter.cpp
@@ +3493,5 @@
>          goto error;
>  }
>  END_CASE(JSOP_DEFVAR)
>  
> +CASE(JSOP_DEFFUN) {

Please leave this on the next line, like it is everywhere else in this file.

Comment 33

[Eric Faust [:efaust]]

Comment on attachment 8651233 [details] [diff] [review]
Part 4: Parser grammar rules

Review of attachment 8651233 [details] [diff] [review]:
-----------------------------------------------------------------

Cancelling review in the face of having to rewrite most of this, as per our discussion. The spec changed.

Comment 34

[mkierski]

Attachment: Part 2: (till) Promises boilerplate

Comment 35

[mkierski]

Attachment: Part 3: Promises implementation

Comment 36

[mkierski]

Attachment: Part 4: Parser grammar rules

Comment 37

[mkierski]

Attachment: Part 5: Bytecode and self-hosted wrapper code

Comment 38

[mkierski]

Attachment: Part 6: Refactored JIT DEFFUN (by efaust)

Comment 39

[Eric Faust [:efaust]]

Comment on attachment 8656864 [details] [diff] [review]
Part 4: Parser grammar rules

Review of attachment 8656864 [details] [diff] [review]:
-----------------------------------------------------------------

OK, a lot of things went right here:

1) The tests look phenomenal. Very careful, with good coverage.
2) Much of the parsing is good and robust.
3) It all mostly does what it claims to do

But a few things went very wrong.

1) There are tons of little stylistic things. The patch needs some more polish before it can go in. I have tried to note as many as I saw below.
2) There are a few small refactoring things that need to be taken care of (the default await throwing case, removing asyncKind from FunctionBox)
3) We cannot increase the size of JSFunction by a word. We have got to find a way to pack the FunctionAsyncKind.
4) I am dubious that we cannot lazily parse an async function, and I'm not seeing a place where we stash that information in the lazyScript, so I suspect we are getting this wrong.

Because of 3 and 4, I have to cancel review for now. We can talk about how to fix these things on Tuesday, when I'm back in the office. Until then, 1 and 2 seem fairly tractable, if somewhat boring, to churn through and fix.

A more complete review follows.

::: js/src/builtin/ReflectParse.cpp
@@ +1798,5 @@
>                     "defaults", defarray,
>                     "body", body,
>                     "rest", rest,
>                     "generator", isGeneratorVal,
> +                   "async", isAsyncVal,

this will do for now. We may well have to revisit this when esprima writes a spec for async/await support. It is likely to look a lot like this, though, so this should be good.

::: js/src/frontend/BytecodeCompiler.cpp
@@ +312,5 @@
>      RootedFunction fun(cx, evalCaller->functionOrCallerFunction());
>      MOZ_ASSERT_IF(fun->strict(), options.strictOption);
>      Directives directives(/* strict = */ options.strictOption);
>      ObjectBox* funbox = parser->newFunctionBox(/* fn = */ nullptr, fun, &parseContext,
> +                                              directives, fun->generatorKind(), fun->asyncKind());

pre-existing nit: directives should be aligned  with the /*, not the (

@@ +863,5 @@
>          return false;
>  
>      Rooted<JSFunction*> fun(cx, lazy->functionNonDelazifying());
>      MOZ_ASSERT(!lazy->isLegacyGenerator());
> +    ParseNode* pn = parser.standaloneLazyFunction(fun, lazy->strict(), lazy->generatorKind(), SyncFunction);

It's not at all clear to me that this is right. What keeps us from lazily compiling an async function?

::: js/src/frontend/BytecodeEmitter.cpp
@@ +8089,5 @@
>              return false;
>          break;
>  
> +      // PNK_AWAIT handling is not yet implemented,
> +      // so currently we just return "true" as a placeholder.

This is replaced later in this stack. OK.

::: js/src/frontend/FoldConstants.cpp
@@ +1738,5 @@
>          MOZ_ASSERT(!pn->pn_kid->maybeExpr());
>          return true;
>  
> +      case PNK_AWAIT:
> +        return true;

We can do better than this. At least we can Fold() the expression, right?

::: js/src/frontend/FullParseHandler.h
@@ +435,2 @@
>          TokenPos pos(begin, value ? value->pn_pos.end : begin + 1);
> +        return new_<BinaryNode>(isAwait ? PNK_AWAIT : PNK_YIELD, op, pos, value, gen);

Why not just add a new one, newAwaitExpression()? Maybe I will understand why below.

::: js/src/frontend/Parser.cpp
@@ +824,5 @@
>      if (!pc->sc->needStrictChecks())
>          return true;
>  
> +    if (name == context->names().eval || name == context->names().arguments ||
> +      (IsKeyword(name) && name != context->names().await)) {

nit: align (IsKeyword( with name above, and move opening brace to next line.

@@ +860,2 @@
>      ParseNode* pn = statements(YieldIsKeyword);
> +    tokenStream.setAwaitIsKeyword(awaitIsKeyword);

Man, this would have been ugly, otherwise, huh?

@@ +1451,5 @@
>          return nullptr;
>      if (options().selfHostingMode)
>          fun->setIsSelfHostedBuiltin();
> +
> +    fun->setAsyncKind(asyncKind);

If the function holds the asyncKind, then the FunctionBox needn't do so. We can just access it though funbox->function()->asyncKind().

@@ +2411,5 @@
>  {
>      MOZ_ASSERT_IF(kind == Statement, funName);
>  
> +    if (asyncKind == AsyncFunction)
> +        generatorKind = StarGenerator;

Shouldn't this be an assertion instead? functionDef has very specific callers, looks like.

@@ +2453,5 @@
>      tokenStream.tell(&start);
>  
>      while (true) {
> +        if (functionArgsAndBody(inHandling, pn, fun, kind, generatorKind, asyncKind,
> +           directives, &newDirectives))

nit: alignment

@@ +2698,5 @@
>  template <>
>  ParseNode*
>  Parser<FullParseHandler>::standaloneLazyFunction(HandleFunction fun, bool strict,
> +                                                 GeneratorKind generatorKind,
> +                                                 FunctionAsyncKind asyncKind)

but isn't this always SyncFunction, above? I don't get it.

@@ +2819,5 @@
>  #endif
>      }
>  
> +    bool beforeAwaitIsKeyword = tokenStream.getAwaitIsKeyword();
> +    tokenStream.setAwaitIsKeyword(fun->isAsync());

This approach is a little kludgey, but nowhere near as bad as what we do for yield. Nice design.

@@ +2869,5 @@
>  }
>  
>  template <typename ParseHandler>
>  typename ParseHandler::Node
> +Parser<ParseHandler>::functionStmt(YieldHandling yieldHandling, DefaultHandling defaultHandling, FunctionAsyncKind asyncKind)

nit: suspected line length. Ensure that it's <100 characters

@@ +2910,5 @@
>          !report(ParseStrictError, pc->sc->strict(), null(), JSMSG_STRICT_FUNCTION_STATEMENT))
>          return null();
>  
> +    if (asyncKind == AsyncFunction)
> +      generatorKind = StarGenerator;

Right, this supports the assertion claims above.

@@ +5806,5 @@
>      if (!noteNameUse(context->names().dotGenerator, generator))
>          return null();
>      if (isYieldStar)
>          return handler.newYieldStarExpression(begin, expr, generator);
> +    return handler.newYieldExpression(begin, expr, generator, JSOP_YIELD, isAwait);

yeah, this is a mistake. we should instead

if (isAwait)
    return handler.newAwaitExpression(...);
return handler.newYieldExpression(...);

@@ +6426,5 @@
>                  report(ParseError, false, propName, JSMSG_BAD_METHOD_DEF);
>                  return null();
>              }
> +            if (propType == PropertyType::AsyncMethod) {
> +                report(ParseError, false, null(), JSMSG_ASYNC_CONSTRUCTOR);

Uhm...Surely the error above will always be thrown if propType == PropertyType::AsyncMethod, so this code is dead.

@@ +6711,5 @@
>  typename ParseHandler::Node
>  Parser<ParseHandler>::expr(InHandling inHandling, YieldHandling yieldHandling,
>                             InvokedPrediction invoked)
>  {
> +    Node pn = assignExpr(inHandling, yieldHandling, true, invoked);

this |true| needs a /* foo = */ for readability.

@@ +6983,5 @@
>      if (tt == TOK_NAME) {
> +        TokenKind nextSameLine;
> +
> +        #ifdef NIGHTLY_BUILD
> +        if (tokenStream.currentName() == context->names().async &&

nit: alignment of if clauses.

@@ +6997,5 @@
>          if (endsExpr)
>              return identifierName(yieldHandling);
>      }
>  
> +    if (tt == TOK_AWAIT && !awaitAllowed) {

This feels out of place here. It should find a way to live in assignExprWithoutYieldOrAwait, or functionArguments().

@@ +7306,5 @@
>          return handler.newDelete(begin, expr);
>        }
>  
> +      case TOK_AWAIT:
> +          TokenKind nextSameLine;

Does this declaration not warn? Normally, you need a new block. See the default block below.

@@ +7315,5 @@
> +              if (nextSameLine != TOK_EOL) {
> +                 Node kid = unaryExpr(yieldHandling);
> +                  if (!kid)
> +                      return null();
> +                  return newYieldExpression(begin, kid, false, true);

These too need /* isYieldStar = */ and so on.

@@ +8241,3 @@
>      if (res && pc->lastYieldOffset != startYieldOffset) {
>          reportWithOffset(ParseError, false, pc->lastYieldOffset,
>                           msg, js_yield_str);

maybe change that to res->isKind(PNK_AWAIT) ? awaitMsg : yieldMsg, or something? Here is infinitely better than inside assignExpression for the await in default message

@@ +8844,5 @@
>          TokenKind tt;
>          if (!tokenStream.getToken(&tt, TokenStream::KeywordIsName))
>              return null();
> +
> +        if (isAsync && (tt == TOK_NAME || tt == TOK_STRING || tt == TOK_NUMBER || tt == TOK_LB)) {

We can do it cleaner than this. Not looking for accessor syntax on async is enough, right?

Then

async get name() { }

is still a syntax error? It looks like it should be.

@@ +8852,3 @@
>          if (tt == TOK_NAME) {
>              propAtom.set(tokenStream.currentName());
> +

nit: This new line is probably not needed.

@@ +8875,3 @@
>              return newNumber(tokenStream.currentToken());
>          }
> +        if (tt == TOK_LB) {

nit: Please remove the braces. Single line ifs do not need them. The blank line is also unnecessary.

@@ +8932,5 @@
>      }
>  
>      if (tt == TOK_LP) {
>          tokenStream.ungetToken();
> +        *propType = isGenerator ? PropertyType::GeneratorMethod : (

nit: indenting here, with ( on end of previous line is really ugly. Indent it one indenture past the start of isGenerator with the entire last ternary parenthesized.

@@ +9185,2 @@
>          return identifierName(yieldHandling);
> +      }

nit: no need for these braces.

::: js/src/frontend/Parser.h
@@ +127,5 @@
>      bool isGenerator() const { return generatorKind() != NotGenerator; }
>      bool isLegacyGenerator() const { return generatorKind() == LegacyGenerator; }
>      bool isStarGenerator() const { return generatorKind() == StarGenerator; }
>  
> +    bool isAsync() const { return sc->isFunctionBox() && sc->asFunctionBox()->function()->isAsync(); }

This confirms that we can not store the async kind in the function box, as we are already not consulting it.

@@ +519,5 @@
>      }
>  
>      // Use when the funbox should be linked to the outerpc's innermost scope.
>      FunctionBox* newFunctionBox(Node fn, HandleFunction fun, ParseContext<ParseHandler>* outerpc,
> +                                Directives directives, GeneratorKind generatorKind, FunctionAsyncKind asyncKind = SyncFunction)

The last parameter should flow to a third line.

@@ +650,5 @@
>       * Some parsers have two versions:  an always-inlined version (with an 'i'
>       * suffix) and a never-inlined version (with an 'n' suffix).
>       */
> +    Node functionStmt(YieldHandling yieldHandling,
> +      DefaultHandling defaultHandling, FunctionAsyncKind asyncKind);

Please align params after the (

::: js/src/frontend/SharedContext.h
@@ +298,5 @@
>      bool            usesApply:1;      /* contains an f.apply() call */
>      bool            usesThis:1;       /* contains 'this' */
>  
>      FunctionContextFlags funCxFlags;
> +    FunctionAsyncKind asyncKind_;

Unnecessary as discussed.

@@ +335,5 @@
> +    }
> +
> +    bool isAsync() {
> +      return asyncKind() == AsyncFunction;
> +    }

These seem dead. Are they? If so, please remove.

::: js/src/js.msg
@@ +513,5 @@
>  MSG_DEF(JSMSG_CANT_DELETE_SUPER, 0, JSEXN_REFERENCEERR, "invalid delete involving 'super'")
> +
> +// Promise
> +MSG_DEF(JSMSG_PROMISE_RESOLVED_WITH_ITSELF, 0, JSEXN_TYPEERR, "A promise cannot be resolved with itself")
> +MSG_DEF(JSMSG_SETTIMEOUT_INTERVAL_NONZERO, 0, JSEXN_TYPEERR, "Intervals other than 0 are not supported")

These don't belong in this patch, still, probably.

::: js/src/jsfun.h
@@ +139,5 @@
>          } i;
>          void*           nativeOrScript;
>      } u;
>      js::HeapPtrAtom  atom_;       /* name for diagnostics and decompiling */
> +    js::FunctionAsyncKind   asyncKind_;

OK, there's no way we can afford to do this. There are /way/ too many functions in the world to make them all a word larger. It's one bit. We need to find a way to fit it into the union above.

::: js/src/tests/ecma_7/AsyncFunctions/syntax.js
@@ +5,5 @@
> +/**
> + * Currently only a part of async/await grammar is supported:
> + * - Async function statements are supported.
> + * - Await expressions are supported (as regular unary expressions).
> + * All other parts of proposal are probably not supported.

This isn't even true! :)

@@ +53,5 @@
> +assertThrows(() => Reflect.parse("async function a() { return await; }"), SyntaxError);
> +
> +// Yield is not allowed in an async function / treated as identifier
> +assertThrows(() => Reflect.parse("async function a() { yield 3; }"), SyntaxError);
> +assertThrows(() => Reflect.parse("async function a() { return yield 3; }"), SyntaxError);

also throw in a test for var yield = 5, just for completeness

@@ +62,5 @@
> +
> +// Await is treated differently depending on context. Various cases.
> +Reflect.parse("var await = 3; async function a() { await 4; }");
> +Reflect.parse("async function a() { await 4; } var await = 5");
> +Reflect.parse("async function a() { function b() { return await; } }");

I can't believe this nonsense is specced to work. Blech. Nice test.

::: js/src/vm/Xdr.h
@@ +32,5 @@
>  static const uint32_t XDR_BYTECODE_VERSION_SUBTRAHEND = 306;
>  static const uint32_t XDR_BYTECODE_VERSION =
>      uint32_t(0xb973c0de - XDR_BYTECODE_VERSION_SUBTRAHEND);
>  
> +static_assert(JSErr_Limit == 416,

When this goes up, the subtrahend must also. Please increment the number above.

Comment 40

[Eric Faust [:efaust]]

Comment on attachment 8656865 [details] [diff] [review]
Part 5: Bytecode and self-hosted wrapper code

Review of attachment 8656865 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good! r=me with questions below answered, and tests guarded.

::: js/src/Makefile.in
@@ +285,5 @@
>  
>  selfhosting_srcs := \
>    $(srcdir)/builtin/Utilities.js \
>    $(srcdir)/builtin/Array.js \
> +	$(srcdir)/builtin/AsyncFunctions.js \

nit: alignment.

::: js/src/builtin/AsyncFunctions.js
@@ +3,5 @@
> + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
> +
> +function AsyncFunction_wrap(genFunction) {
> +  var fun = function() {
> +    try {

Please add a comment here that the try/catch is for function defaults throwing.

@@ +24,5 @@
> +function AsyncFunction_resume(gen, v, method) {
> +    let result;
> +    try {
> +      result = callFunction(method, gen, v);
> +      // get back into async function, run to next await point

nit: This should move to above the line of code it describes.

::: js/src/frontend/BytecodeEmitter.cpp
@@ +2002,5 @@
>          return true;
>  
>        case PNK_YIELD_STAR:
>        case PNK_YIELD:
> +      case PNK_AWAIT:

Oooops. Yes. Should this go in the previous patch?

@@ +5970,5 @@
> +BytecodeEmitter::emitAsyncWrapper(unsigned index, bool needsHomeObject) {
> +    JSAtom* atom = Atomize(cx, "AsyncFunction_wrap", 18);
> +    if (!atom)
> +        return false;
> +    if (needsHomeObject && !emitIndex32(JSOP_LAMBDA, index))

Please add a comment here describing the handshake with propertyList() and why we are doing it in that order.

@@ +7176,5 @@
>              propdef->pn_right->pn_funbox->needsHomeObject())
>          {
>              MOZ_ASSERT(propdef->pn_right->pn_funbox->function()->allowSuperProperty());
> +            bool isAsync = propdef->pn_right->pn_funbox->function()->isAsync();
> +            if (isAsync && !emit1(JSOP_SWAP))

Maybe the comment should actually live here, and above should be "see comment in propertyList()", or visa versa. Either way.

::: js/src/shell/js.cpp
@@ +1582,5 @@
> +        return false;
> +
> +    args.rval().set(rval);
> +    return true;
> +}

Are these impls not mandatory to getting the promise code working? Maybe just testing functions?

::: js/src/tests/ecma_7/AsyncFunctions/1.1.1.js
@@ +12,5 @@
> +assertThrowsSE("async function a() { super.prop(); }");
> +assertThrowsSE("async function a() { super(); }");
> +
> +// FIXME this should work
> +// assertThrowsSE("async function a(k = await 3) {}");

Why doesn't this work? Does it work now?

::: js/src/tests/ecma_7/AsyncFunctions/1.1.2.js
@@ +6,5 @@
> +
> +var anon = async function() { }
> +
> +assertEq(test.name, "test");
> +assertEq(anon.name, "");

Nice test.

::: js/src/tests/ecma_7/AsyncFunctions/methods.js
@@ +46,5 @@
> +  assertEventuallyEq(y.getBaseClassName(), 'X'),
> +]).then(() => {
> +  if (typeof reportCompare === "function")
> +      reportCompare(true, true);
> +});

All these tests need some kind of mechanism guarding them against failure when the code is no longer on nightly. When the nightly branch merges to Aurora (dev edition), these will start to fail, because neither classes nor async functions will parse.

::: js/src/tests/ecma_7/AsyncFunctions/semantics.js
@@ +91,5 @@
> +  return n !== 0 && await isEven(n - 1);
> +}
> +
> +// recursion, take three!
> +// FIXME this example doesn't work...

I thought you had showed me this working? It's certainly not commented out, so...

::: js/src/vm/GeneratorObject.cpp
@@ +61,5 @@
>  bool
>  GeneratorObject::suspend(JSContext* cx, HandleObject obj, AbstractFramePtr frame, jsbytecode* pc,
>                           Value* vp, unsigned nvalues)
>  {
> +    MOZ_ASSERT(*pc == JSOP_INITIALYIELD || *pc == JSOP_YIELD || *pc == JSOP_GENERATOR);

How can /that/ happen? Can it?

::: js/src/vm/ScopeObject.cpp
@@ +237,5 @@
>       */
>      if (callee->isNamedLambda()) {
> +        if (callee->isAsync()) {
> +            RootedFunction fun(cx, &callee->getExtendedSlot(1).toObject().as<JSFunction>());
> +            scopeChain = DeclEnvObject::create(cx, scopeChain, fun);

A beautiful hack :P

Comment 41

[Eric Faust [:efaust]]

Comment on attachment 8656866 [details] [diff] [review]
Part 6: Refactored JIT DEFFUN (by efaust)

This is really part of part 5, but I wrote it, so I can't review it.

The purpose is to make JSOP_DEFFUN take the function value from the stack, instead of from the bytecode stream.

Comment 42

[Jan de Mooij [:jandem]]

Comment on attachment 8656866 [details] [diff] [review]
Part 6: Refactored JIT DEFFUN (by efaust)

Review of attachment 8656866 [details] [diff] [review]:
-----------------------------------------------------------------

::: js/src/jit/BaselineCompiler.cpp
@@ +2473,5 @@
>  
>  bool
>  BaselineCompiler::emit_JSOP_DEFFUN()
>  {
> +

Uber nit: remove this empty line.

Comment 43

[mkierski]

Attachment: Part 2: (till) Promises boilerplate

Comment 44

[mkierski]

Attachment: Part 3: Promises implementation

Comment 45

[mkierski]

Attachment: Part 4: Parser grammar rules

Comment 46

[mkierski]

Attachment: Part 5: Bytecode and self-hosted wrapper code

Comment 47

[mkierski]

Attachment: Part 6: Refactored JIT DEFFUN (by efaust)

Comment 48

[Till Schneidereit [:till]]

Comment on attachment 8662651 [details] [diff] [review]
Part 3: Promises implementation

Review of attachment 8662651 [details] [diff] [review]:
-----------------------------------------------------------------

My deepest apologies for sitting on this for so long! Given that you're leaving, please do the barest minimum of what I'm asking below, and I'll pick it up from there.

The most important part is documentation of how things work. Mostly, that means explanations of the roles of the SLOTs on Promise instances: what do they do, how are they used? Also, much of the code isn't exactly self-explanatory, and it would be a great help to have some targeted comments in there.

Next is making this all tamper-proof. Right now, content code can change Promises' behavior by changing methods on Array instances. It's very important to ensure that whatever content code does, the behavior of builtins stays as it is intended to be.

I didn't look at the patches that actually introduce async/await, so ignore this if it doesn't apply, but: if the async/await implementation retrieves the Promise constructor from the current global and uses its methods, then it isn't tamper-proof in the same way. The best thing to do there would be to somehow stash the original value somewhere (for both the constructor and the set of methods used, which then should be invoked using callFunction), or use an internal version of promises that isn't affected by things being done to window.Promise.

Finally, I think it'd make sense to only expose this with an environment var set or something like that. Unconditionally landing it even in Nightly seems a bit risky, given that it's not tamper-proof right now. But again, I'm willing to pick it up and drive it in.

::: js/src/builtin/Promise.cpp
@@ +6,5 @@
>  
>  #include "builtin/Promise.h"
>  
>  #include "jscntxt.h"
> +#include "SelfHostingDefines.h"

This needs to be "builtin/SelfHostingDefines.h", and be moved up to below the Promise.h include.

::: js/src/builtin/Promise.js
@@ +20,5 @@
> + * this setTimeout implementation must be manually triggered by calling
> + * runEvents().
> + */
> +
> +var eventQueue = [];

Use `new List()` here. That way, the queue's behavior can't be changed by modifying Array.prototype. With the current implementation, content code could derail the Promise implementation by changing Array.prototype.push/pop.

@@ +24,5 @@
> +var eventQueue = [];
> +
> +function runEvents() {
> +  while (eventQueue.length > 0) {
> +    //var evt = eventQueue.pop();

Remove this line. (And the empty one below it.)

@@ +100,5 @@
> +}
> +
> +function Promise_all(promises) {
> +  var length = promises.length;
> +  var results = NewDenseArray(length);

This should just use `[]` instead of NewDenseArray. Below, the `results[index] = ..` should be replaced with `_DefineDataProperty(results, index, valueOrPromise)`. Then just remove the `NewDenseArray` function.

Right now, changing Array.prototype.push would again break Promise's behavior.

@@ +135,5 @@
> +    reject(value);
> +  });
> +}
> +
> +function Promise_resolve(value) {

What happens if you do this?

function Foo() {}
Foo.prototype.resolve = Promise.prototype.resolve;
var foo = new Foo();
foo.resolve();

I *think* it should assert in a debug build.

Comment 49

[mkierski]

> Foo.prototype.resolve = Promise.prototype.resolve;
It's a static function Promise.resolve - not Promise.prototype.resolve. You can change it but it's all right, isn't it?

I addressed all of your comments and you're right - it's possible to break the implementation of async functions by tampering with Promise prototype. Fortunately it looks like it's easy to fix.

Comment 50

[Eric Faust [:efaust]]

Comment on attachment 8662652 [details] [diff] [review]
Part 4: Parser grammar rules

Review of attachment 8662652 [details] [diff] [review]:
-----------------------------------------------------------------

Much better, just a few little nits.

::: js/src/builtin/ReflectParse.cpp
@@ +3113,5 @@
>        case PNK_INSTANCEOF:
>          return leftAssociate(pn, dst);
>  
>        case PNK_POW:
> +	      return rightAssociate(pn, dst);

nit: remove bogus indent

::: js/src/frontend/BytecodeCompiler.cpp
@@ +312,5 @@
>      RootedFunction fun(cx, evalCaller->functionOrCallerFunction());
>      MOZ_ASSERT_IF(fun->strict(), options.strictOption);
>      Directives directives(/* strict = */ options.strictOption);
>      ObjectBox* funbox = parser->newFunctionBox(/* fn = */ nullptr, fun, &parseContext,
> +                                              directives, fun->generatorKind(), fun->asyncKind());

nit: 'd' under '/', please

@@ +863,5 @@
>          return false;
>  
>      Rooted<JSFunction*> fun(cx, lazy->functionNonDelazifying());
>      MOZ_ASSERT(!lazy->isLegacyGenerator());
> +    ParseNode* pn = parser.standaloneLazyFunction(fun, lazy->strict(), lazy->generatorKind(), SyncFunction);

This is still wrong, right? this should be lazy->asyncKind()

::: js/src/frontend/Parser.cpp
@@ +1456,5 @@
>                   : JSFunction::INTERPRETED_GENERATOR);
>      }
>  
> +    if (asyncKind == AsyncFunction) {
> +        allocKind = gc::AllocKind::FUNCTION_EXTENDED;

A comment here being like

// We store the async wrapper in a slot for later access

would be nice

@@ -2654,5 @@
>          return false;
>  
>      if (!leaveFunction(pn, outerpc, kind))
>          return false;
> -

nit: please don't remove this line

@@ +2927,5 @@
>          !report(ParseStrictError, pc->sc->strict(), null(), JSMSG_STRICT_FUNCTION_STATEMENT))
>          return null();
>  
> +    if (asyncKind == AsyncFunction)
> +      generatorKind = StarGenerator;

nit: 4 space indent

@@ +7542,5 @@
> +      case TOK_AWAIT:
> +      {
> +          TokenKind nextSameLine;
> +
> +          #ifdef NIGHTLY_BUILD

nit: #ifdef always fully unindented to the far left

@@ +7550,5 @@
> +                 Node kid = unaryExpr(yieldHandling);
> +                  if (!kid)
> +                      return null();
> +                  return newYieldExpression(begin, kid, /* isYieldStar = */ false, /* isAwait = */ true);
> +            } else {

nit: indenture here makes little sense.

@@ +8475,5 @@
> +    if (!tokenStream.peekToken(&tt, TokenStream::Operand))
> +        return null();
> +
> +    if (tt == TOK_AWAIT) {
> +        report(ParseError, false, null(), JSMSG_AWAIT_IN_DEFAULT);

Yeah. This is much better. Thanks :)

::: js/src/frontend/Parser.h
@@ +657,5 @@
>       */
> +    Node functionStmt(YieldHandling yieldHandling,
> +      DefaultHandling defaultHandling, FunctionAsyncKind asyncKind);
> +    Node functionExpr(InvokedPrediction invoked = PredictUninvoked,
> +      FunctionAsyncKind asyncKind = SyncFunction);

Indenture here still wrong

::: js/src/frontend/SyntaxParseHandler.h
@@ +282,5 @@
>      bool addPropertyDefinition(Node literal, Node name, Node expr) { return true; }
>      bool addShorthand(Node literal, Node name, Node expr) { return true; }
>      bool addObjectMethodDefinition(Node literal, Node name, Node fn, JSOp op) { return true; }
>      bool addClassMethodDefinition(Node literal, Node name, Node fn, JSOp op, bool isStatic) { return true; }
> +    Node newYieldExpression(uint32_t begin, Node value, Node gen, JSOp op) { return NodeUnparenthesizedYieldExpr; }

why did this grow a JSOp?

::: js/src/jsfun.h
@@ +285,5 @@
>          flags_ |= RESOLVED_NAME;
>      }
>  
> +    void setAsyncKind(js::FunctionAsyncKind asyncKind) {
> +        if (isInterpretedLazy())

We talked about your fix here. That's fine.

Comment 51

[Eric Faust [:efaust]]

Comment on attachment 8662653 [details] [diff] [review]
Part 5: Bytecode and self-hosted wrapper code

Review of attachment 8662653 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good to me. Going to ask Till to look over AsyncFunctions.js, just to be sure his fears from his last review are assuaged.

::: js/src/frontend/BytecodeCompiler.cpp
@@ +863,5 @@
>          return false;
>  
>      Rooted<JSFunction*> fun(cx, lazy->functionNonDelazifying());
>      MOZ_ASSERT(!lazy->isLegacyGenerator());
> +    ParseNode* pn = parser.standaloneLazyFunction(fun, lazy->strict(), lazy->generatorKind(), lazy->asyncKind());

Ah, this fix is here! OK. That's fine, then.

::: js/src/frontend/Parser.cpp
@@ +2639,5 @@
>          return false;
>  
>      if (!functionArgsAndBodyGeneric(inHandling, yieldHandling, pn, fun, kind))
>          return false;
> +    

nit: whitespace on empty line

::: js/src/shell/js.cpp
@@ +998,5 @@
>  static bool
>  CacheEntry_setBytecode(JSContext* cx, HandleObject cache, uint8_t* buffer, uint32_t length)
>  {
>      MOZ_ASSERT(CacheEntry_isCacheEntry(cache));
> +  

nit: whitespace on empty line

Comment 52

[Eric Faust [:efaust]]

Comment on attachment 8662654 [details] [diff] [review]
Part 6: Refactored JIT DEFFUN (by efaust)

Review of attachment 8662654 [details] [diff] [review]:
-----------------------------------------------------------------

Carrying Jandem's review. I cannot review this, as I wrote it.

Comment 53

[Till Schneidereit [:till]]

(In reply to mkierski from comment #49)
> > Foo.prototype.resolve = Promise.prototype.resolve;
> It's a static function Promise.resolve - not Promise.prototype.resolve. You
> can change it but it's all right, isn't it?

Oh, right. In that case: what happens if you pass in an arbitrary object, like {}? Normal objects shouldn't have 4 reserved slots, so this should assert.

> I addressed all of your comments and you're right - it's possible to break
> the implementation of async functions by tampering with Promise prototype.
> Fortunately it looks like it's easy to fix.

Excellent. Self-hosted JS is simple on the surface, and quite finicky in the details, unfortunately.

Comment 54

[Till Schneidereit [:till]]

Comment on attachment 8662653 [details] [diff] [review]
Part 5: Bytecode and self-hosted wrapper code

Review of attachment 8662653 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good except for the comments below. Plus: this now always uses the self-hosted implementation of Promises. That's fine, as long as that doesn't interfere with the browser's implementation. What's more problematic is that even the SpiderMonkey-implementation relies on a browser-provided builtin: setTimeout. That can again be changed by content, wreaking havoc on async/await, IINM. To verify: what happens if you do `window.setTimeout = function() {throw "bazinga"}` and then use async/await in that window? If that works fine, great. If not, we need to somehow store the original value of `setTimeout` - or use the same fake implementation as in the shell. In the latter case, move it from a shell builtin to a self-hosting intrinsic and just use it like Promise_foo. That means async/await don't work any better in the browser than in the shell, but at least it should work exactly the same.

f+ with this feedback addressed.

::: js/src/builtin/AsyncFunctions.js
@@ +36,5 @@
> +      return Promise_resolve(result.value);
> +    }
> +
> +    // If we get here, `await` occurred. `gen` is paused at a yield point.
> +    return Promise_resolve(result.value).then(

This still isn't right: Promise.prototype.then can be change by content, changing the behavior here. `return callFunction(Promise_then, Promise_resolve(result.value), ...)` should fix that.

Note that I'm not well-versed enough in the details of promises to know whether the `result.value` and similar things are ok. It seems so, but I'm not sure.

::: js/src/vm/SelfHosting.cpp
@@ +1239,5 @@
> +intrinsic_SetFunctionExtendedSlot(JSContext* cx, unsigned argc, Value* vp)
> +{
> +    CallArgs args = CallArgsFromVp(argc, vp);
> +    MOZ_ASSERT(args.length() == 3);
> +    MOZ_ASSERT(args[0].isObject());

Uber-nit: this assert isn't required, because it's contained in `toObject` in the line below. Feel free to ignore.

Comment 55

[mkierski]

Attachment: Rolled up testing patch

STR

open dev console

async function() a { }
run |a().then(console.log)| until assertion failure

Comment 56

[Eric Faust [:efaust]]

(In reply to mkierski from comment #55)
> Created attachment 8665172 [details] [diff] [review]
> Rolled up testing patch
> 
> STR
> 
> open dev console
> 
> async function() a { }
> run |a().then(console.log)| until assertion failure

I pared this down to |a()| until failure, just to simplify.

The problem is that the clone of AsyncFunction_Wrap in the content compartment is relazified, despite having an internal function, which escapes.

I have the following evidence for this:

Program received signal SIGSEGV, Segmentation fault.
0x00007fffe7d7ef1e in JSFunction::needsCallObject (this=0x7fffb130ebf0) at /home/efaust/m-classes/js/src/jsfun.h:148
148	        MOZ_ASSERT(!isInterpretedLazy());
(gdb) where
#0  0x00007fffe7d7ef1e in JSFunction::needsCallObject() const (this=0x7fffb130ebf0) at /home/efaust/m-classes/js/src/jsfun.h:148
#1  0x00007fffe7d91c01 in js::StaticScopeIter<(js::AllowGC)0>::hasSyntacticDynamicScopeObject() const (this=0x7fffffff18b0) at /home/efaust/m-classes/js/src/vm/ScopeObject-inl.h:107
#2  0x00007fffe7f052a5 in AssertDynamicScopeMatchesStaticScope(JSContext*, JSScript*, JSObject*) (cx=0x7fffd7e44500, script=0x7fffc16af8b0, scope=0x7fffb1306600) at /home/efaust/m-classes/js/src/vm/Stack.cpp:150
#3  0x00007fffe7f058bb in js::InterpreterFrame::prologue(JSContext*) (this=0x7fffd486e4e8, cx=0x7fffd7e44500) at /home/efaust/m-classes/js/src/vm/Stack.cpp:218
#4  0x00007fffe7e5352b in Interpret(JSContext*, js::RunState&) (cx=0x7fffd7e44500, state=...) at /home/efaust/m-classes/js/src/vm/Interpreter.cpp:1952
#5  0x00007fffe7e4f282 in js::RunScript(JSContext*, js::RunState&) (cx=0x7fffd7e44500, state=...) at /home/efaust/m-classes/js/src/vm/Interpreter.cpp:709
#6  0x00007fffe7e4f733 in js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) (cx=0x7fffd7e44500, args=..., construct=js::NO_CONSTRUCT) at /home/efaust/m-classes/js/src/vm/Interpreter.cpp:786
BACKTRACE CONTINUES

(gdb) up
#2  0x00007fffe7f052a5 in AssertDynamicScopeMatchesStaticScope (cx=0x7fffd7e44500, script=0x7fffc16af8b0, scope=0x7fffb1306600) at /home/efaust/m-classes/js/src/vm/Stack.cpp:150
150	        } else if (i.hasSyntacticDynamicScopeObject()) {
(gdb) p enclosingScope.get()->as<JSFunction>().isInterpretedLazy()
$13 = true
(gdb) p enclosingScope.get()
$14 = (JSObject * const&) @0x7fffffff18d0: 0x7fffb130ebf0
(gdb) p i.obj()
Invalid data type for function to be called.
(gdb) p i.obj
$15 = {<js::RootedBase<JSObject*>> = {<No data fields>}, ptr = 0x7fffb130ebf0}
(gdb) p i.obj.get()
$16 = (JSObject * const&) @0x7fffffff18b0: 0x7fffb130ebf0

|enclosingScope| here is args.callee()->nonLazyScript()->enclosingStaticScope(), in terms of the Invoke call.

So it's the immediately enclosing function to the one being called that got delazified, which is AsyncFunction_Wrap in this case.

One fix, which Mariusz found, was to make a function CreateAsyncWrapper() that's called by AsyncFunction_Wrap(). This will ensure that the version in the selfhosted compartment is always delazified and avoid this bug, but it seems bad to evade it.

All told, it seems way more likely that we want to do some checks and respect the doNotRelazify_ flag here in some way, but it's late and I'm not going to find the exact right spot.

ni? till as he said he would take up the torch tomorrow to fix the rest of this. It should be fairly easy for someone familiar with the code, I would think.

Comment 57

[Till Schneidereit [:till]]

Shell test case:

var g = newGlobal();
g.eval("async function a(){ }");
gc();
g.a();

Comment 58

[mkierski]

Tested, this one works.

Comment 59

[mkierski]

Attachment: Part 1: (terrence) fixes for the StoreBuffer

Comment 60

[mkierski]

Attachment: Part 1: (till) Promises boilerplate

Comment 61

[mkierski]

Attachment: Part 2: Promises implementation

Comment 62

[mkierski]

Attachment: Part 3: Parser grammar rules

Comment 63

[mkierski]

Attachment: Part 4: Bytecode and self-hosted wrapper code

Comment 64

[mkierski]

Attachment: Part 5: Refactored JIT DEFFUN (by efaust)

Comment 65

[Till Schneidereit [:till]]

Comment on attachment 8665561 [details] [diff] [review]
Part 1: (till) Promises boilerplate

Review of attachment 8665561 [details] [diff] [review]:
-----------------------------------------------------------------

I think we can just carry efaust's r+ here, as it's just a straight rename. Please fix the author information and append r=efaust to the description before landing.

Comment 66

[Till Schneidereit [:till]]

Comment on attachment 8665562 [details] [diff] [review]
Part 2: Promises implementation

Review of attachment 8665562 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good by and large. r=me with the feedback addressed.

The {Set,Get}FunName intrinsics introduced here (and presumably used in the next patches) shouldn't be required: _DefineDataProperty(myFunction, "name", "myName") *should* work. And just using `var name = myFunction.name` for getting the name should, too.

::: js/src/builtin/Promise.cpp
@@ +33,5 @@
>      CallArgs args = CallArgsFromVp(argc, vp);
>  
>      JSObject* obj = NewBuiltinClassInstance(cx, &ShellPromiseObject::class_);
>      if (!obj)
>          return false;

These three lines snuck back in: the last version of the patch correctly removed them.

@@ +55,5 @@
> +        return false;
> +
> +    JSObject* deferredsArray = JS_NewArrayObject(cx, 0);
> +    if (!deferredsArray)
> +        return false;

Oh, I didn't see that this was wrong in the last version, good catch.

@@ +57,5 @@
> +    JSObject* deferredsArray = JS_NewArrayObject(cx, 0);
> +    if (!deferredsArray)
> +        return false;
> +
> +    JS_SetReservedSlot(promise, PROMISE_STATE_SLOT, NumberValue(PROMISE_STATE_PENDING)); // state

The comments for these three lines should be dropped: they just repeat what the code says.

@@ +59,5 @@
> +        return false;
> +
> +    JS_SetReservedSlot(promise, PROMISE_STATE_SLOT, NumberValue(PROMISE_STATE_PENDING)); // state
> +    JS_SetReservedSlot(promise, PROMISE_VALUE_SLOT, NullValue()); // value
> +    JS_SetReservedSlot(promise, PROMISE_DEFERREDS_SLOT, ObjectValue(*deferredsArray)); // deferreds

I just realize that this slot is set in the self-hosted Promise_init, too. Given that, just replace this with `NullValue()` (and the creation of `deferredsArray` above). We could do it in either place, except the self-hosted version can use the internal `List` class which content can't modify.

@@ +94,5 @@
>      return cx->global()->createBlankPrototype(cx, &ShellPromiseObject::protoClass_);
>  }
>  
>  const Class ShellPromiseObject::class_ = {
> +   "ShellPromise",

Nit: revert this change.

@@ +108,5 @@
>      nullptr, /* finalize */
>      nullptr, /* call */
>      nullptr, /* hasInstance */
>      nullptr, /* construct */
> +    nullptr, /* trace  */

Nit: revert this change.

::: js/src/builtin/Promise.h
@@ +17,5 @@
>  {
>    public:
> +     static const unsigned RESERVED_SLOTS = 3;
> +     static const Class class_;
> +     static const Class protoClass_;

These three lines were somehow indented one space further. Please revert.

::: js/src/builtin/Promise.js
@@ +24,5 @@
> +var eventQueue = new List();
> +
> +function runEvents() {
> +  while (eventQueue.length > 0) {
> +    var evt = callFunction(std_Array_pop, eventQueue);

With `eventQueue` being a list now, .pop can just be called normally. However, it's probably clearer to still use callFunction, so leave this as-is.

@@ +71,5 @@
> +  return UnsafeGetReservedSlot(promise, PROMISE_VALUE_SLOT);
> +}
> +
> +function Promise_init(promise, fn) {
> +  Promise_setDeferreds(promise, []);

Use `new List()` here instead of [].

@@ +125,5 @@
> +}
> +
> +function Promise_resolve(value) {
> +  if (value && typeof value === 'object' &&
> +    IsPromise(value)) {

This should all fit into one line. Then, the braces can be left off.

::: js/src/vm/SelfHosting.cpp
@@ +1241,5 @@
>      return true;
>  }
>  
> +bool
> +intrinsic_SetFunctionExtendedSlot(JSContext* cx, unsigned argc, Value* vp)

This intrinsic isn't used in the patch. Is it used in a consecutive patch? If so, it's ok to keep, but if not, please remove.

Comment 67

[Till Schneidereit [:till]]

Oh, one thing I forgot: I still would really like to see documentation of how all this works. Either in one big block comment in Promise.js, or spread out over various comments in the complicated places.

Comment 68

[mkierski]

Attachment: Part 2: Promises implementation

Comment 69

[Eric Faust [:efaust]]

Comment on attachment 8665575 [details] [diff] [review]
Part 5: Refactored JIT DEFFUN (by efaust)

Carrying jandem's r+

Comment 70

[Eric Faust [:efaust]]

Comment on attachment 8665678 [details] [diff] [review]
Part 2: Promises implementation

Carrying till's r+

Comment 71

[Eric Faust [:efaust]]

Comment on attachment 8665572 [details] [diff] [review]
Part 3: Parser grammar rules

Review of attachment 8665572 [details] [diff] [review]:
-----------------------------------------------------------------

Carrying. This did not require additional review.

Comment 72

[Eric Faust [:efaust]]

Comment on attachment 8665574 [details] [diff] [review]
Part 4: Bytecode and self-hosted wrapper code

Review of attachment 8665574 [details] [diff] [review]:
-----------------------------------------------------------------

Carrying. This did not require additional review.

Comment 73

[Eric Faust [:efaust]]

Attachment: Part 6: Clean up the NIGHTLY_ONLY guards for the feature, and allow tests to uplift cleanly

Lots of context, by request

Comment 74

[Jeff Walden [:Waldo]]

Comment on attachment 8668698 [details] [diff] [review]
Part 6: Clean up the NIGHTLY_ONLY guards for the feature, and allow tests to uplift cleanly

Review of attachment 8668698 [details] [diff] [review]:
-----------------------------------------------------------------

I'd prefer putting JS_HAS_ASYNC_FUNCS directly into code as a condition, so that async-handling code always builds and can't be accidentally broken for an uplift where it gets turned off again, but this is somewhat of a nit.

::: js/src/tests/ecma_7/AsyncFunctions/methods.js
@@ +59,5 @@
> +
> +`;
> +
> +if (classesEnabled() && asyncFunctionsEnabled())
> +    eval(test);

inb4philor

::: js/src/tests/ecma_7/AsyncFunctions/shell.js
@@ +30,5 @@
> +    try {
> +        eval("async function f()  { }");
> +        return true;
> +    } catch (e if e instanceof SyntaxError) {
> +        return false;

Use

  if (e instanceof SyntaxError)
    return false;
  throw e;

to avoid non-standard syntax

::: js/src/tests/shell.js
@@ +871,5 @@
>      fullmsg += " - " + msg;
>    throw new Error(fullmsg);
>  };
>  
> +function classesEnabled(testCode = "class Foo { constructor() {} }") {

Hmm, doesn't this want to be removed from ecma_6/shell.js or wherever it is?

Comment 75

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/c3d780ac9879
https://hg.mozilla.org/integration/mozilla-inbound/rev/9c74d17cf769
https://hg.mozilla.org/integration/mozilla-inbound/rev/102aa11bc1aa
https://hg.mozilla.org/integration/mozilla-inbound/rev/abf4f0c6f42e
https://hg.mozilla.org/integration/mozilla-inbound/rev/ede6564c4d13
https://hg.mozilla.org/integration/mozilla-inbound/rev/6bd1aab9f146

Comment 76

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

All backed out in https://hg.mozilla.org/integration/mozilla-inbound/rev/77416253dff5 for build bustage: https://treeherder.mozilla.org/logviewer.html#?job_id=15202761&repo=mozilla-inbound

Comment 77

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/3afd56565e23
https://hg.mozilla.org/integration/mozilla-inbound/rev/8073e7c4cf94
https://hg.mozilla.org/integration/mozilla-inbound/rev/5174ef291f0c
https://hg.mozilla.org/integration/mozilla-inbound/rev/a3c1f65f0dff
https://hg.mozilla.org/integration/mozilla-inbound/rev/446ea07800c9
https://hg.mozilla.org/integration/mozilla-inbound/rev/8453ae71b30d

Comment 78

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Back out again for SM bustage: https://treeherder.mozilla.org/logviewer.html#?job_id=15241228&repo=mozilla-inbound

https://hg.mozilla.org/integration/mozilla-inbound/rev/54129d68a053

Comment 79

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

You're also failing this mulet mochitest-4 test: https://treeherder.mozilla.org/logviewer.html#?job_id=15244267&repo=mozilla-inbound

Comment 80

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

There's actually several things failing: 
https://treeherder.mozilla.org/logviewer.html#?job_id=15247970&repo=mozilla-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=15248057&repo=mozilla-inbound


I'd suggest a try run that runs pretty much everything at this point before relanding.

Comment 81

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/deb987f79ee8
https://hg.mozilla.org/integration/mozilla-inbound/rev/d26bbcce9e10
https://hg.mozilla.org/integration/mozilla-inbound/rev/50d74e6e8937
https://hg.mozilla.org/integration/mozilla-inbound/rev/e0552bf93a5a
https://hg.mozilla.org/integration/mozilla-inbound/rev/2829205d2c81
https://hg.mozilla.org/integration/mozilla-inbound/rev/abdb25290c42

Comment 82

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

All backed out in https://hg.mozilla.org/integration/mozilla-inbound/rev/2384726c541f for the mulet m(4) failures: https://treeherder.mozilla.org/logviewer.html#?job_id=15335340&repo=mozilla-inbound

Comment 83

[Eric Faust [:efaust]]

Attachment: Fix busted serviceworkers interfaces test for adding ShellPromise

Comment 84

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

JSReftests, too: https://treeherder.mozilla.org/logviewer.html#?job_id=15339033&repo=mozilla-inbound

Comment 85

[Pulsebot]

https://hg.mozilla.org/mozilla-central/rev/0ddd61fbe36f

Comment 86

[Xidorn Quan [:xidorn] UTC+10]

What's the status of this project now? It seems Blink started working on this now: https://groups.google.com/a/chromium.org/d/msg/blink-dev/jlSoAmWa1i4/7xowBIOSBAAJ

Comment 87

[Eric Faust [:efaust]]

These patches are waiting on the spidermonkey promises work that till is doing. There is a lot of interest in getting it finished an in, and that will happen as soon as we can.

Comment 88

[Tooru Fujisawa [:arai]]

will take this over :)
now rebasing and testing

Comment 89

[Tooru Fujisawa [:arai]]

looks like some parts of the spec are not implemented or doesn't match.
  * async arrow functions
  * export default async function

I guess the spec had been updated since then.
now fixing those parts

Comment 90

[Tooru Fujisawa [:arai]]

efaust, do you know the purpose of the following Early Error rule?

https://tc39.github.io/ecmascript-asyncawait/#async-decls-exprs-static-semantics-early-errors
> * It is a Syntax Error if ContainsUseStrict of AsyncFunctionBody is true and IsSimpleParameterList of FormalParameters is false.

IIUC, it means the following code throws SyntaxError, but I'm not sure why it should.

  async function f(a = 1) { `use strict`; }

as following doesn't throw SyntaxError:

  `use strict`; async function f(a = 1) {}

Comment 91

[Tooru Fujisawa [:arai]]

(In reply to Tooru Fujisawa [:arai] from comment #90)
> efaust, do you know the purpose of the following Early Error rule?
> 
> https://tc39.github.io/ecmascript-asyncawait/#async-decls-exprs-static-
> semantics-early-errors
> > * It is a Syntax Error if ContainsUseStrict of AsyncFunctionBody is true and IsSimpleParameterList of FormalParameters is false.
> 
> IIUC, it means the following code throws SyntaxError, but I'm not sure why
> it should.
> 
>   async function f(a = 1) { `use strict`; }
> 
> as following doesn't throw SyntaxError:
> 
>   `use strict`; async function f(a = 1) {}

Sorry, just noticed that it's just same as sync function's case, and SpiderMonkey haven't yet implemented it.
https://tc39.github.io/ecma262/#sec-function-definitions-static-semantics-early-errors

Comment 92

[Tooru Fujisawa [:arai]]

Attachment: Part 1: Refactor JSOP_DEFFUN.

Separated JSOP_DEFFUN refactoring from previous Part 3, and merged with Part 5.

Comment 93

[Tooru Fujisawa [:arai]]

Attachment: Part 2: Add parser support for Async functions. r=efaust

Rebased previous Part 3, and merged parser part of previous Part 6.
JS_HAS_ASYNC_FUNCS is defined only if SPIDERMONKEY_PROMISE is defined.

> +#ifdef SPIDERMONKEY_PROMISE
> +/* Support for ES7 Async Functions. */
> +#define JS_HAS_ASYNC_FUNCS 1
> +#endif // SPIDERMONKEY_PROMISE

Comment 94

[Tooru Fujisawa [:arai]]

Attachment: Part 4: Fix function name for export default async function.

Rebased previous Part 4, and merged remaining part of previous Part 6, and also merged sel-hosting intrinsic from previous Part 2 (SetFunctionExtendedSlot, SetFunName, GetFunName).

Comment 95

[Tooru Fujisawa [:arai]]

Comment on attachment 8753181 [details] [diff] [review]
Part 4: Fix function name for export default async function.

oops, this was wrong one.

Comment 96

[Tooru Fujisawa [:arai]]

Attachment: Part 3: Implement async functions. r=efaust,jandem

Rebased previous Part 4, and merged remaining part of previous Part 6, and also merged sel-hosting intrinsic from previous Part 2 (SetFunctionExtendedSlot, SetFunName, GetFunName).

Comment 97

[Tooru Fujisawa [:arai]]

Attachment: Part 4: Fix function name for export default async function.

`export default async function() {}` should be named '*default*', and it needs separated handling than other expression after `export default`.

https://tc39.github.io/ecmascript-asyncawait/#async-decls-exprs-static-semantics-BoundNames

Comment 98

[Tooru Fujisawa [:arai]]

Attachment: Part 5: Treat yield as an identifier in async function parameters and body.

`yield` is always an identifier in async function parameters and body. (but not function name).

https://tc39.github.io/ecmascript-asyncawait/#async-function-definitions

Removed JSMSG_YIELD_IN_ASYNC, as it's not valid error.

Comment 99

[Tooru Fujisawa [:arai]]

Attachment: Part 6: Support async arrow function.

Added async arrow support.

Async arrow function with expression body |async () => 1| is converted into STATEMENTLIST body, to insert initial yield, and converted back to expression body in Reflect.parse.

then, in ecma_7/AsyncFunctions/arrow.js, there are some disabled tests for `yield` and `await` binding in strict mode.  is there any bug already filed for the `yield` binding issue?  (if so, I'll add the bug number there.  if not, I'll file it)


Also, added comment about the stack convention in emitAsyncWrapper.

Comment 100

[Tooru Fujisawa [:arai]]

Attachment: Part 7: Add parser test for async/await.

Added parser test for modifier and error handling.

Comment 101

[Tooru Fujisawa [:arai]]

Attachment: Part 8: Add Reflect.parse test for async/await.

Added Reflect.parse test.

failure cases are already tested in Part 2, so this tests whole structure for successful cases.

Comment 102

[Eric Faust [:efaust]]

Comment on attachment 8753185 [details] [diff] [review]
Part 5: Treat yield as an identifier in async function parameters and body.

Review of attachment 8753185 [details] [diff] [review]:
-----------------------------------------------------------------

Nice. A few nits below.

::: js/src/frontend/Parser.cpp
@@ +2827,5 @@
>  }
>  
>  template <typename ParseHandler>
>  typename ParseHandler::Node
> +Parser<ParseHandler>::functionDef(InHandling inHandling,

oh, oops. Nice catch that this was dead.

@@ +3162,5 @@
>      else if (fun->isGetter())
>          syntaxKind = Getter;
>      else if (fun->isSetter())
>          syntaxKind = Setter;
> +    YieldHandling yieldHandling = GetYieldHandling(generatorKind, asyncKind);

nit: please insert blank line above this.

@@ +3290,5 @@
>  template <typename ParseHandler>
>  bool
>  Parser<ParseHandler>::checkYieldNameValidity()
>  {
> +    if (pc->isAsync())

Add comment being like "async functions are implemented as star generators, but not syntactically. Allow yield" or something.

@@ +3296,1 @@
>      // In star generators and in JS >= 1.7, yield is a keyword.  Otherwise in

nit: blank line before comment

@@ +8037,5 @@
>          if (endsExpr)
>              return stringLiteral();
>      }
>  
>      if (tt == TOK_YIELD && yieldExpressionsSupported()) {

nit: can remove braces from single line if.

Comment 103

[Eric Faust [:efaust]]

Comment on attachment 8753190 [details] [diff] [review]
Part 6: Support async arrow function.

Review of attachment 8753190 [details] [diff] [review]:
-----------------------------------------------------------------

This is good, but we also need tests for async() being a function call. As I understand things, this should be the case with no backtracking. Is that true?

::: js/src/builtin/ReflectParse.cpp
@@ +3497,1 @@
>          builder.function(type, &pn->pn_pos, id, args, defaults, body,

can we align the builder. with the functionArgs above?

::: js/src/frontend/BytecodeEmitter.cpp
@@ +6478,5 @@
>      /* Non-hoisted functions simply emit their respective op. */
>      if (!pn->functionIsHoisted()) {
>          /* JSOP_LAMBDA_ARROW is always preceded by a new.target */
>          MOZ_ASSERT(fun->isArrow() == (pn->getOp() == JSOP_LAMBDA_ARROW));
> +        if (needsProto) {

Does this case really need to move? It's only for class constructors, right?

@@ +6564,5 @@
>                     bi->kind() == Binding::ARGUMENT);
>          MOZ_ASSERT(bi.argOrLocalIndex() < JS_BIT(20));
>  #endif
>          if (funbox->isAsync()) {
> +            if (!emitAsyncWrapper(index, false, false))

please comment these bools with argument names, for future readers

::: js/src/frontend/Parser.cpp
@@ +1389,5 @@
>              return null();
>      } else {
>          MOZ_ASSERT(type == ExpressionBody);
>  
> +        Node stmtList = null();

A comment here about star generators being assumed to be statement lists would be nice.

Comment 104

[Eric Faust [:efaust]]

Comment on attachment 8753193 [details] [diff] [review]
Part 8: Add Reflect.parse test for async/await.

Review of attachment 8753193 [details] [diff] [review]:
-----------------------------------------------------------------

Good.

::: js/src/builtin/ReflectParse.cpp
@@ +3529,5 @@
>          if (pnstart && pnstart->isKind(PNK_YIELD)) {
>              MOZ_ASSERT(pnstart->getOp() == JSOP_INITIALYIELD);
>              pnstart = pnstart->pn_next;
>          }
> +        // Async arrow with expression body is converted into STATEMENTLIST

nit: blank line before comment

Comment 105

[Tooru Fujisawa [:arai]]

Thank you for reviewing :)

I found that the yield binding issue in strict mode is just a bug in my patch, will fix it.

Comment 106

[Tooru Fujisawa [:arai]]

Attachment: Part 4: Fix function name for export default async function. r=efaust

Comment 107

[Tooru Fujisawa [:arai]]

Attachment: Part 5: Treat yield as an identifier in async function parameters and body. r=efaust

Comment 108

[Tooru Fujisawa [:arai]]

Attachment: Part 6: Support async arrow function. r=efaust

Comment 109

[Tooru Fujisawa [:arai]]

Attachment: Part 6.1: Fix yield handling in async function.

yield handling in async arrow is inherited from enclosing script, so, `GetYieldHandling` needs some more information:
  * YieldHandling of encloding script
  * FunctionSyntaxKind for the function (whether it's arrow or not)
  * YieldHandldingContext (whether current context is arguments or body)

Also, added YieldIsNotInherited to YieldHandling, to clarify that the parameter has no meaning, in the case yield handling is not inherited.

also, yieldHandling differs for arguments and body, so moved GetYieldHandling call into functionArgsAndBodyGeneric, and call it for functionArguments and functionBody.

Comment 110

[Tooru Fujisawa [:arai]]

Attachment: Part 6.2: Fix await handling in async function.

await should be set to keyword for whole function parameter, and also name of function expression.

destructuring parameter's default value can also contain await, and default parameter can be complex expression that contains await.
So, we should check the existence of await expression just like yield, in destructuring parameter and default parameter, by lastAwaitOffset.


Also added AutoAwaitIsKeyword, to automatically reset TokenStream's awaitIsKeyword to previous value, when leaving the scope.

Comment 111

[Tooru Fujisawa [:arai]]

Attachment: Part 7: Add parser test for async/await. r=efaust

Comment 112

[Tooru Fujisawa [:arai]]

Attachment: Part 8: Add Reflect.parse test for async/await. r=efaust

Comment 113

[Tooru Fujisawa [:arai]]

so far, here are things to fix:
  * AsyncFunction constructor and prototype
  * length of the async function
  * toString
  * syntax priority for async function expression

may be some more.
some are already fixed locally.

Comment 114

[Mark Hammond [:markh] [:mhammond]]

(In reply to Tooru Fujisawa [:arai] from comment #113)
> so far, here are things to fix:

I'm not following this closely enough to know if this is sane, but is there any possibility of hiding the feature behind a nightly-only pref and having followup bugs that block it riding the trains?

Comment 115

[Tooru Fujisawa [:arai]]

(In reply to Mark Hammond [:markh] from comment #114)
> (In reply to Tooru Fujisawa [:arai] from comment #113)
> > so far, here are things to fix:
> 
> I'm not following this closely enough to know if this is sane, but is there
> any possibility of hiding the feature behind a nightly-only pref and having
> followup bugs that block it riding the trains?

don't worry :)
No patches have been landed, and I won't land these patches until those issues are fixed.
also, async/await is behind two switches, NIGHTLY_BUILD and SPIDERMONKEY_PROMISE, that means even if I land them, it won't be available in Nightly.

Comment 116

[Mark Hammond [:markh] [:mhammond]]

(In reply to Tooru Fujisawa [:arai] from comment #115)
> don't worry :)
> No patches have been landed, and I won't land these patches until those
> issues are fixed.

I worry they *will not* land ASAP - I want to play with them!

> also, async/await is behind two switches, NIGHTLY_BUILD and
> SPIDERMONKEY_PROMISE

Right - that sounds great! I'm wondering if we can land that way before those other problems are fixed, and only remove those switches once those other issues are fixed?

> that means even if I land them, it won't be available
> in Nightly.

Oh - it looks like SPIDERMONKEY_PROMISE isn't enabled anywhere by default? I must have mis-guessed what that means...

Comment 117

[Tooru Fujisawa [:arai]]

ah, I see.
Will try fixing remaining things shortly, now toString is the only remaining not-yet-fixed-locally issue :)

Comment 118

[Till Schneidereit [:till]]

(In reply to Mark Hammond [:markh] from comment #116)
> Oh - it looks like SPIDERMONKEY_PROMISE isn't enabled anywhere by default? I
> must have mis-guessed what that means...

It means I have to get this green: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ce5f80201fe1

Comment 119

[Tooru Fujisawa [:arai]]

Attachment: Part 9: Fix syntax priority of async function expression.

async function expression was parsed as assignment expression, but should be primary expression

https://tc39.github.io/ecmascript-asyncawait/#PrimaryExpression

Comment 120

[Tooru Fujisawa [:arai]]

Attachment: Part 10: Add AsyncFunction constructor and prototype.

Added AsyncFunction constructor, and AsyncFunctionPrototype.

wrapper's prototype is dynamically changed to AsyncFunctionPrototype in AsyncFunction_wrap.

AsyncFunction constructor and prototype are created just like GeneratorFunction constructor and prototype.

AsyncFunctionConstructor is the implementation of AsyncFunction constructor.
there is one tricky thing.  function created in FunctionConstructor is unwrapped function, so it uses GeneratorFunction as its prototype.
then, it's wrapped in AsyncFunctionConstructor, just like we do in bytecode.

Also, modified intrinsic_GetBuiltinConstructor to return AsyncFunction constructor, that has no JSProtoKey.

Comment 121

[Tooru Fujisawa [:arai]]

Attachment: Part 11: Set length of async function.

Also, length of async function was not set to unwrapped function's one.
so, changed to define it inside AsyncFunction_wrap

Comment 122

[Tooru Fujisawa [:arai]]

Attachment: Part 12: Support async function in Function.prototype.toString.

to support toString, we need to do following steps:
  1. check if given function is wrapper
  2. extract unwrapped function from wrapper
  3. apply toString on unwrapped function


for 1, stored "AsyncWrapper" string to wrapper's first reserved slot.
as, self-hosted function's first reserved slot is mostly used to store original function name, but it's not used for inner function.
it would be better to store name there for this case too, as we can use IsSelfHostedFunctionWithName to check it.

for 2, stored unwrapped function into wrapper's second reserved slot.


Then, just as a cleanup, moved slot handling done in CallObject::createForFunction into GetAsyncFunctionWrapper,
and added GetUnwrappedAsyncFunction for other way.

Comment 123

[Till Schneidereit [:till]]

Comment on attachment 8755554 [details] [diff] [review]
Part 10: Add AsyncFunction constructor and prototype.

Review of attachment 8755554 [details] [diff] [review]:
-----------------------------------------------------------------

::: js/src/builtin/AsyncFunctions.js
@@ +11,5 @@
>              var promiseCtor = GetBuiltinConstructor('Promise');
>              return callFunction(Promise_static_reject, promiseCtor, e);
>          }
>      };
> +    std_Object_setPrototypeOf(wrapper, GetBuiltinPrototype("AsyncFunction"));

It'd be great if we could somehow avoid this, because it costs us in terms of optimization potential. Specifically, every typeset the async function becomes part of will be made fully generic.

For a first implementation, that's probably fine, but we should ideally fix this before shipping.

Comment 124

[Tooru Fujisawa [:arai]]

With all those patches applied, following tests still fails in async/await test262 ( https://github.com/tc39/test262/pull/479 ), but all of them are not async/await specific issue.


implement toStringTag (bug 1114580)
  * built-ins/AsyncFunction/AsyncFunctionPrototype-to-string.js


toString should reflect the comment inside arguments (bug 755821)
  * built-ins/Function/prototype/toString/AsyncFunction.js
  * built-ins/Function/prototype/toString/async-function-declaration.js
  * built-ins/Function/prototype/toString/async-function-expression.js
  * built-ins/Function/prototype/toString/async-method.js


"use strict" in body with default/destructuring should throw (bug 1272784)
  * language/expressions/async-arrow-function/early-errors-arrow-NSPL-with-USD.js
  * language/expressions/async-function/early-errors-expression-NSPL-with-USD.js
  * language/expressions/object/method-definition/early-errors-object-method-NSPL-with-USD.js
  * language/statements/async-function/early-errors-declaration-NSPL-with-USD.js
  * language/statements/class/definition/early-errors-class-method-NSPL-with-USD.js


redeclaration should be SyntaxError, but we're using TypeError (no bug)
  * language/expressions/async-arrow-function/early-errors-arrow-formals-body-duplicate.js
  * language/expressions/async-function/early-errors-expression-formals-body-duplicate.js
  * language/statements/async-function/early-errors-declaration-formals-body-duplicate

Comment 125

[Tooru Fujisawa [:arai]]

(In reply to Till Schneidereit [:till] from comment #123)
> Comment on attachment 8755554 [details] [diff] [review]
> Part 10: Add AsyncFunction constructor and prototype.
> 
> Review of attachment 8755554 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: js/src/builtin/AsyncFunctions.js
> @@ +11,5 @@
> >              var promiseCtor = GetBuiltinConstructor('Promise');
> >              return callFunction(Promise_static_reject, promiseCtor, e);
> >          }
> >      };
> > +    std_Object_setPrototypeOf(wrapper, GetBuiltinPrototype("AsyncFunction"));
> 
> It'd be great if we could somehow avoid this, because it costs us in terms
> of optimization potential. Specifically, every typeset the async function
> becomes part of will be made fully generic.
> 
> For a first implementation, that's probably fine, but we should ideally fix
> this before shipping.

Thank you for pointing that out :)
okay, will try finding a better solution.

Comment 126

[Tooru Fujisawa [:arai]]

Comment on attachment 8755554 [details] [diff] [review]
Part 10: Add AsyncFunction constructor and prototype.

Okay, I got another solution without setPrototypeOf.
will post it after some more test.

Comment 127

[Tooru Fujisawa [:arai]]

Comment on attachment 8755555 [details] [diff] [review]
Part 11: Set length of async function.

This should be fixed at once

Comment 128

[Tooru Fujisawa [:arai]]

Comment on attachment 8755557 [details] [diff] [review]
Part 12: Support async function in Function.prototype.toString.

This will also be affected by the change.

Comment 129

[Tooru Fujisawa [:arai]]

Attachment: Part 10: Add AsyncFunction constructor and prototype.

Moved everything into CreateAsyncFunction intrinsic.
There a new function is created with %AsyncFunctionPrototype%,
reusing the script and the environment of `wrapper` function,
and the name and the length of `unwrapped` function.

For now, I'm using 3 names for each function:
  * wrapper   -- a function defined inside AsyncFunction_wrap
  * unwrapped -- a function created by compiling actual async function
  * wrapped   -- a function created by CreateAsyncFunction
                 its script and environment are `wrapper`'s,
                 and name and length are `unwrapped`'s.

  wrapper
     |
     | script, environment
     |
     |   unwrapped
     |       |
     |       | name, length
     |       |
     v       v
CreateAsyncFunction
     |
     v
  wrapped

Comment 130

[Tooru Fujisawa [:arai]]

Attachment: Part 11: Support async function in Function.prototype.toString.

Moved most part to Part 10.

Comment 131

[Tooru Fujisawa [:arai]]

just as a note, |(async function f(){}).isGenerator()| returns false.
not sure how people expects the value to be tho.
(related to bug 1119777)

Comment 132

[Tooru Fujisawa [:arai]]

Filed bug 1275240 for redeclaration of formal parameter

Comment 133

[Tooru Fujisawa [:arai]]

Attachment: Part 1: Refactor JSOP_DEFFUN. r=efaust,jandem

rebasing on bug 911216, and some trivial fix on some parts.

Comment 134

[Tooru Fujisawa [:arai]]

Attachment: Part 2: Add parser support for Async functions. r=efaust,jwalden

Comment 135

[Tooru Fujisawa [:arai]]

Attachment: Part 3: Implement async functions. r=efaust,jandem

Comment 136

[Tooru Fujisawa [:arai]]

Attachment: Part 4: Fix function name for export default async function. r=efaust

Comment 137

[Tooru Fujisawa [:arai]]

Attachment: Part 5: Treat yield as an identifier in async function parameters and body. r=efaust

Comment 138

[Tooru Fujisawa [:arai]]

Attachment: Part 6: Support async arrow function. r=efaust

Comment 139

[Tooru Fujisawa [:arai]]

Attachment: Part 6.1: Fix yield handling in async function.

details in comment #109

Comment 140

[Tooru Fujisawa [:arai]]

Attachment: Part 6.2: Fix await handling in async function.

details in comment #110

Comment 141

[Tooru Fujisawa [:arai]]

Attachment: Part 7: Add parser test for async/await. r=efaust

Comment 142

[Tooru Fujisawa [:arai]]

Attachment: Part 8: Add Reflect.parse test for async/await. r=efaust

Comment 143

[Tooru Fujisawa [:arai]]

Attachment: Part 9: Fix syntax priority of async function expression.

details in comment #119

Comment 144

[Tooru Fujisawa [:arai]]

Attachment: Part 10: Add AsyncFunction constructor and prototype.

details in comment #129

Comment 145

[Tooru Fujisawa [:arai]]

Attachment: Part 11: Support async function in Function.prototype.toString.

details in comment #122 and comment #130

Comment 146

[Tooru Fujisawa [:arai]]

Comment on attachment 8771761 [details] [diff] [review]
Part 6.1: Fix yield handling in async function.

will rebase all these patches and cleanup the patch order/granularity.

Comment 147

[Tooru Fujisawa [:arai]]

with new EnvironmentObject, there seems to need some more tricks to mix wrapper/unwrapped functions.
will try finding a solution.

Comment 148

[Tooru Fujisawa [:arai]]

The issue is that, we need pseudo NamedLambdaObject for named async function expression,
that points wrapped function, instead of unwrapped function.

  var wrapped = async function func() {
    return func;
  };
  var resolved;
  wrapped().then(r=>resolved=r);
  drainJobQueue();
  assertEq(resolved, wrapped);

Comment 149

[Tooru Fujisawa [:arai]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1da939fe950c

Comment 150

[Tooru Fujisawa [:arai]]

Attachment: Part 0.1: Refactor JSOP_DEFFUN. r=efaust,jandem

just rebased.
no change

Comment 151

[Tooru Fujisawa [:arai]]

Attachment: Part 0.2: Pass GeneratorKind to Parser::functionFormalParametersAndBody and calculate YieldHandling inside it. r?efaust

Later in the patch series, we'll need to calculate YieldHandling for function arguments and body separately.
As a preparation, changed Parser::functionFormalParametersAndBody to receive GeneratorKind instead of YieldHandling, and calculate YieldHandling inside it.

Comment 152

[Tooru Fujisawa [:arai]]

Attachment: Part 0.3: Pass YieldHandling to Parser::checkYieldNameValidity for later use. r?efaust

Also, later in the patch, we need to check yield name validity depends on current yield handling.
As a preparation, changed the signature of Parser::checkYieldNameValidity to receive YieldHandling.

Comment 153

[Tooru Fujisawa [:arai]]

Attachment: Part 0.4: Pass YieldHandling to Parser::functionFormalParametersAndBody for later use. r?efaust

One more preparation.
Parser::functionFormalParametersAndBody was changed to receive GeneratorKind instead of YieldHandling, but actually we need another YieldHandling parameter, that is the YieldHandling for outside of the function (instead of inside the function, that was previous one), that affects the YieldHandling of async arrow arguments.

YieldHandling enum is extended to 3 states YieldIsName, YieldIsKeyword, and YieldIsNotInherited.
YieldIsNotInherited explicitly represents yield handling is not inherited for the context, like non-arrow.

Comment 154

[Tooru Fujisawa [:arai]]

Attachment: Part 1: Add AsyncFunction flag in FunctionBox, JSScript, and LazyScript. r=efaust

Separated the change for FunctionBox, JSScript, and LazyScript.
no changes except minor styling.

Comment 155

[Tooru Fujisawa [:arai]]

Attachment: Part 2: Add FunctionAsyncKind parameter to Parser methods that receives GeneratorKind. r=efaust

For most cases, we need FunctionAsyncKind parameter where the Parser methods receives GeneratorKind.
Separated the change for it.
no changes except rebase on different new methods from frontend rewrite.

Comment 156

[Tooru Fujisawa [:arai]]

Attachment: Part 3: Add await token. r=efaust

Separated the change for TokenStream, for TOK_AWAIT.
Removed unused TOK_ASYNC.
No other changes except moving awaitIsKeyword to private.

Comment 157

[Tooru Fujisawa [:arai]]

Attachment: Part 4: Add AutoAwaitIsKeyword class to set TokenStream.awaitIsKeyword. r?efaust

Previously we were directly modifying awaitIsKeyword property, but it should be better the property is changed back to original state automatically when leaving the context.
So added AutoAwaitIsKeyword class that sets and resets awaitIsKeyword.

Comment 158

[Tooru Fujisawa [:arai]]

Attachment: Part 5.1: Support async function declaration in Parser. r=efaust,jwalden

Separated async function declaration part.
no changes.

Comment 159

[Tooru Fujisawa [:arai]]

Attachment: Part 5.2: Support async function declaration in Reflect.parse. r=efaust

Also separated async function declaration part of Reflect.parse.
no changes.

Comment 160

[Tooru Fujisawa [:arai]]

Attachment: Part 5.3: Support await expression in Parser. r?efaust

Await handling was changed in previous not-yet-reviewed patch.
Separated that part.

  * Added ParseHandler::newAwaitExpression, instead of reusing newYieldExpression.
  * Throw error when destructuring parameter contains await,
    just in the same way as yield

Comment 161

[Tooru Fujisawa [:arai]]

Attachment: Part 5.4: Support await expression in Reflect.parse. r=efaust

Separated await part of Reflect.parse.
no changes.

Comment 162

[Tooru Fujisawa [:arai]]

Attachment: Part 5.5: Add parser test for async function declaration. r=efaust

Separated basic parser test for async function declaration.
no changes except BUGNUMBER and summary for each test,
and using anonymous function in shell.js.

Comment 163

[Tooru Fujisawa [:arai]]

Attachment: Part 5.6: Add parser test for yield in async function declaration. r?efaust

Test for yield handling in async function.

Comment 164

[Tooru Fujisawa [:arai]]

Attachment: Part 6.1: Support async function expression in Parser. r?efaust

Async function expression's priority was fixed in previous not-yet-reviewed patch.
separated that part.

Comment 165

[Tooru Fujisawa [:arai]]

Attachment: Part 6.2: Add parser test for async function expression. r=efaust

Separated async function expression part.
no changes.

Comment 166

[Tooru Fujisawa [:arai]]

Attachment: Part 6.3: Add parser test for yield in async function expression. r?efaust

test for yield in async function expression.

Comment 167

[Tooru Fujisawa [:arai]]

Attachment: Part 7.1: Support async method in Parser. r=efaust,jwalden

Separated async method handling.
no changes.

Comment 168

[Tooru Fujisawa [:arai]]

Attachment: Part 7.2: Add parser test for async method. r=efaust

Separated async method part.
no changes.

Comment 169

[Tooru Fujisawa [:arai]]

Attachment: Part 7.3: Add parser test for yield in async method. r?efaust

test for yield in async method.

Comment 170

[Tooru Fujisawa [:arai]]

Attachment: Part 8.1: Treat await as keyword in module. r=efaust

Changed to use AutoAwaitIsKeyword in module body.
no other changes.

Comment 171

[Tooru Fujisawa [:arai]]

Attachment: Part 8.2: Add parser test for await in module. r=efaust

Separated async function test for modules.
no changes.

Comment 172

[Tooru Fujisawa [:arai]]

Attachment: Part 9.1: Support async function statement in export default in Parser. r=efaust

Separated async function in export default.
no changes.

Comment 173

[Tooru Fujisawa [:arai]]

Attachment: Part 9.2: Add parser test for async function statement in export default. r=efaust

Separated test for async function in export default.
no changes.

Comment 174

[Tooru Fujisawa [:arai]]

Attachment: Part 9.3: Add parser test for yield in async function statement in export default. r?efaust

test for yield in async function in export default.

Comment 175

[Tooru Fujisawa [:arai]]

Attachment: Part 10.1: Support async arrow function in Parser. r?efaust

Now YieldHandling is calculated separately for arguments and body, especially for async arrow.
Async arrow function inherits yield handling from enclosing context.

other await/yield handling is separated to Part 5.3.

Comment 176

[Tooru Fujisawa [:arai]]

Attachment: Part 10.2: Support async arrow function in Reflect.parse. r=efaust

Separated async arrow part of Reflect.parse.
no changes.

Comment 177

[Tooru Fujisawa [:arai]]

Attachment: Part 10.3: Add parser test for async arrow function. r=efaust

Separated test for async arrow.
no changes.

Comment 178

[Tooru Fujisawa [:arai]]

Attachment: Part 11.1: Implement async functions. r?efaust

Previous part 10.
No change from previous one except NamedLambdaObject.

Summary from comment 129, and some extra.

Async function instance is created by CreateAsyncFunction intrinsic, called from AsyncFunction_wrap.

There a new function is created with %AsyncFunctionPrototype%,
reusing the script and the environment of `wrapper` function,
and the name and the length of `unwrapped` function.

For now, I'm using 3 names for each function:
  * wrapper   -- a function defined inside AsyncFunction_wrap
  * unwrapped -- a function created by compiling actual async function
  * wrapped   -- a function created by CreateAsyncFunction
                 its script and environment are `wrapper`'s,
                 and name and length are `unwrapped`'s.

  wrapper
     |
     | script, environment
     |
     |   unwrapped
     |       |
     |       | name, length
     |       |
     v       v
CreateAsyncFunction
     |
     v
  wrapped


`unwrapped` function can be retrieved from `wrapped` by `GetUnwrappedAsyncFunction`,
and `wrapped` can be retrieved from `unwrapped` by `GetWrappedAsyncFunction`.
Both references are stored in 2nd extended slot.

Here, fun->isAsync() is true for `unwrapped`, not `wrapped`.
`IsWrappedAsyncFunction` can be used to detect `wrapped`, it checks self hosted name.

If unwrapped function is named lambda, NamedLambdaObject is created with wrapped function for the name binding.
NamedLambdaObject::create is modified to accept `callee` and function value separately, to support this.

Comment 179

[Tooru Fujisawa [:arai]]

Attachment: Part 11.2: Add helper functions for async/await test. r?efaust.

With previous helper functions, it's hard to see where the test fails,
so changed it to use drainJobQueue and check instantly.
So that if one test fails, we can see the stack trace.

Comment 180

[Tooru Fujisawa [:arai]]

Attachment: Part 11.3: Add semantics test for async/await. r=efaust.

test for semantics, no changes except BUGNUMBER/summary headers and changes from helper functions.

Comment 181

[Tooru Fujisawa [:arai]]

Attachment: Part 11.4: Add function length test for async function. r?efaust.

Added test for length property, as we're emulating unwrapped length in wrapped function.

Comment 182

[Tooru Fujisawa [:arai]]

Attachment: Part 11.5: Add async function constructor and prototype. r?efaust.

Added constructor/prototype tests.

Comment 183

[Tooru Fujisawa [:arai]]

Attachment: Part 11.6: Add test for async function expression binding identity. r?efaust

named lambda now uses NamedLambdaObject to store/get the binding.
added test for it.

Comment 184

[Tooru Fujisawa [:arai]]

Attachment: Part 12: Return wrapped function for arguments.callee. r?efaust

`arguments.callee` should return some value that makes sense.
without this patch, it returns `unwrapped` function, that is not exposed elsewhere.
changed it to return `wrapped`.

maybe we could just throw error, or return undefined, as there is no backward compat issue,
as this is implementation dependent.

Comment 185

[Tooru Fujisawa [:arai]]

Attachment: Part 13: Support async function in Function.prototype.toString. r?efaust

async function should return string representation of `unwrapped`, for toString call on `wrapped`,
so if it detects the `this` is async, calls FunctionToString with `unwrapped`.

Comment 186

[Tooru Fujisawa [:arai]]

Comment on attachment 8785841 [details] [diff] [review]
Part 0.3: Pass YieldHandling to Parser::checkYieldNameValidity for later use. r?efaust

checkYieldNameValidity is now removed.

Comment 187

[Tooru Fujisawa [:arai]]

Attachment: Part 0.1: Refactor JSOP_DEFFUN. r=efaust,jandem

rebased.

will post remaining patches with bzexport.

Comment 188

[Tooru Fujisawa [:arai]]

Attachment: Part 0.2: Pass GeneratorKind to Parser::functionFormalParametersAndBody and calculate YieldHandling inside it

Comment 189

[Tooru Fujisawa [:arai]]

no... I wasn't about to steal assignee :/

Comment 190

[Tooru Fujisawa [:arai]]

Attachment: Part 0.3: Pass YieldHandling to Parser::functionFormalParametersAndBody for later use

Comment 191

[Tooru Fujisawa [:arai]]

Attachment: Part 1: Add AsyncFunction flag in FunctionBox, JSScript, and LazyScript

Comment 192

[Tooru Fujisawa [:arai]]

Attachment: Part 2: Add FunctionAsyncKind parameter to Parser methods that receives GeneratorKind

Comment 193

[Tooru Fujisawa [:arai]]

Attachment: Part 3: Add await token

Comment 194

[Tooru Fujisawa [:arai]]

Attachment: Part 4: Add AutoAwaitIsKeyword class to set TokenStream.awaitIsKeyword

Comment 195

[Tooru Fujisawa [:arai]]

Attachment: Part 5.1: Support async function declaration in Parser

Comment 196

[Tooru Fujisawa [:arai]]

Attachment: Part 5.2: Support async function declaration in Reflect.parse

Comment 197

[Tooru Fujisawa [:arai]]

Attachment: Part 5.3: Support await expression in Parser

Comment 198

[Tooru Fujisawa [:arai]]

Attachment: Part 5.4: Support await expression in Reflect.parse

Comment 199

[Tooru Fujisawa [:arai]]

Attachment: Part 5.5: Add parser test for async function declaration

Comment 200

[Tooru Fujisawa [:arai]]

Attachment: Part 5.6: Add parser test for yield in async function declaration

Comment 201

[Tooru Fujisawa [:arai]]

Attachment: Part 6.1: Support async function expression in Parser

Comment 202

[Tooru Fujisawa [:arai]]

Attachment: Part 6.2: Add parser test for async function expression

Comment 203

[Tooru Fujisawa [:arai]]

Attachment: Part 6.3: Add parser test for yield in async function expression

Comment 204

[Tooru Fujisawa [:arai]]

Attachment: Part 7.1: Support async method in Parser

Comment 205

[Tooru Fujisawa [:arai]]

Attachment: Part 7.2: Add parser test for async method

Comment 206

[Tooru Fujisawa [:arai]]

Attachment: Part 7.3: Add parser test for yield in async method

Comment 207

[Tooru Fujisawa [:arai]]

Attachment: Part 8.1: Treat await as keyword in module

Comment 208

[Tooru Fujisawa [:arai]]

Attachment: Part 8.2: Add parser test for await in module

Comment 209

[Tooru Fujisawa [:arai]]

Attachment: Part 9.1: Support async function statement in export default in Parser

Comment 210

[Tooru Fujisawa [:arai]]

Attachment: Part 9.2: Add parser test for async function statement in export default

Comment 211

[Tooru Fujisawa [:arai]]

Attachment: Part 9.3: Add parser test for yield in async function statement in export default