Closed Bug 1194022 Opened 5 years ago Closed 5 years ago

Assertion failure: next.type != TOK_DIV && next.type != TOK_REGEXP (next token requires contextual specifier to be parsed unambiguously), at frontend/TokenStream.h

Categories

(Core :: JavaScript Engine, defect)

x86_64
macOS
defect
Not set
critical

Tracking

()

RESOLVED FIXED
mozilla43
Tracking Status
firefox43 --- fixed

People

(Reporter: gkw, Assigned: arai)

References

Details

(Keywords: assertion, regression, testcase, Whiteboard: [fuzzblocker][jsbugmon:update])

Attachments

(2 files)

function f()
    yield
    /x/

asserts js debug shell on m-c changeset 7649ffe28b67 with --fuzzing-safe --no-threads --no-ion at Assertion failure: next.type != TOK_DIV && next.type != TOK_REGEXP (next token requires contextual specifier to be parsed unambiguously), at frontend/TokenStream.h.

Configure options:

CC="clang -Qunused-arguments" CXX="clang++ -Qunused-arguments" AR=ar AUTOCONF=/usr/local/Cellar/autoconf213/2.13/bin/autoconf213 sh /Users/skywalker/trees/mozilla-central/js/src/configure --target=x86_64-apple-darwin12.5.0 --enable-debug --enable-nspr-build --enable-more-deterministic --with-ccache --enable-gczeal --enable-debug-symbols --disable-tests

python -u ~/funfuzz/js/compileShell.py -b "--enable-debug --enable-more-deterministic --enable-nspr-build" -r 7649ffe28b67

=== Treeherder Build Bisection Results by autoBisect ===

The "good" changeset has the timestamp "20150812050310" and the hash "2c3c8a927e3fcc3c7d63c2ecf5ff9a189d1cbd96".
The "bad" changeset has the timestamp "20150812054307" and the hash "a8493abd3c62b77e4741da181239af5549d84f80".

Likely regression window: https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=2c3c8a927e3fcc3c7d63c2ecf5ff9a189d1cbd96&tochange=a8493abd3c62b77e4741da181239af5549d84f80

Setting needinfo? from :arai as per the regression window.

In the meantime, trying to get a better bisection.
Flags: needinfo?(arai.unmht)
Attached file stack
(lldb) bt 5
* thread #1: tid = 0x1f21aa, 0x000000010004caa4 js-dbg-64-dm-nsprBuild-darwin-7649ffe28b67`js::frontend::TokenStream::addModifierException(this=<unavailable>, modifierException=<unavailable>) + 372 at TokenStream.h:437, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x000000010004caa4 js-dbg-64-dm-nsprBuild-darwin-7649ffe28b67`js::frontend::TokenStream::addModifierException(this=<unavailable>, modifierException=<unavailable>) + 372 at TokenStream.h:437
    frame #1: 0x000000010004ea7a js-dbg-64-dm-nsprBuild-darwin-7649ffe28b67`js::frontend::Parser<js::frontend::FullParseHandler>::yieldExpression(this=0x00007fff5fbfe658, inHandling=<unavailable>) + 554 at Parser.cpp:5771
    frame #2: 0x0000000100049851 js-dbg-64-dm-nsprBuild-darwin-7649ffe28b67`js::frontend::Parser<js::frontend::FullParseHandler>::assignExpr(this=<unavailable>, inHandling=InAllowed, yieldHandling=<unavailable>, invoked=<unavailable>) + 705 at Parser.cpp:6699
    frame #3: 0x0000000100048e46 js-dbg-64-dm-nsprBuild-darwin-7649ffe28b67`js::frontend::Parser<js::frontend::FullParseHandler>::functionBody(this=0x00007fff5fbfe658, inHandling=InAllowed, yieldHandling=YieldIsName, kind=Statement, type=ExpressionBody) + 150 at Parser.cpp:1046
    frame #4: 0x000000010004a538 js-dbg-64-dm-nsprBuild-darwin-7649ffe28b67`js::frontend::Parser<js::frontend::FullParseHandler>::functionArgsAndBodyGeneric(this=0x00007fff5fbfe658, inHandling=InAllowed, yieldHandling=YieldIsName, pn=0x00000001028ae020, fun=<unavailable>, kind=Statement) + 472 at Parser.cpp:2694
(lldb)
thank you for finding it out!
it should be bug 1089045.
I'll try fixing it shortly
Blocks: 1089045
autoBisect shows this is probably related to the following changeset:

The first bad revision is:
changeset:   https://hg.mozilla.org/mozilla-central/rev/05f838caf076
user:        Tooru Fujisawa
date:        Fri Aug 07 04:11:59 2015 +0900
summary:     Bug 1089045 - Part 1: Supply consistent modifiers to TokenStream. r=Waldo

Yes, you're right! :)
Whiteboard: [jsbugmon:update] → [fuzzblocker][jsbugmon:update]
This is triggering often due to its simplicity, so setting as [fuzzblocker].
this is the most exceptional case with modifier.

the TOK_REGEXP token is gotten with Operand by yieldExpression at first, and with None by MatchOrInsertSemicolon, and finally with Operand by unaryExpr in next statement.  We know that there ASI will happen and the token will be finally gotten with Operand, while getting the token in yieldExpression, as it gets TOK_EOL.

Simply we should add another type of exception for EOL after yield, which allows TOK_REGEXP but TOK_DIV.
I'll post patch shortly
Flags: needinfo?(arai.unmht)
Added NoneIsOperandYieldEOL exception, which is specified when yield expression has no operand and is followed by EOL or EOF.  Then, if we handle that with dedicated exception, we don't have to use bitset, but just the ModifierException enum itself, as there's no mo more mixed state.

Now, addModifierException can be called for a token with no exception, or a token after yield without operand.  In latter case, modifierException should be OperandIsNone, which is for ASI after the yield  expression.  in that case, token's exception is not updated, since the token is already gotten with Operand, and OperandIsNone has no meaning.

This doesn't happen with break or continue, because MatchOrInsertSemicolon is called with Operand for them.

Then, I found that comments for NoneIsOperand and OperandIsNone were swapped in last commit, fixed in this patch, sorry.
Assignee: nobody → arai.unmht
Attachment #8647314 - Flags: review?(jwalden+bmo)
Comment on attachment 8647314 [details] [diff] [review]
Add another exception for a token after yield expression without operand and followed by EOL.

Review of attachment 8647314 [details] [diff] [review]:
-----------------------------------------------------------------

I don't see anything wrong here, but my mind is fuzzing over the more I look at this.  If this stuff weren't finding bugs I would rip it out in a heartbeat as basically inscrutable.  :-(  Alas.

::: js/src/frontend/TokenStream.h
@@ +458,5 @@
> +                           "next token requires contextual specifier to be parsed unambiguously");
> +
> +            // Do not update modifierException.
> +            return;
> +        }

Blank line after }.
Attachment #8647314 - Flags: review?(jwalden+bmo) → review+
https://hg.mozilla.org/mozilla-central/rev/a466d341647f
Status: NEW → RESOLVED
Closed: 5 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → mozilla43
You need to log in before you can comment on or make changes to this bug.