1545751 - [BinAST] Update the tokenizer API to transmit context information.

Status: RESOLVED FIXED

(Core :: JavaScript Engine, task)

Comment 1

[David Teller [:Yoric] - still alive but not very active]

Attachment: Bug 1545751 - In BinAST, pass Context while parsing;r?arai

The Context 0.1 format specifies that the (de)compression of nodes is determined by:

• its node type;
• its parent type;
• the index in the parent node;
• whether the node is an element of an array.

This patch modifies the parser generator to pass the information during parsing.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a4847e3f634e
In BinAST, pass Context while parsing;r=arai

Comment 3

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/29be14061be5
Backed out changeset a4847e3f634e for bustages on BinASTTokenReaderBase. CLOSED TREE

Comment 4

[Cosmin Sabou [:CosminS]]

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=classified&fromchange=a4847e3f634ec14d42e78cbf0be98314080726d4&tochange=29be14061be5681129c2a3b31d6fef6a3e247657&selectedJob=242253061

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=242253008&repo=autoland&lineNumber=4753

Backout link: https://hg.mozilla.org/integration/autoland/rev/29be14061be5681129c2a3b31d6fef6a3e247657

[task 2019-04-24T09:42:17.802Z] 09:42:17 INFO - In file included from /builds/worker/workspace/build/src/js/src/frontend/BinASTParser.cpp:12:
[task 2019-04-24T09:42:17.803Z] 09:42:17 INFO - In file included from /builds/worker/workspace/build/src/js/src/frontend/BinASTParser.h:18:
[task 2019-04-24T09:42:17.803Z] 09:42:17 INFO - In file included from /builds/worker/workspace/build/src/js/src/frontend/BinASTParserPerTokenizer.h:23:
[task 2019-04-24T09:42:17.803Z] 09:42:17 INFO - In file included from /builds/worker/workspace/build/src/js/src/frontend/BinASTTokenReaderMultipart.h:14:
[task 2019-04-24T09:42:17.803Z] 09:42:17 ERROR - /builds/worker/workspace/build/src/js/src/frontend/BinASTTokenReaderBase.h:32:21: error: aggregate initialization of type 'js::frontend::BinASTTokenReaderBase::Context' with user-declared constructors is incompatible with C++2a [-Werror,-Wc++2a-compat]
[task 2019-04-24T09:42:17.803Z] 09:42:17 INFO - return Context{BinASTKind::_Null, 0, ElementOf::TaggedTuple};
[task 2019-04-24T09:42:17.803Z] 09:42:17 INFO - ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[task 2019-04-24T09:42:17.807Z] 09:42:17 ERROR - /builds/worker/workspace/build/src/js/src/frontend/BinASTTokenReaderBase.h:36:21: error: aggregate initialization of type 'js::frontend::BinASTTokenReaderBase::Context' with user-declared constructors is incompatible with C++2a [-Werror,-Wc++2a-compat]
[task 2019-04-24T09:42:17.809Z] 09:42:17 INFO - return Context{kind, fieldIndex, ElementOf::Array};
[task 2019-04-24T09:42:17.810Z] 09:42:17 INFO - ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[task 2019-04-24T09:42:17.810Z] 09:42:17 ERROR - /builds/worker/workspace/build/src/js/src/frontend/BinASTTokenReaderBase.h:41:21: error: aggregate initialization of type 'js::frontend::BinASTTokenReaderBase::Context' with user-declared constructors is incompatible with C++2a [-Werror,-Wc++2a-compat]
[task 2019-04-24T09:42:17.811Z] 09:42:17 INFO - return Context{kind, 0, ElementOf::TaggedTuple};
[task 2019-04-24T09:42:17.812Z] 09:42:17 INFO - ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[task 2019-04-24T09:42:17.812Z] 09:42:17 INFO - 3 errors generated.
[task 2019-04-24T09:42:17.813Z] 09:42:17 INFO - /builds/worker/workspace/build/src/config/rules.mk:805: recipe for target 'BinASTParser.o' failed
[task 2019-04-24T09:42:17.814Z] 09:42:17 ERROR - make[4]: *** [BinASTParser.o] Error 1
[task 2019-04-24T09:42:17.814Z] 09:42:17 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/js/src/frontend'
[task 2019-04-24T09:42:17.815Z] 09:42:17 INFO - /builds/worker/workspace/build/src/config/recurse.mk:74: recipe for target 'js/src/frontend/target' failed
[task 2019-04-24T09:42:17.816Z] 09:42:17 ERROR - make[3]: *** [js/src/frontend/target] Error 2
[task 2019-04-24T09:42:17.816Z] 09:42:17 INFO - make[3]: *** Waiting for unfinished jobs....

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d0eebfb3d916
In BinAST, pass Context while parsing;r=arai

Comment 6

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out changeset d0eebfb3d916 (Bug 1545751) for spidermonkey builds bustages

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=d0eebfb3d9160645793e1ad8a04b89b76faad529&selectedJob=242591868

Backout link: https://hg.mozilla.org/integration/autoland/rev/48eeeb94b5c2a246fdd78cf445a903b58716e825

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=242591868&repo=autoland&lineNumber=17660

[task 2019-04-25T14:17:34.134Z] TEST-PASS | js/src/jit-test/tests/binast/lazy/basic/bug586499-regexp.binjs | Success (code 0, args "--no-baseline --no-ion --more-compartments") [0.0 s]
[task 2019-04-25T14:17:34.138Z] TEST-PASS | js/src/jit-test/tests/binast/lazy/basic/bug586499-regexp.binjs | Success (code 0, args "--baseline-eager") [0.0 s]
[task 2019-04-25T14:17:34.149Z] /builds/worker/workspace/build/src/js/src/jit-test/tests/binast/lazy/basic/bug586917.binjs:SyntaxError: BinAST Parsing Error: Buffer exceeds length
[task 2019-04-25T14:17:34.149Z] Exit code: 3
[task 2019-04-25T14:17:34.149Z] FAIL - binast/lazy/basic/bug586917.binjs
[task 2019-04-25T14:17:34.149Z] TEST-UNEXPECTED-FAIL | js/src/jit-test/tests/binast/lazy/basic/bug586917.binjs | /builds/worker/workspace/build/src/js/src/jit-test/tests/binast/lazy/basic/bug586917.binjs:SyntaxError: BinAST Parsing Error: Buffer exceeds length (code 3, args "") [0.0 s]
[task 2019-04-25T14:17:34.150Z] INFO exit-status : 3
[task 2019-04-25T14:17:34.150Z] INFO timed-out : False
[task 2019-04-25T14:17:34.150Z] INFO stderr 2> /builds/worker/workspace/build/src/js/src/jit-test/tests/binast/lazy/basic/bug586917.binjs:SyntaxError: BinAST Parsing Error: Buffer exceeds length
[task 2019-04-25T14:17:34.156Z] /builds/worker/workspace/build/src/js/src/jit-test/tests/binast/lazy/basic/bug586917.binjs:SyntaxError: BinAST Parsing Error: Buffer exceeds length
[task 2019-04-25T14:17:34.156Z] Exit code: 3
[task 2019-04-25T14:17:34.156Z] FAIL - binast/lazy/basic/bug586917.binjs
[task 2019-04-25T14:17:34.157Z] TEST-UNEXPECTED-FAIL | js/src/jit-test/tests/binast/lazy/basic/bug586917.binjs | /builds/worker/workspace/build/src/js/src/jit-test/tests/binast/lazy/basic/bug586917.binjs:SyntaxError: BinAST Parsing Error: Buffer exceeds length (code 3, args "--ion-eager --ion-offthread-compile=off --more-compartments") [0.0 s]
[task 2019-04-25T14:17:34.157Z] INFO exit-status : 3
[task 2019-04-25T14:17:34.157Z] INFO timed-out : False
[task 2019-04-25T14:17:34.157Z] INFO stderr 2> /builds/worker/workspace/build/src/js/src/jit-test/tests/binast/lazy/basic/bug586917.binjs:SyntaxError: BinAST Parsing Error: Buffer exceeds length
[task 2019-04-25T14:17:34.157Z] /builds/worker/workspace/build/src/js/src/jit-test/tests/binast/lazy/basic/bug586917.binjs:SyntaxError: BinAST Parsing Error: Buffer exceeds length
[task 2019-04-25T14:17:34.158Z] Exit code: 3
[task 2019-04-25T14:17:34.158Z] FAIL - binast/lazy/basic/bug586917.binjs
[task 2019-04-25T14:17:34.158Z] TEST-UNEXPECTED-FAIL | js/src/jit-test/tests/binast/lazy/basic/bug586917.binjs | /builds/worker/workspace/build/src/js/src/jit-test/tests/binast/lazy/basic/bug586917.binjs:SyntaxError: BinAST Parsing Error: Buffer exceeds length (code 3, args "--ion-eager --ion-offthread-compile=off --ion-check-range-analysis --ion-extra-checks --no-sse3 --no-threads") [0.0 s]
[task 2019-04-25T14:17:34.158Z] INFO exit-status : 3
[task 2019-04-25T14:17:34.158Z] INFO timed-out : False

Comment 7

[David Teller [:Yoric] - still alive but not very active]

This is really, really odd. The change that I'm landing should effectively be a noop.

Comment 8

[David Teller [:Yoric] - still alive but not very active]

Ahah, a uint32_t had been mistakenly turned into a uint8_t. Doesn't work so well for a test that fills it with 1000+ elements.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e516adb0ff9e
In BinAST, pass Context while parsing;r=arai

Comment 10

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/e516adb0ff9e