1639547 - Perma SM(cgc) line Assertion failure: source->hasUncompressedSource(), at /builds/worker/workspace/build/src/js/src/vm/JSScript.cpp:2625 even for successful tasks

Status: RESOLVED FIXED

(Core :: JavaScript: Internationalization API, defect)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Click on the "Failure Summary" tab for the top-most.

The tasks are shown as successful.

Log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=302888851&repo=autoland&lineNumber=53473

[task 2020-05-19T12:35:12.184Z] TEST-PASS | testThTesting with principals 'IsSystem'
[task 2020-05-19T12:35:12.213Z] Testing with principals 'IsNotSystem'
[task 2020-05-19T12:35:12.239Z] Testing with principals 'testPrincipals'
[task 2020-05-19T12:35:12.269Z] Testing with principals 'nullptr principals'
[task 2020-05-19T12:35:15.767Z] Assertion failure: source->hasUncompressedSource(), at /builds/worker/workspace/build/src/js/src/vm/JSScript.cpp:2625
[task 2020-05-19T12:35:15.780Z] in directory /builds/worker/workspace/build/src/obj-spider, running ['setarch', 'x86_64', '-R', 'make', 'check-jstests']
[task 2020-05-19T12:35:15.784Z] make -C js/src check-jstests
[task 2020-05-19T12:35:15.790Z] make[1]: Entering directory '/builds/worker/workspace/build/src/obj-spider/js/src'
[task 2020-05-19T12:35:15.790Z] ../../dist/bin/run-mozilla.sh /builds/worker/workspace/build/src/obj-spider/_virtualenvs/init_py3/bin/python -u /builds/worker/workspace/build/src/js/src/tests/jstests.py
[task 2020-05-19T12:35:15.790Z] --no-progress --format=automation --timeout 300
[task 2020-05-19T12:35:15.790Z] --args='--dll /builds/worker/workspace/breakpad-tools/libbreakpadinjector.so' --exclude-file=/builds/worker/workspace/build/src/js/src/devtools/automation/cgc-jstests-slow.txt
[task 2020-05-19T12:35:15.790Z] ../../dist/bin/js

Comment 1

[André Bargull [:anba]]

This looks like a pre-existing issue, uncovered by bug 1557727, because GC timings changed.

Comment 2

[André Bargull [:anba]]

The test calls js::SynchronouslyCompressSource, which in turn calls js::RunPendingSourceCompressions. RunPendingSourceCompressions calls GlobalHelperThreadState::startHandlingCompressionTasks to start any pending compression tasks via GlobalHelperThreadState::scheduleCompressionTasks. But scheduleCompressionTasks calls SourceCompressionTask::shouldStart to decide which tasks should be queued and shouldStart depends on the current GC count.

Adding an override to scheduleCompressionTasks to start any pending jobs even when shouldStart returns false, fixes the bug.

Comment 3

[André Bargull [:anba]]

Try looks good: https://treeherder.mozilla.org/#/jobs?repo=try&revision=01b82032ae69c77586fda3ee31b105b68a465d2b

And a debug output also confirmes my hypothesis from comment #2:

JS_GC_ZEAL=IncrementalMultipleSlices dist/bin/jsapi-tests testScriptSourceCompression_spansMultipleMiddleChunks
testScriptSourceCompression_spansMultipleMiddleChunks
Sync compress for source-id: 2
pending source-id 2, can start=no
Sync compress for source-id: 4
pending source-id 4, can start=no
TEST-PASS | testScriptSourceCompression_spansMultipleMiddleChunks | ok

Comment 4

[André Bargull [:anba]]

Attachment: Bug 1639547: Add override switch to startHandlingCompressionTasks. r=jonco!

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1557727

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7c21a5f6881c
Add override switch to startHandlingCompressionTasks. r=jonco

Comment 7

[Cosmin Sabou [:CosminS]]

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&searchStr=cppunit&fromchange=7c21a5f6881c6b46b5dde69b6e32b1078331e011&tochange=b57a096b5ba4955abe131079820801457ee397af&selectedTaskRun=U3EJ1GfsSlG7Vwkqula41w-0

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=303270035&repo=autoland

Backout link: https://hg.mozilla.org/integration/autoland/rev/e9369a667cd1888711d9c4a5691a38b6c4e3432d

[task 2020-05-21T20:29:21.258Z] 20:29:21     INFO -  TEST-PASS | testThreadingThreadDetach | ok
[task 2020-05-21T20:29:21.258Z] 20:29:21     INFO -  tesTesting with principals 'IsSystem'
[task 2020-05-21T20:29:21.258Z] 20:29:21     INFO -  Testing with principals 'IsNotSystem'
[task 2020-05-21T20:29:21.258Z] 20:29:21     INFO -  Testing with principals 'testPrincipals'
[task 2020-05-21T20:29:21.258Z] 20:29:21     INFO -  Testing with principals 'nullptr principals'
[task 2020-05-21T20:29:21.258Z] 20:29:21     INFO -  AddressSanitizer:DEADLYSIGNAL
[task 2020-05-21T20:29:21.259Z] 20:29:21     INFO -  =================================================================
[task 2020-05-21T20:29:21.259Z] 20:29:21    ERROR -  ==1474==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000001 (pc 0x55dfe03f2d3b bp 0x7ffe84573e90 sp 0x7ffe84573e90 T0)
[task 2020-05-21T20:29:21.259Z] 20:29:21     INFO -  ==1474==The signal is caused by a WRITE memory access.
[task 2020-05-21T20:29:21.259Z] 20:29:21     INFO -  ==1474==Hint: address points to the zero page.
[task 2020-05-21T20:29:21.260Z] 20:29:21     INFO -      #0 0x55dfe03f2d3a in operator()<char16_t, js::SourceRetrievable::No> /builds/worker/checkouts/gecko/js/src/vm/JSScript.h:997:7
[task 2020-05-21T20:29:21.260Z] 20:29:21     INFO -      #1 0x55dfe03f2d3a in decltype(auto) mozilla::detail::VariantImplementation<unsigned char, 6ul, js::ScriptSource::Compressed<char16_t, (js::SourceRetrievable)1>, js::ScriptSource::Uncompressed<char16_t, (js::SourceRetrievable)1>, js::ScriptSource::Retrievable<mozilla::Utf8Unit>, js::ScriptSource::Retrievable<char16_t>, js::ScriptSource::Missing, js::ScriptSource::BinAST>::match<js::ScriptSource::TriggerConvertToCompressedSourceFromTask, mozilla::Variant<js::ScriptSource::Compressed<mozilla::Utf8Unit, (js::SourceRetrievable)0>, js::ScriptSource::Uncompressed<mozilla::Utf8Unit, (js::SourceRetrievable)0>, js::ScriptSource::Compressed<mozilla::Utf8Unit, (js::SourceRetrievable)1>, js::ScriptSource::Uncompressed<mozilla::Utf8Unit, (js::SourceRetrievable)1>, js::ScriptSource::Compressed<char16_t, (js::SourceRetrievable)0>, js::ScriptSource::Uncompressed<char16_t, (js::SourceRetrievable)0>, js::ScriptSource::Compressed<char16_t, (js::SourceRetrievable)1>, js::ScriptSource::Uncompressed<char16_t, (js::SourceRetrievable)1>, js::ScriptSource::Retrievable<mozilla::Utf8Unit>, js::ScriptSource::Retrievable<char16_t>, js::ScriptSource::Missing, js::ScriptSource::BinAST> >(js::ScriptSource::TriggerConvertToCompressedSourceFromTask&&, mozilla::Variant<js::ScriptSource::Compressed<mozilla::Utf8Unit, (js::SourceRetrievable)0>, js::ScriptSource::Uncompressed<mozilla::Utf8Unit, (js::SourceRetrievable)0>, js::ScriptSource::Compressed<mozilla::Utf8Unit, (js::SourceRetrievable)1>, js::ScriptSource::Uncompressed<mozilla::Utf8Unit, (js::SourceRetrievable)1>, js::ScriptSource::Compressed<char16_t, (js::SourceRetrievable)0>, js::ScriptSource::Uncompressed<char16_t, (js::SourceRetrievable)0>, js::ScriptSource::Compressed<char16_t, (js::SourceRetrievable)1>, js::ScriptSource::Uncompressed<char16_t, (js::SourceRetrievable)1>, js::ScriptSource::Retrievable<mozilla::Utf8Unit>, js::ScriptSource::Retrievable<char16_t>, js::ScriptSource::Missing, js::ScriptSource::BinAST>&) /builds/worker/workspace/obj-build/dist/include/mozilla/Variant.h:257:16
[task 2020-05-21T20:29:21.260Z] 20:29:21     INFO -      #2 0x55dfe03c89f0 in match<js::ScriptSource::TriggerConvertToCompressedSourceFromTask> /builds/worker/workspace/obj-build/dist/include/mozilla/Variant.h:795:12
[task 2020-05-21T20:29:21.260Z] 20:29:21     INFO -      #3 0x55dfe03c89f0 in triggerConvertToCompressedSourceFromTask /builds/worker/checkouts/gecko/js/src/vm/JSScript.cpp:2608:8
[task 2020-05-21T20:29:21.260Z] 20:29:21     INFO -      #4 0x55dfe03c89f0 in js::SourceCompressionTask::complete() /builds/worker/checkouts/gecko/js/src/vm/JSScript.cpp:2614:13
[task 2020-05-21T20:29:21.261Z] 20:29:21     INFO -      #5 0x55dfe0204a95 in js::AttachFinishedCompressions(JSRuntime*, js::AutoLockHelperThreadState&) /builds/worker/checkouts/gecko/js/src/vm/HelperThreads.cpp:2360:24
[task 2020-05-21T20:29:21.261Z] 20:29:21     INFO -      #6 0x55dfe0204d82 in js::RunPendingSourceCompressions(JSRuntime*) /builds/worker/checkouts/gecko/js/src/vm/HelperThreads.cpp:2380:3
[task 2020-05-21T20:29:21.261Z] 20:29:21     INFO -      #7 0x55dfe03c8bb5 in js::SynchronouslyCompressSource(JSContext*, JS::Handle<js::BaseScript*>) /builds/worker/checkouts/gecko/js/src/vm/JSScript.cpp:2633:3
[task 2020-05-21T20:29:21.261Z] 20:29:21     INFO -      #8 0x55dfdfbd7638 in CompressSourceSync(JS::Handle<JSFunction*>, JSContext*) /builds/worker/checkouts/gecko/js/src/jsapi-tests/testScriptSourceCompression.cpp:105:3
[task 2020-05-21T20:29:21.261Z] 20:29:21     INFO -      #9 0x55dfdfbdbd3c in bool cls_testScriptSourceCompression_endsAtBoundaryInOneChunk::run<mozilla::Utf8Unit>() /builds/worker/checkouts/gecko/js/src/jsapi-tests/testScriptSourceCompression.cpp:256:3
[task 2020-05-21T20:29:21.262Z] 20:29:21     INFO -      #10 0x55dfdfbdb04e in cls_testScriptSourceCompression_endsAtBoundaryInOneChunk::run(JS::Handle<JSObject*>) /builds/worker/checkouts/gecko/js/src/jsapi-tests/testScriptSourceCompression.cpp:234:3
[task 2020-05-21T20:29:21.262Z] 20:29:21     INFO -      #11 0x55dfdfc97cfa in main /builds/worker/checkouts/gecko/js/src/jsapi-tests/tests.cpp:131:15
[task 2020-05-21T20:29:21.262Z] 20:29:21     INFO -      #12 0x7f4d5b6c8b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
[task 2020-05-21T20:29:21.262Z] 20:29:21     INFO -      #13 0x55dfdfa38dca in _start (/builds/worker/workspace/build/tests/cppunittest/jsapi-tests+0x1477dca)
[task 2020-05-21T20:29:21.263Z] 20:29:21     INFO -  AddressSanitizer can not provide additional info.
[task 2020-05-21T20:29:21.263Z] 20:29:21     INFO -  SUMMARY: AddressSanitizer: SEGV /builds/worker/checkouts/gecko/js/src/vm/JSScript.h:997:7 in operator()<char16_t, js::SourceRetrievable::No>
[task 2020-05-21T20:29:21.263Z] 20:29:21     INFO -  ==1474==ABORTING
[task 2020-05-21T20:29:21.263Z] 20:29:21     INFO -  mozcrash checking /tmp/tmpMBynpW for minidumps...
[task 2020-05-21T20:29:21.263Z] 20:29:21  WARNING -  TEST-UNEXPECTED-FAIL | jsapi-tests | test failed with return code 1
[task 2020-05-21T20:29:21.263Z] 20:29:21     INFO -  TEST-INFO took 14572ms
[task 2020-05-21T20:29:21.264Z] 20:29:21     INFO -  SUITE-END | took 39s
[task 2020-05-21T20:29:21.264Z] 20:29:21     INFO -  Result summary:
[task 2020-05-21T20:29:21.264Z] 20:29:21     INFO -  cppunittests INFO | Passed: 61
[task 2020-05-21T20:29:21.264Z] 20:29:21  WARNING -  cppunittests INFO | Failed: 1
[task 2020-05-21T20:29:21.264Z] 20:29:21  WARNING -  One or more unittests failed.
[task 2020-05-21T20:29:21.296Z] 20:29:21    ERROR - Return code: 1
[task 2020-05-21T20:29:21.296Z] 20:29:21     INFO - TinderboxPrint: cppunittest-cppunittest<br/>61/<em class="testfail">1</em>
[task 2020-05-21T20:29:21.296Z] 20:29:21    ERROR - # TBPL FAILURE #
[task 2020-05-21T20:29:21.297Z] 20:29:21  WARNING - setting return code to 2
[task 2020-05-21T20:29:21.297Z] 20:29:21    ERROR - The cppunittest suite: cppunittest ran with return status: FAILURE

Comment 8

[André Bargull [:anba]]

I think I've found the reason for the other test failure:
js::RunPendingSourceCompressions contains this loop

  // Wait for all in-process compression tasks to complete.
  while (!HelperThreadState().compressionWorklist(lock).empty()) {
    HelperThreadState().wait(lock, GlobalHelperThreadState::CONSUMER);
  }

But the comment doesn't quite match the code, because the loop doesn't ensure all currently active compression threads have actually finished their work. Adding

HelperThreadState().waitForAllThreadsLocked(lock);

after the loop fixes the failure when testing locally.

Btw, the test failure is only (intermittently!) reproducible for me with ASAN + non-debug + optimised builds. For example I didn't manage to reproduce it with ASAN + debug + optimised builds.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8f6fbfc7558c
Add override switch to startHandlingCompressionTasks. r=jonco

Comment 10

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/8f6fbfc7558c