1706560 - Skip failing Fission TSan WPT tests and enable test-linux1804-64-tsan/opt

Status: RESOLVED FIXED

(Testing :: web-platform-tests, defect, P2)

Description

[Chris Peterson [:cpeterson]]

ahal tried to enable WPT for Fission on linux1804-64-tsan/opt in bug 1696042. Unfortunately, there are some new TSan test failures with Fission:

https://treeherder.mozilla.org/jobs?repo=try&revision=e2487391f036693313bd60adfb1ce49516ba959f&searchStr=tsan

Like the tsan failures in mochitest-plain bug 1698193, the failures are nearly all timeouts. The tsan tests are slow and Fission will make them slower. Like bug 1698193, the Fission team is not concerned about these timeouts, since they don't indicate tsan bugs. Perhaps we can run these Fission tsan tests in smaller batches and/or increase the timeout?

Comment 1

[Chris Peterson [:cpeterson]]

Most of these Fission TSan failures are test timeouts. Perhaps we can run these Fission TSan tests in smaller batches and/or increase the timeout?

The one test failure from https://treeherder.mozilla.org/jobs?repo=try&revision=e2487391f036693313bd60adfb1ce49516ba959f&searchStr=tsan that was not an obvious timeout was wpt8:

TEST-UNEXPECTED-FAIL | /html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-parse-error-external-module.html | <script> error: Move parse-error external module script to iframe before-prepare - assert_unreached: Script error evennt fired unexpectedly Reached unreachable code

This looks like bug 1636883 bug 1714790, a rare intermittent test failure that happens without Fission or TSan. Perhaps Fission with TSan makes that rare intermittent more reproducible? I think we can just skip this failing test case (before-prepare-iframe-parse-error-external-module.html) since it is an existing test failure, not consistently-reproducible for Fission without TSan, and not a TSan error.

Comment 2

[Chris Peterson [:cpeterson]]

James, I think this Linux TSan Fission failure is the same as non-Fission bug 1714790 - Intermittent /moving-between-documents/before-prepare-iframe-parse-error-external-module.html | <script> error: Move parse-error external module script to iframe before-prepare - assert_unreached: Script error evennt fired unexpectedly Reached unreachable

If these failures are duplicates, can we go ahead an enable test-linux1804-64-tsan/opt for Fission in this bug now?

This test is already annotated as intermittent for Linux opt, Linux debug, and macOS debug. This test seems pretty flaky already, but these intermittent Linux failures might become more frequent for a slow configuration like Fission + TSan.

https://searchfox.org/mozilla-central/rev/79d73b4aff88dd4a0f06dd3789e1148c49b0de60/testing/web-platform/meta/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-parse-error-external-module.html.ini

[before-prepare-iframe-parse-error-external-module.html]
  [<script> error: Move parse-error external module script to iframe before-prepare]
    expected:
      if (os == "linux") and not fission and not webrender and not debug and (processor == "x86_64"): ["PASS", "FAIL"]
      if (os == "linux") and not fission and not webrender and not debug and (processor == "x86"): ["PASS", "FAIL"]
      if (os == "linux") and not fission and not webrender and debug: ["PASS", "FAIL"]
      if (os == "linux") and not fission and webrender: ["PASS", "FAIL"]
      if (os == "mac") and debug: ["PASS", "FAIL"]

Comment 3

[James Graham [:jgraham]]

Yeah I'm happy for it to be enabled. A few months ago we had general problems with timeouts on TSAN that are now fixed. It would be useful to have a try push with TSAN enabled to update the metadata from, but if you're confident it's just a small number of known issues we can also do it by hand.

Comment 4

[Chris Peterson [:cpeterson]]

Andrew, when you have time, can you please run a try push with Fission WPT enabled for test-linux1804-64-tsan/opt? The try push in comment 0 is two months old, so those failures might not be reproducible today.

(In reply to James Graham [:jgraham] from comment #3)

Yeah I'm happy for it to be enabled. A few months ago we had general problems with timeouts on TSAN that are now fixed. It would be useful to have a try push with TSAN enabled to update the metadata from, but if you're confident it's just a small number of known issues we can also do it by hand.

Can wpt-sync generate the patch to update the WPT expectations from a try push of just one platform (the not-yet-enabled Fission Linux TSAN)? Or does wpt-sync need a full push with all platforms?

Comment 5

[James Graham [:jgraham]]

It's not clever enough to know how to add annotations for a specific configuration without also seeing data for other configurations. So a full try push with retriggers just for the TSAN jobs to cover intermittents would be ideal. I can probably just grab ahal's patches and do that.

Comment 6

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1706560 - [ci] web-platform-test: Run fission linux64-tsan on trunk, r?jgraham!

This enables the following configuration on central / autoland:

test-linux1804-64-tsan/opt-web-platform-tests-fis-e10s

Comment 7

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1706560 - [ci] web-platform-test-reftest: Run fission linux64-tsan/opt on trunk, r?jgraham!

This enables the following configuration on autoland/central:

test-linux1804-64-tsan/opt-web-platform-tests-reftest-fis-e10s

Depends on D110555

Comment 8

[Andrew Halberstadt [:ahal]]

Try push here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=de6cf763be1516dd43c873ec2262aa752b66fef7

Comment 9

[Chris Peterson [:cpeterson]]

(In reply to Andrew Halberstadt [:ahal] from comment #8)

Try push here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=de6cf763be1516dd43c873ec2262aa752b66fef7

Assigning this bug to ahal because he has a patch.

Looks like there were some test failures:

• html/browsers/history/the-history-interface/001.html
  • TEST-UNEXPECTED-FAIL | /html/browsers/history/the-history-interface/001.html | pushState must not alter location.hash when no URL is provided - assert_equals: location.hash changed when an unrelated state was pushed expected "test" but got "test2"
  • TEST-UNEXPECTED-FAIL | /html/browsers/history/the-history-interface/001.html | popstate event should pass the state data - assert_true: state information does not contain the expected value - browser is probably stuck in the wrong history position expected true got false
• html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-fetch-error-external-classic.html
  • html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-fetch-error-external-module.html
  • html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-parse-error-external-classic.html
  • html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-parse-error-external-module.html
  • html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-success-external-classic.html
  • html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-success-external-module.html
  • TEST-UNEXPECTED-FAIL | /html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-parse-error-external-module.html | <script> error: Move parse-error external module script to iframe before-prepare - assert_unreached: Script error evennt fired unexpectedly Reached unreachable code
  • All these "before-prepare-iframe-....html" tests seem to have the same failures that look like existing bug 1636883 or bug 1714790. But why do these tests all fail in the same before-prepare-iframe-parse-error-external-module.html file? We probably only need to file one bug that covers all these "before-prepare-iframe-....html" failures, even though they are in different test files.
• old-tests/submission/Microsoft/history/history_000.htm
  • TEST-UNEXPECTED-FAIL | /old-tests/submission/Microsoft/history/history_000.htm | history.replaceState does not clear forward entries - assert_equals: Three additional travel entries add to history.length expected 11 but got 10
  • These failures all look like bug 1715344, which was just three days ago. So I don't think we need to skip this history_000.htm test.

Comment 10

[Andrew Halberstadt [:ahal]]

Thanks, I'm working on this one next (have some PTO coming up though).

Comment 11

[Andrew Halberstadt [:ahal]]

https://treeherder.mozilla.org/jobs?repo=try&revision=2eb2b0d526fa0ab3220f13267ca6357c8bba5aad

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/df6201bac1c8
[ci] web-platform-test: Run fission linux64-tsan on trunk, r=jgraham
https://hg.mozilla.org/integration/autoland/rev/2b25f5c0015f
[ci] web-platform-test-reftest: Run fission linux64-tsan/opt on trunk, r=jgraham

Comment 13

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/df6201bac1c8
https://hg.mozilla.org/mozilla-central/rev/2b25f5c0015f