1675020 - Disable frequent intermittents / permafail DevTools tests on Fission platforms

Status: RESOLVED FIXED

(DevTools :: General, task, P3)

Description

[Julian Descottes [:jdescottes]]

We should move DevTools tests on Fission platforms to Tier 1 as soon as possible.
Let's review the current failures we have and skip tests which fail too frequently or are permafails.

Comment 1

[Julian Descottes [:jdescottes]]

Try push with Fission enabled for DevTools tests:
https://treeherder.mozilla.org/jobs?repo=try&selectedTaskRun=P8z5CdYQTNaoC8R6-K7XzA.0&revision=87a52a8bd7cc5e3008ce37549e919d326d150320

frequent failures

• devtools/client/debugger/test/mochitest/browser_dbg-inline-preview.js
  existing intermittent, frequent (https://bugzilla.mozilla.org/show_bug.cgi?id=1607636)
• devtools/client/debugger/test/mochitest/browser_dbg-toolbox-workers.js
  existing intermittent, frequent (https://bugzilla.mozilla.org/show_bug.cgi?id=1595553)
• devtools/client/dom/test/browser_dom_fission_target_switching.js
  existing intermittent, rare (https://bugzilla.mozilla.org/show_bug.cgi?id=1615737)
• devtools/client/jsonview/test/browser_jsonview_save_json.js
  existing intermittent, frequent (https://bugzilla.mozilla.org/show_bug.cgi?id=1650268)
• devtools/shared/resources/tests/browser_resources_stylesheets.js
  existing intermittent, frequent (https://bugzilla.mozilla.org/show_bug.cgi?id=1665073)
• devtools/client/framework/test/browser_source_map-reload.js
  existing intermittent, frequent (https://bugzilla.mozilla.org/show_bug.cgi?id=1668677)
• devtools/client/framework/test/browser_toolbox_options_enable_serviceworkers_testing.js
  existing intermittent, frequent (https://bugzilla.mozilla.org/show_bug.cgi?id=1669650)

single failure

• devtools/client/netmonitor/test/browser_net_worker_stacks.js
  existing intermittent, frequent (https://bugzilla.mozilla.org/show_bug.cgi?id=1552618)
• devtools/client/application/test/browser/browser_application_panel_telemetry-debug-worker.js
  existing intermittent, rare (https://bugzilla.mozilla.org/show_bug.cgi?id=1654354)
• devtools/client/inspector/markup/test/browser_markup_events_react_development_15.4.1_jsx.js
  existing intermittent, rare fission only (?) (https://bugzilla.mozilla.org/show_bug.cgi?id=1672062)
• devtools/client/shared/test/browser_html_tooltip_arrow-02.js
  existing intermittent, rare (https://bugzilla.mozilla.org/show_bug.cgi?id=1619292)

other frequent failures

The following tests frequently failed, but only after other debugger/test already failed in the suite, wait for more results before taking a decision for them.

• devtools/client/debugger/test/mochitest/browser_dbg-watchpoints.js
• devtools/client/debugger/test/mochitest/browser_dbg-windowless-service-workers.js
• devtools/client/debugger/test/mochitest/browser_dbg-xhr-run-to-completion.js
• devtools/client/webconsole/test/browser/browser_toolbox_console_new_process.js
• devtools/client/webconsole/test/browser/browser_console_clear_cache.js

I will try to skip if all the tests mentioned in the first section for now.

Comment 2

[Julian Descottes [:jdescottes]]

Attachment: Bug 1675020 - [devtools] Disable frequent intermittents / permafail DevTools tests on Fission platforms

Temporarily add skip-if=fission in order to move devtools tests on Fission platforms to Tier1

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f7b612f7840e
[devtools] Disable frequent intermittents / permafail DevTools tests on Fission platforms r=Honza

Comment 4

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/f7b612f7840e