1698528 - TV-Fis should not run mochitest-chrome tests in Fission mode ("Intermittent Fission is not supported without e10s.")

Status: RESOLVED DUPLICATE of bug 1698849

(Testing :: General, defect, P3)

Description

[Treeherder Bug Filer]

Filed by: smolnar [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=333247209&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/E6Zr_A9jQH6x9wK4GOMCmg/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/E6Zr_A9jQH6x9wK4GOMCmg/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

 INFO - Calling ['/builds/worker/workspace/build/venv/bin/python', '-u', '/builds/worker/workspace/build/tests/mochitest/runtests.py', '--enable-webrender', '--setpref=fission.autostart=true', '--setpref=dom.serviceWorkers.parent_intercept=true', '--setpref=media.peerconnection.mtransport_process=false', '--setpref=network.process.enabled=false', '--setpref=layers.d3d11.enable-blacklist=false', '--appname=/builds/worker/workspace/build/application/firefox/firefox', '--utility-path=tests/bin', '--extra-profile-file=tests/bin/plugins', u'--symbols-path=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/K0IwytWvSlilghswGFZrCQ/artifacts/public/build/target.crashreporter-symbols.zip', '--certificate-path=tests/certs', '--setpref=webgl.force-enabled=true', '--quiet', '--log-raw=/builds/worker/workspace/build/blobber_upload_dir/mochitest-chrome-test1_raw.log', '--log-errorsummary=/builds/worker/workspace/build/blobber_upload_dir/mochitest-chrome-test1_errorsummary.log', '--use-test-media-devices', '--screenshot-on-fail', '--cleanup-crashes', '--marionette-startup-timeout=180', '--sandbox-read-whitelist=/builds/worker/workspace/build', '--log-raw=-', '--flavor=chrome', '--chunk-by-dir=4', '--disable-e10s', '--verify-max-time=900', '--verify', u'tools/profiler/tests/chrome/test_profile_worker.html'] with output_timeout 1000
[task 2021-03-15T12:31:13.911Z] 12:31:13     INFO - :::
[task 2021-03-15T12:31:13.911Z] 12:31:13     INFO - ::: Running test verification step "1. Run each test 10 times in one browser."...
[task 2021-03-15T12:31:13.911Z] 12:31:13     INFO - :::
[task 2021-03-15T12:31:13.911Z] 12:31:13     INFO - Checking for ssltunnel processes...
[task 2021-03-15T12:31:13.918Z] 12:31:13     INFO - Checking for xpcshell processes...
[task 2021-03-15T12:31:13.981Z] 12:31:13     INFO - SUITE-START | Running 1 tests
[task 2021-03-15T12:31:13.982Z] 12:31:13    ERROR - Fission is not supported without e10s.
[task 2021-03-15T12:31:13.983Z] 12:31:13     INFO - Buffered messages finished
[task 2021-03-15T12:31:13.984Z] 12:31:13     INFO - SUITE-END | took 0s
[task 2021-03-15T12:31:13.984Z] 12:31:13     INFO - SUITE-START | Running 1 tests
[task 2021-03-15T12:31:13.985Z] 12:31:13     INFO - :::
[task 2021-03-15T12:31:13.986Z] 12:31:13     INFO - ::: Test verification summary for:
[task 2021-03-15T12:31:13.986Z] 12:31:13     INFO - :::
[task 2021-03-15T12:31:13.987Z] 12:31:13     INFO - ::: tools/profiler/tests/chrome/test_profile_worker.html
[task 2021-03-15T12:31:13.988Z] 12:31:13     INFO - :::
[task 2021-03-15T12:31:13.989Z] 12:31:13     INFO - ::: 1. Run each test 10 times in one browser. : FAIL
[task 2021-03-15T12:31:13.989Z] 12:31:13     INFO - ::: 2. Run each test 5 times in a new browser each time. : not run / incomplete
[task 2021-03-15T12:31:13.990Z] 12:31:13     INFO - ::: 3. Run each test 10 times in one browser, in chaos mode. : not run / incomplete
[task 2021-03-15T12:31:13.990Z] 12:31:13     INFO - ::: 4. Run each test 5 times in a new browser each time, in chaos mode. : not run / incomplete
[task 2021-03-15T12:31:13.991Z] 12:31:13     INFO - :::
[task 2021-03-15T12:31:13.992Z] 12:31:13     INFO - ::: Test verification FAILED!
[task 2021-03-15T12:31:13.992Z] 12:31:13     INFO - :::
[task 2021-03-15T12:31:13.993Z] 12:31:13     INFO - Buffered messages finished
[task 2021-03-15T12:31:13.993Z] 12:31:13     INFO - SUITE-END | took 0s
[task 2021-03-15T12:31:14.062Z] 12:31:14     INFO - Return code: 0
[task 2021-03-15T12:31:14.064Z] 12:31:14    ERROR - No checks run.
[task 2021-03-15T12:31:14.064Z] 12:31:14     INFO - TinderboxPrint: mochitest-mochitest-chrome<br/><em class="testfail">T-FAIL</em>
[task 2021-03-15T12:31:14.064Z] 12:31:14    ERROR - # TBPL FAILURE #
[task 2021-03-15T12:31:14.064Z] 12:31:14  WARNING - setting return code to 2
[task 2021-03-15T12:31:14.065Z] 12:31:14    ERROR - TinderboxPrint: Per-test run of .../chrome/test_profile_worker.html<br/>: FAILURE
[task 2021-03-15T12:31:14.065Z] 12:31:14     INFO - Per-test suites: {}
[task 2021-03-15T12:31:14.065Z] 12:31:14     INFO - Per-test suites: {}
INFO - Running post-action listener: _package_coverage_data
[task 2021-03-15T12:31:14.066Z] 12:31:14     INFO - Running post-action listener: _resource_record_post_action
[task 2021-03-15T12:31:14.066Z] 12:31:14     INFO - Running post-action listener: process_java_coverage_data
[task 2021-03-15T12:31:14.067Z] 12:31:14     INFO - [mozharness: 2021-03-15 12:31:14.066884Z] Finished run-tests step (success)
[task 2021-03-15T12:31:14.067Z] 12:31:14     INFO - Running post-run listener: _resource_record_post_run
[task 2021-03-15T12:31:14.128Z] 12:31:14     INFO - Validating Perfherder data against /builds/worker/workspace/mozharness/external_tools/performance-artifact-schema.json
[task 2021-03-15T12:31:14.132Z] 12:31:14     INFO - PERFHERDER_DATA: {"framework": {"name": "job_resource_usage"}, "suites": [{"subtests": [{"name": "cpu_percent", "value": 50.69166666666667}, {"name": "io_write_bytes", "value": 6004736}, {"name": "io.read_bytes", "value": 16384}, {"name": "io_write_time", "value": 1668}, {"name": "io_read_time", "value": 0}], "extraOptions": ["e10s", "taskcluster-m5.large"], "name": ".overall"}, {"subtests": [{"name": "time", "value": 0.055082082748413086}], "name": ".start-pulseaudio"}, {"subtests": [{"name": "time", "value": 11.92185091972351}, {"name": "cpu_percent", "value": 50.4}], "name": ".install"}, {"subtests": [{"name": "time", "value": 0.0022301673889160156}], "name": ".stage-files"}, {"subtests": [{"name": "time", "value": 0.4993619918823242}], "name": ".run-tests"}]}
[task 2021-03-15T12:31:14.133Z] 12:31:14     INFO - Total resource usage - Wall time: 12s; CPU: Can't collect data; Read bytes: 16384; Write bytes: 6004736; Read time: 0; Write time: 1668
[task 2021-03-15T12:31:14.134Z] 12:31:14     INFO - TinderboxPrint: I/O read bytes / time<br/>16,384 / 0
[task 2021-03-15T12:31:14.134Z] 12:31:14     INFO - TinderboxPrint: I/O write bytes / time<br/>6,004,736 / 1,668
[task 2021-03-15T12:31:14.135Z] 12:31:14     INFO - TinderboxPrint: CPU idle<br/>11.8 (49.1%)
[task 2021-03-15T12:31:14.135Z] 12:31:14     INFO - TinderboxPrint: CPU system<br/>0.6 (2.3%)
[task 2021-03-15T12:31:14.135Z] 12:31:14     INFO - TinderboxPrint: CPU user<br/>11.6 (48.4%)
[task 2021-03-15T12:31:14.135Z] 12:31:14     INFO - TinderboxPrint: Swap in / out<br/>0 / 0
[task 2021-03-15T12:31:14.135Z] 12:31:14     INFO - start-pulseaudio - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2021-03-15T12:31:14.135Z] 12:31:14     INFO - install - Wall time: 12s; CPU: 50.0%; Read bytes: 0; Write bytes: 2465792; Read time: 0; Write time: 948
[task 2021-03-15T12:31:14.135Z] 12:31:14     INFO - stage-files - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2021-03-15T12:31:14.135Z] 12:31:14     INFO - run-tests - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2021-03-15T12:31:14.138Z] 12:31:14  WARNING - returning nonzero exit status 2```

Comment 1

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

It's strange that it only happened when auto-landing bug 1687631 (enabling "cpu" feature by default). I'm not sure why "cpu" would have this effect.
It happened when running tools/profiler/tests/chrome/test_profile_worker.html, but I don't see this test in other TV-fis tasks (are they picked semi-randomly?) I'm guessing modifying test_profiler_worker.html triggered its testing.

In the log, there's

Per-test suites: {'mochitest-chrome': ('--flavor=chrome', '--chunk-by-dir=4', '--disable-e10s')}

I'm not sure where the --disable-e10s comes from.

To be monitored...

Comment 2

[Intermittent Failures Robot]

6 failures in 4257 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 6

Platform and build breakdown:

• macosx1014-64-qr: 2
  • opt: 1
  • debug: 1
• windows10-64-qr: 2
  • debug: 1
  • opt: 1
• linux1804-64-qr: 2
  • opt: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1698528&startday=2021-03-15&endday=2021-03-21&tree=all

Comment 3

[Intermittent Failures Robot]

1 failures in 3854 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows7-32: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1698528&startday=2021-04-05&endday=2021-04-11&tree=all

Comment 4

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Very low frequency, and I think it's more a test configuration issue that something in the profiler itself.
So I won't monitor actively anymore, but moving priority to P3 so the team may go back to it eventually...

Comment 5

[Chris Peterson [:cpeterson]]

Andrew, I have learned that TV-Fis should not run mochitest-chrome tests in Fission mode. This is causing mochitest-chrome test failures because TV knows to --disable-e10s, but is trying to enable Fission. Since mochitest-chrome tests are already run without Fission, we don't need to include them in TV-Fis.

From the test log:

[task 2021-03-15T12:31:13.625Z] 12:31:13 INFO - Copy/paste: /builds/worker/workspace/build/venv/bin/python -u /builds/worker/workspace/build/tests/mochitest/runtests.py --enable-webrender --setpref=fission.autostart=true

Per-test suites: {'mochitest-chrome': ('--flavor=chrome', '--chunk-by-dir=4', '--disable-e10s')}

Comment 6

[Andrew Halberstadt [:ahal]]

This should already be fixed as of bug 1698849. Looks like that landed shortly after this was filed. Please re-open if it's still an issue.