Closed Bug 1628050 Opened 4 months ago Closed 4 months ago

Perma raptor-main Critical: TEST-UNEXPECTED-FAIL: test 'raptor-wasm-godot-cranelift-firefox' timed out loading test page: http://localhost:60685/wasm-godot/index.html

Categories

(Testing :: Raptor, defect, P5)

Version 3
defect

Tracking

(firefox76 unaffected, firefox77 fixed)

RESOLVED FIXED
mozilla77
Tracking Status
firefox76 --- unaffected
firefox77 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: bbouvier)

References

(Regression)

Details

(Keywords: intermittent-failure)

Attachments

(1 file)

Filed by: apavel [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=296650855&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/BXXEGGFdRjCSweyhYG3aYg/runs/0/artifacts/public/logs/live_backing.log


[task 2020-04-07T17:53:55.733Z] 17:53:55 INFO - PID 3647 | console.info: "[raptor-runnerjs] results pending..."
[task 2020-04-07T17:53:55.991Z] 17:53:55 INFO - PID 3647 | console.info: "[raptor-runnerjs] results pending..."
[task 2020-04-07T17:53:56.234Z] 17:53:56 INFO - PID 3647 | console.info: "[raptor-runnerjs] results pending..."
[task 2020-04-07T17:53:56.250Z] 17:53:56 INFO - PID 3647 | console.error: "ERROR: [raptor-runnerjs] raptor-page-timeout on http://localhost:60685/wasm-godot/index.html"
[task 2020-04-07T17:53:56.252Z] 17:53:56 INFO - raptor-control-server Info: received webext_raptor-page-timeout: [u'raptor-wasm-godot-cranelift-firefox', u'http://localhost:60685/wasm-godot/index.html']
[task 2020-04-07T17:53:56.259Z] 17:53:56 INFO - PID 3647 | console.info: "[raptor-runnerjs] capturing screenshot"
[task 2020-04-07T17:53:56.307Z] 17:53:56 INFO - raptor-control-server Info: received webext_screenshot
[task 2020-04-07T17:53:56.308Z] 17:53:56 INFO - perftest-results-handler Info: received screenshot
[task 2020-04-07T17:53:56.311Z] 17:53:56 INFO - raptor-control-server Info: received webext_status: found active tab with id 3
[task 2020-04-07T17:53:56.315Z] 17:53:56 INFO - raptor-control-server Info: received webext_status: closing Tab: 3
[task 2020-04-07T17:53:56.336Z] 17:53:56 INFO - raptor-control-server Info: received webext_status: closed tab: 3
[task 2020-04-07T17:53:56.340Z] 17:53:56 INFO - PID 3647 | console.info: "[raptor-runnerjs] benchmark test finished"
[task 2020-04-07T17:53:56.340Z] 17:53:56 INFO - raptor-control-server Info: received request to shutdown the browser
[task 2020-04-07T17:53:56.340Z] 17:53:56 INFO - raptor-control-server Info: shutting down browser (pid: 3647)
[task 2020-04-07T17:53:56.485Z] 17:53:56 INFO - PID 3647 | console.info: "[raptor-runnerjs] results pending..."
[task 2020-04-07T17:53:56.736Z] 17:53:56 INFO - PID 3647 | console.info: "[raptor-runnerjs] results pending..."
[task 2020-04-07T17:53:56.986Z] 17:53:56 INFO - PID 3647 | console.info: "[raptor-runnerjs] results pending..."
[task 2020-04-07T17:53:57.236Z] 17:53:57 INFO - PID 3647 | console.info: "[raptor-runnerjs] results pending..."
[task 2020-04-07T17:53:58.231Z] 17:53:58 INFO - raptor-webext Info: removing webext /home/cltbld/tasks/task_1586278338/build/tests/raptor/raptor/webextension/../../webext/raptor
[task 2020-04-07T17:53:58.232Z] 17:53:58 INFO - perftest-results-handler Info: summarizing raptor test results
[task 2020-04-07T17:53:58.232Z] 17:53:58 INFO - perftest-output Error: no raptor test results found for raptor-wasm-godot-cranelift-firefox
[task 2020-04-07T17:53:58.233Z] 17:53:58 INFO - perftest-output Info: error: no raptor test results found, so no need to combine browser cycles
[task 2020-04-07T17:53:58.233Z] 17:53:58 INFO - perftest-output Error: no summarized raptor results found for raptor-wasm-godot-cranelift-firefox
[task 2020-04-07T17:53:58.233Z] 17:53:58 INFO - perftest-output Info: screen captures can be found locally at: /home/cltbld/tasks/task_1586278338/build/screenshots.html
[task 2020-04-07T17:53:58.249Z] 17:53:58 INFO - perftest-results-handler Critical: PERFHERDER_DATA was seen 0 times, expected 1.
[task 2020-04-07T17:53:58.249Z] 17:53:58 INFO - raptor-control-server Info: shutting down control server
[task 2020-04-07T17:53:58.345Z] 17:53:58 INFO - raptor-webext Info: finished
[task 2020-04-07T17:53:58.345Z] 17:53:58 ERROR - raptor-main Critical: TEST-UNEXPECTED-FAIL: test 'raptor-wasm-godot-cranelift-firefox' timed out loading test page: http://localhost:60685/wasm-godot/index.html
[task 2020-04-07T17:53:58.389Z] 17:53:58 ERROR - Return code: 1
[task 2020-04-07T17:53:58.389Z] 17:53:58 WARNING - setting return code to 1
[task 2020-04-07T17:53:58.389Z] 17:53:58 INFO - Copying Raptor results to upload dir:
[task 2020-04-07T17:53:58.389Z] 17:53:58 INFO - /home/cltbld/tasks/task_1586278338/build/blobber_upload_dir/perfherder-data.json
[task 2020-04-07T17:53:58.390Z] 17:53:58 INFO - Copying raptor results from /home/cltbld/tasks/task_1586278338/build/raptor.json to /home/cltbld/tasks/task_1586278338/build/blobber_upload_dir/perfherder-data.json
[task 2020-04-07T17:53:58.390Z] 17:53:58 CRITICAL - Error copying results /home/cltbld/tasks/task_1586278338/build/raptor.json to upload dir /home/cltbld/tasks/task_1586278338/build/blobber_upload_dir/perfherder-data.json
[task 2020-04-07T17:53:58.390Z] 17:53:58 INFO - [Errno 2] No such file or directory: u'/home/cltbld/tasks/task_1586278338/build/raptor.json'
[task 2020-04-07T17:53:58.391Z] 17:53:58 INFO - /home/cltbld/tasks/task_1586278338/build/blobber_upload_dir/screenshots.html
[task 2020-04-07T17:53:58.391Z] 17:53:58 INFO - Copying raptor results from /home/cltbld/tasks/task_1586278338/build/screenshots.html to /home/cltbld/tasks/task_1586278338/build/blobber_upload_dir/screenshots.html
[task 2020-04-07T17:53:58.391Z] 17:53:58 INFO - Running post-action listener: _package_coverage_data
[task 2020-04-07T17:53:58.391Z] 17:53:58 INFO - Running post-action listener: _resource_record_post_action
[task 2020-04-07T17:53:58.391Z] 17:53:58 INFO - Running post-action listener: process_java_coverage_data
[task 2020-04-07T17:53:58.391Z] 17:53:58 INFO - Running post-action listener: stop_device
[task 2020-04-07T17:53:58.392Z] 17:53:58 INFO - [mozharness: 2020-04-07 17:53:58.391950Z] Finished run-tests step (success)
[task 2020-04-07T17:53:58.392Z] 17:53:58 INFO - Running post-run listener: _resource_record_post_run
[task 2020-04-07T17:53:58.437Z] 17:53:58 INFO - Total resource usage - Wall time: 134s; CPU: 2.0%; Read bytes: 26628096; Write bytes: 546603008; Read time: 156; Write time: 54608
[task 2020-04-07T17:53:58.437Z] 17:53:58 INFO - TinderboxPrint: CPU usage<br/>1.8%
[task 2020-04-07T17:53:58.437Z] 17:53:58 INFO - TinderboxPrint: I/O read bytes / time<br/>26,628,096 / 156
[task 2020-04-07T17:53:58.437Z] 17:53:58 INFO - TinderboxPrint: I/O write bytes / time<br/>546,603,008 / 54,608
[task 2020-04-07T17:53:58.437Z] 17:53:58 INFO - TinderboxPrint: CPU idle<br/>1,048.6 (98.1%)

This is a new perma failure since yesterday and specifically for this merge to mozilla-central:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=godot-c&revision=6c6c67870c37b2e414f898e390ed5247d8e1e94a

I'm fairly sure that this is a regression from bug 1626967. Benjamin, can you please have a look?

Flags: needinfo?(bbouvier)

I actually wonder why this affecting patch hasn't been backed-out yet given that it affects a Tier1 job.

Flags: needinfo?(aryx.bugmail)

Why is that task Tier-1 if it only runs on central? That's against the Tier Policy.

Flags: needinfo?(aryx.bugmail)

(In reply to Sebastian Hengst [:aryx] (needinfo on intermittent or backout) from comment #4)

Why is that task Tier-1 if it only runs on central? That's against the Tier Policy.

Thanks! Florin and Greg, can you please get this fixed. Whether we have to also run it on autoland or de-promote it to Tier2.

Flags: needinfo?(gmierz2)
Flags: needinfo?(fstrugariu)

Yep, clearly a regression from bug 1626967; investigating.

Regressed by: 1626967

Benjamin explained in Matrix the jobs should be tier 2.

Yep, clearly a regression from bug 1626967; investigating.

Not so clearly; i now suspect it's bug 1625927, since:

  1. i couldn't reproduce the failures locally when running the benchmarks outside the infra
  2. the above bug enabled multi-value by default, thus disabling Cranelift (which doesn't implement multi value).

Fortunately, this means the patch for this is trivial; working on it.

Flags: needinfo?(bbouvier)
Regressed by: 1625927
No longer regressed by: 1626967
Assignee: nobody → bbouvier
Status: NEW → ASSIGNED

(In reply to Sebastian Hengst [:aryx] (needinfo on intermittent or backout) from comment #7)

Benjamin explained in Matrix the jobs should be tier 2.

Florin, can someone please do the move to Tier2?

Pushed by bbouvier@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a85bb2c6b781
Disable wasm multi-value when benchmarking Cranelift; r=perftest-reviewers,Bebe
Flags: needinfo?(gmierz2)
Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla77

I think this is fixed

Flags: needinfo?(fstrugariu)

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #5)

Thanks! Florin and Greg, can you please get this fixed. Whether we have to also run it on autoland or de-promote it to Tier2.

Florin, the request was set in regards of moving the job to tier2. Can you please re-check? Thanks.

Flags: needinfo?(fstrugariu)

Bug 1629502 - Move raptor-wasm-godot-cranelift-firefox to tire 2

Flags: needinfo?(fstrugariu)
You need to log in before you can comment on or make changes to this bug.