1379348 - sitespeed.io job on Jenkins prod (fx-test-jenkins.stage.*) stalls/hangs, opening AMO staging URL

Status: RESOLVED INVALID

(Cloud Services :: FXTest-infra, defect)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

The original broken job/run was https://fx-test-jenkins.stage.mozaws.net/job/sitespeed.io/25/

https://fx-test-jenkins.stage.mozaws.net/job/sitespeed.io/ stalls/hangs/times out[0], trying to open https://addons.allizom.org/en-US/firefox/ 

So far, even after adding --debug and -vvv (triple verbosity, srsly) to the sitespeed.io commandline-run params, all we can currently see is:

docker run --privileged -v /home/jenkins/slavehome/workspace/sitespeed.io:/sitespeed.io sitespeedio/sitespeed.io --outputFolder output https://addons.allizom.org/en-US/firefox/ -b firefox --debug -vvv

<snip>

Mozilla Firefox 54.0
[2017-07-08 04:27:02] INFO: [sitespeedio] Versions OS: linux 3.10.0-327.36.2.el7.x86_64 nodejs: v6.10.3 sitespeed.io: 5.4.1 browsertime: 1.5.3 coach: 0.34.1
[2017-07-08 04:27:03] INFO: [sitespeedio.plugin.messagelogger] {
  "uuid": "85e23c12-d6a8-40b9-aab4-80f5dc4d2fce",
  "type": "url",
  "timestamp": "2017-07-08T04:27:03+00:00",
  "source": "url-reader",
  "data": {},
  "url": "https://addons.allizom.org/en-US/firefox/",
  "group": "addons.allizom.org"
}
[2017-07-08 04:27:03] INFO: [sitespeedio.plugin.browsertime] Starting firefox for analysing https://addons.allizom.org/en-US/firefox/ 3 time(s)
[2017-07-08 04:27:03] INFO: [root] Testing url https://addons.allizom.org/en-US/firefox/ run 1
[2017-07-08 04:27:06] DEBUG: [root] Requesting url https://addons.allizom.org/en-US/firefox/
[2017-07-08 04:27:08] DEBUG: [root] Waiting for script 'return (function() {try { var end = window.performance.timing.loadEventEnd;return (end > 0) && (Date.now() > end + 2000);} catch(e) {return true;}})()' at most 300000 ms
[2017-07-08 04:32:08] ERROR: [root] Could not load URLTimeoutError: Running page complete check 'return (function() {try { var end = window.performance.timing.loadEventEnd;return (end > 0) && (Date.now() > end + 2000);} catch(e) {return true;}})()' took too long.
[2017-07-08 04:32:08] DEBUG: [root] Telling browser to quit.

This same job works fine on https://fx-test-jenkins-dev.stage.mozaws.net:8443/job/sitespeed.io, and typically takes around 1 min 43 seconds to run: https://fx-test-jenkins-dev.stage.mozaws.net:8443/job/sitespeed.io/51/

[0] https://fx-test-jenkins.stage.mozaws.net/job/sitespeed.io/32/console
[1] https://fx-test-jenkins.stage.mozaws.net/job/sitespeed.io/25/

Comment 1

[Stephen Donner [:stephend] Not actively reading bugmail]

(Just moving to the newly-created, fx-test-engineering-specific component.)

Comment 2

[Stephen Donner [:stephend] Not actively reading bugmail]

(I'm still looking through the changelog, here: https://github.com/sitespeedio/sitespeed.io/blob/master/CHANGELOG.md as well as open Issues in their GitHub repo, for pointers, just FYI.)

Comment 3

[Stephen Donner [:stephend] Not actively reading bugmail]

I've filed this now as https://github.com/sitespeedio/sitespeed.io/issues/1680 and will close out this bug when (very likely) it's proven to be a bug on their side.

Comment 4

[Stephen Donner [:stephend] Not actively reading bugmail]

This is INVALID/FIXED - the tl;dr is there was a missing --shm-size=1g call needed (absent from the particular doc I used).

See https://fx-test-jenkins-dev.stage.mozaws.net:8443/job/sitespeed.io/60/console for the "fixed" run and moar details are in the GitHub Issue, above.