1023935 - Use 'runner-startup' instead of 'automation' for self.last_test in mozrunner's runner.py

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[Ed Morley [:emorley]]

So that failures on TBPL can be more easily starred, it would help is these type of failures (ie timeouts on startup):
https://tbpl.mozilla.org/php/getParsedLog.php?id=41513905&tree=B2g-Inbound
{
04:09:27     INFO -  Mochitest INFO | runtestsb2g.py | Running tests: start.
04:09:27     INFO -  RemoteRunner INFO | using /data/local/tests as test_root
04:09:40     INFO -  1402484980852	Marionette	INFO	marionette enabled via build flag and pref
04:09:43     INFO -  1402484983344	Marionette	INFO	marionette-server.js loaded
04:09:44     INFO -  1402484984520	Marionette	INFO	B2G emulator: yes
04:09:44     INFO -  1402484984531	Marionette	INFO	Device detected is generic
04:09:44     INFO -  1402484984536	Marionette	INFO	Bypassing offline status.
04:09:44     INFO -  1402484984556	Marionette	INFO	Listening on port 2828
04:09:44     INFO -  1402484984566	Marionette	INFO	Marionette server ready
04:09:44     INFO -  System JS : WARNING resource://gre/modules/Preferences.jsm:378 - mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create
04:09:47     INFO -  1402484987351	Marionette	DEBUG	accepted connection on 127.0.0.1:34011
04:09:58     INFO -  1402484997966	Marionette	DEBUG	accepted connection on 127.0.0.1:34012
04:10:41     INFO -  1402485041682	Marionette	INFO	loaded marionette-listener.js
04:10:41     INFO -  1402485041734	Marionette	INFO	sendToClient: {"from":"0","value":{"browserName":"B2G","browserVersion":"33.0a1","platformName":"ANDROID","platformVersion":"33.0a1","handlesAlerts":false,"nativeEvents":false,"rotatable":true,"secureSsl":false,"takesElementScreenshot":true,"takesScreenshot":true,"platform":"ANDROID","XULappId":"{3c2e2abc-06d4-11e1-ac3b-374f68613e61}","appBuildId":"20140611023009","device":"qemu","version":"33.0a1","b2g":true}}, {62c5e716-2ec2-4bca-b6ea-79d0230378f6}, {62c5e716-2ec2-4bca-b6ea-79d0230378f6}
04:10:42     INFO -  ###################################### forms.js loaded
04:10:42     INFO -  ############################### browserElementPanning.js loaded
04:10:42     INFO -  ######################## BrowserElementChildPreload.js loaded
04:10:42     INFO -  *** UTM:SVC TimerManager:registerTimer - id: user-agent-updates-timer
04:10:44     INFO -  1402485044228	Marionette	DEBUG	Got request: setContext, data: {"to":"0","sessionId":{"rotatable":true,"browserVersion":"33.0a1","takesScreenshot":true,"appBuildId":"20140611023009","XULappId":"{3c2e2abc-06d4-11e1-ac3b-374f68613e61}","secureSsl":false,"platform":"ANDROID","browserName":"B2G","version":"33.0a1","device":"qemu","b2g":true,"nativeEvents":false,"platformVersion":"33.0a1","takesElementScreenshot":true,"platformName":"ANDROID","handlesAlerts":false},"name":"setContext","parameters":{"value":"chrome"}}, id: {be858541-ec25-4a8d-9499-64f6d5501ff2}
04:10:44     INFO -  1402485044258	Marionette	INFO	sendToClient: {"from":"0","ok":true}, {be858541-ec25-4a8d-9499-64f6d5501ff2}, {be858541-ec25-4a8d-9499-64f6d5501ff2}
04:10:45     INFO -  1402485045380	Marionette	DEBUG	Got request: execute, data: {"to":"0","sessionId":{"rotatable":true,"browserVersion":"33.0a1","takesScreenshot":true,"appBuildId":"20140611023009","XULappId":"{3c2e2abc-06d4-11e1-ac3b-374f68613e61}","secureSsl":false,"platform":"ANDROID","browserName":"B2G","version":"33.0a1","device":"qemu","b2g":true,"nativeEvents":false,"platformVersion":"33.0a1","takesElementScreenshot":true,"platformName":"ANDROID","handlesAlerts":false},"name":"executeScript","parameters":{"scriptTimeout":null,"specialPowers":false,"script":"\n                Components.utils.import(\"resource://gre/modules/Services.jsm\");\n                Services.io.manageOfflineStatus = false;\n                Services.io.offline = false;\n                ","newSandbox":true,"args":[],"filename":"remote.py","line":222}}, id: {ed65871b-589d-41db-aa7a-b695aa82cf30}
04:10:45     INFO -  1402485045546	Marionette	INFO	sendToClient: {"from":"0","value":null}, {ed65871b-589d-41db-aa7a-b695aa82cf30}, {ed65871b-589d-41db-aa7a-b695aa82cf30}
04:10:46     INFO -  1402485045863	Marionette	DEBUG	Got request: setContext, data: {"to":"0","sessionId":{"rotatable":true,"browserVersion":"33.0a1","takesScreenshot":true,"appBuildId":"20140611023009","XULappId":"{3c2e2abc-06d4-11e1-ac3b-374f68613e61}","secureSsl":false,"platform":"ANDROID","browserName":"B2G","version":"33.0a1","device":"qemu","b2g":true,"nativeEvents":false,"platformVersion":"33.0a1","takesElementScreenshot":true,"platformName":"ANDROID","handlesAlerts":false},"name":"setContext","parameters":{"value":"chrome"}}, id: {b591b028-2bfd-49fa-b852-1fe7c6fc1d06}
04:10:46     INFO -  1402485045892	Marionette	INFO	sendToClient: {"from":"0","ok":true}, {b591b028-2bfd-49fa-b852-1fe7c6fc1d06}, {b591b028-2bfd-49fa-b852-1fe7c6fc1d06}
04:10:46     INFO -  1402485046196	Marionette	DEBUG	Got request: execute, data: {"to":"0","sessionId":{"rotatable":true,"browserVersion":"33.0a1","takesScreenshot":true,"appBuildId":"20140611023009","XULappId":"{3c2e2abc-06d4-11e1-ac3b-374f68613e61}","secureSsl":false,"platform":"ANDROID","browserName":"B2G","version":"33.0a1","device":"qemu","b2g":true,"nativeEvents":false,"platformVersion":"33.0a1","takesElementScreenshot":true,"platformName":"ANDROID","handlesAlerts":false},"name":"executeScript","parameters":{"scriptTimeout":null,"specialPowers":false,"script":"/* This Source Code Form is subject to the terms of the Mozilla Public\n * License, v. 2.0. If a copy of the MPL was not distributed with this\n * file, You can obtain one at http://mozilla.org/MPL/2.0/. */\n\nlet outOfProcess = __marionetteParams[0]\nlet mochitestUrl = __marionetteParams[1]\nlet onDevice = __marionetteParams[2]\nlet wifiSettings = __marionetteParams[3]\nlet prefs = Components.classes[\"@mozilla.org/preferences-service;1\"].\n                            getService(Components.interfaces.nsIPrefBranch)\nlet settings = window.navigator.mozSettings;\n\nif (wifiSettings)\n  wifiSettings = JSON.parse(wifiSettings);\n\nconst CHILD_SCRIPT = \"chrome://specialpowers/content/specialpowers.js\";\nconst CHILD_SCRIPT_API = \"chrome://specialpowers/content/specialpowersAPI.js\";\nconst CHILD_LOGGER_SCRIPT = \"chrome://specialpowers/content/MozillaLogger.js\";\n\nlet homescreen = document.getElementById('systemapp');\nlet container = homescreen.contentWindow.document.getElementById('test-container');\n\nfunction openWindow(aEvent) {\n  var popupIframe = aEvent.detail.frameElement;\n  popupIframe.style = 'position: absolute; left: 0; top: 0px; background: white;';\n\n  // This is to size the iframe to what is requested in the window.open call,\n  // e.g. window.open(\"\", \"\", \"width=600,height=600\");\n  if (aEvent.detail.features.indexOf('width') != -1) {\n    let width = aEvent.detail.features.substr(aEvent.detail.features.indexOf('width')+6);\n    width = width.substr(0,width.indexOf(',') == -1 ? width.length : width.indexOf(','));\n    popupIframe.style.width = width + 'px';\n  }\n  if (aEvent.detail.features.indexOf('height') != -1) {\n    let height = aEvent.detail.features.substr(aEvent.detail.features.indexOf('height')+7);\n    height = height.substr(0, height.indexOf(',') == -1 ? height.length : height.indexOf(','));\n    popupIframe.style.height = height + 'px';\n  }\n\n  popupIframe.addEventListener('mozbrowserclose', function(e) {\n    container.parentNode.removeChild(popupIframe);\n    container.focus();\n  });\n\n  // yes, the popup can call window.open too!\n  popupIframe.addEventListener('mozbrowseropenwindow', openWindow);\n\n  popupIframe.addEventListener('mozbrowserloadstart', function(e) {\n    popupIframe.focus();\n    let mm = popupIframe.QueryInterface(Ci.nsIFrameLoaderOwner).frameLoader.messageManager;\n    mm.loadFrameScript(CHILD_LOGGER_SCRIPT, true);\n    mm.loadFrameScript(CHILD_SCRIPT_API, true);\n    mm.loadFrameScript(CHILD_SCRIPT, true);\n    mm.loadFrameScript('data:,attachSpecialPowersToWindow%28content%29%3B', true);\n  });\n\n  container.parentNode.appendChild(popupIframe);\n}\ncontainer.addEventListener('mozbrowseropenwindow', openWindow);\n\nif (outOfProcess) {\n  let specialpowers = {};\n  let loader = Cc[\"@mozilla.org/moz/jssubscript-loader;1\"].getService(Ci.mozIJSSubScriptLoader);\n  loader.loadSubScript(\"chrome://specialpowers/content/SpecialPowersObserver.js\", specialpowers);\n  let specialPowersObserver = new specialpowers.SpecialPowersObserver();\n\n  let mm = container.QueryInterface(Ci.nsIFrameLoaderOwner).frameLoader.messageManager;\n  specialPowersObserver.init(mm);\n\n  mm.addMessageListener(\"SPPrefService\", specialPowersObserver);\n  mm.addMessageListener(\"SPProcessCrashService\", specialPowersObserver);\n  mm.addMessageListener(\"SPPingService\", specialPowersObserver);\n  mm.addMessageListener(\"SpecialPowers.Quit\", specialPowersObserver);\n  mm.addMessageListener(\"SpecialPowers.Focus\", specialPowersObserver);\n  mm.addMessageListener(\"SPPermissionManager\", specialPowersObserver);\n  mm.addMessageListener(\"SPLoadChromeScript\", specialPowersObserver);\n  mm.addMessageListener(\"SPChromeScriptMessage\", specialPowersObserver);\n\n  mm.loadFrameScript(CHILD_LOGGER_SCRIPT, true);\n  mm.loadFrameScript(CHILD_SCRIPT_API, true);\n  mm.loadFrameScript(CHILD_SCRIPT, true);\n  //Workaround for bug 848411, once that bug is fixed, the following line can be removed\n  mm.loadFrameScript('data:,addEventListener%28%22DOMWindowCreated%22%2C%20function%28e%29%20%7B%0A%20%20removeEventListener%28%22DOMWindowCreated%22%2C%20arguments.callee%2C%20false%29%3B%0A%20%20var%20window%20%3D%20e.target.defaultView%3B%0A%20%20window.wrappedJSObject.SpecialPowers.addPermission%28%22allowXULXBL%22%2C%20true%2C%20window.document%29%3B%0A%7D%0A%29%3B', true);\n\n  specialPowersObserver._isFrameScriptLoaded = true;\n}\n\n\nif (onDevice) {\n  var cpuLock = Cc[\"@mozilla.org/power/powermanagerservice;1\"]\n                      .getService(Ci.nsIPowerManagerService)\n                      .newWakeLock(\"cpu\");\n\n  let manager = navigator.mozWifiManager;\n  let con = manager.connection;\n  manager.setPowerSavingMode(false);\n\n  manager.onenabled = function () {\n    if(wifiSettings) {\n      var req = manager.getKnownNetworks();\n      req.onsuccess = function () {\n        var networks = req.result;\n        for (var i = 0; i < networks.length; ++i){\n          var network = networks[i];\n          if(network.ssid == wifiSettings.ssid) {\n            manager.forget(network);\n          }\n        }\n        manager.associate(wifiSettings);\n      };\n    }\n  };\n\n  manager.onstatuschange = function (event) {\n    prefs.setIntPref(\"network.proxy.type\", 2);\n    if (event.status == 'connected') {\n      container.src = mochitestUrl;\n    }\n  };\n\n  if(wifiSettings) {\n    var req = settings.createLock().set({\n       'wifi.enabled': false,\n       'wifi.suspended': false\n     });\n\n    req.onsuccess = function () {\n      dump(\"----------------------enabling wifi------------------\\n\");\n      var req1 = settings.createLock().set({\n          'wifi.enabled': true,\n          'wifi.suspended': false});\n    };\n  }\n} else {\n  container.src = mochitestUrl;\n}\n","newSandbox":true,"args":[true,"http://mochi.test:8888/tests/?autorun=1&closeWhenDone=1&logFile=%2Fdata%2Flocal%2Ftests%2Flog%2Fmochitest.log&fileLevel=INFO&consoleLevel=INFO&totalChunks=9&thisChunk=7&hideResultsTable=1&dumpOutputDirectory=%2Ftmp&quiet=true",false,false],"filename":"remote.py","line":240}}, id: {05709d06-90d8-488e-afdf-1c95667f3348}
04:10:46     INFO -  1402485046456	Marionette	INFO	sendToClient: {"from":"0","value":null}, {05709d06-90d8-488e-afdf-1c95667f3348}, {05709d06-90d8-488e-afdf-1c95667f3348}
04:10:55     INFO -  ###################################### forms.js loaded
04:10:55     INFO -  ############################### browserElementPanning.js loaded
04:10:55     INFO -  ######################## BrowserElementChildPreload.js loaded
04:11:43     INFO -  *** UTM:SVC TimerManager:notify - notified @mozilla.org/b2g/webapps-update-timer;1
04:13:43     INFO -  *** UTM:SVC TimerManager:notify - notified timerID: user-agent-updates-timer
04:13:44     INFO -  *** UTM:SVC TimerManager:registerTimer - id: user-agent-updates-timer
04:19:14     INFO -  B2GRunner TEST-UNEXPECTED-FAIL | automation | application timed out after 330.0 seconds with no output
}

Actually output something unique to B2G (eg "B2G Startup") as the 'test name', otherwise we'll get bugs from all platforms being suggested in TBPL.

It's set here:
http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/mozrunner/mozrunner/remote.py#132

Comment 1

[Ed Morley [:emorley]]

I guess we could give a more friendly names for the other places we do this too:
http://mxr.mozilla.org/mozilla-central/search?string=lastTestSeen+%3D&tree=mozilla-central

Comment 2

[Ed Morley [:emorley]]

Attachment: Use 'B2G-Startup' as the test name during startup

Until we run the first test, set the test name to 'B2G-Startup' rather then 'automation' so any errors are less likely to result in false positives in TBPL, eg:
TEST-UNEXPECTED-FAIL | B2G-Startup | application timed out after 330.0 seconds with no output

Comment 3

[Andrew Halberstadt [:ahal]]

Comment on attachment 8442770 [details] [diff] [review]
Use 'B2G-Startup' as the test name during startup

Review of attachment 8442770 [details] [diff] [review]:
-----------------------------------------------------------------

The patch in bug 997244 gets rid of the B2GRunner class. This is still do-able, but I'd prefer to wait for the other bug to land as it is already very complicated.

Comment 4

[Ed Morley [:emorley]]

Attachment: Use 'B2G-Startup' as the test name during startup

Same as before, except rebased on top of bug 997244.

Comment 5

[Andrew Halberstadt [:ahal]]

Comment on attachment 8468501 [details] [diff] [review]
Use 'B2G-Startup' as the test name during startup

Review of attachment 8468501 [details] [diff] [review]:
-----------------------------------------------------------------

This is the base runner class which is used for all mozrunners including desktop Firefox. In this version of mozrunner specific "applications" are isolated from the various runner classes. So b2g specific things go in the B2GContext class in application.py. So we can either set the "last_test" property on the app_ctx object, or else change this string to "Runner-Startup" or something similar. Let me know what you prefer and if you'd like me to come up with a patch.

Comment 6

[Ed Morley [:emorley]]

Bah I wasn't paying attention to the filename after grepping, yeah agree - mozrunner-startup or something will do for now.

Comment 7

[Ed Morley [:emorley]]

Attachment: Use 'mozrunner-startup' instead of 'automation' as the test name during startup

Comment 8

[Ed Morley [:emorley]]

Thanks :-)

remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/968a007bad3f

Comment 9

[Nigel Babu [:nigelb]]

https://hg.mozilla.org/mozilla-central/rev/968a007bad3f