1037500 - Clock test fails from script timeout, preventing results from appearing in Datazilla

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::PerformanceTest, defect, P1)

Description

[:Eli Perelman]

The Clock test is not showing any results on Datazilla. After digging into the logs in Jenkins it appears that there is a socket timeout occurring which errors the tests for the Clock.

Logs:
=====

ScriptTimeout',
  stack: 'ScriptTimeout: (28) timed out
  Remote Stack:
	<none>
    at Error.MarionetteError (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/error.js:67:13)
    at Object.Client._handleCallback (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:476:19)
    at /var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:510:21
    at TcpSync.send (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/drivers/tcp-sync.js:100:10)
    at Object.send (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:457:36)
    at Object.Client._sendCommand (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:503:19)
    at Object._executeScript (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:1468:19)
    at Object.executeAsyncScript (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:1249:19)
    at Object.PerfTestApp.waitForPerfEvents (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/tests/performance/app.js:95:17)
    at Object.PerformanceHelper.waitForPerfEvent (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/tests/performance/performance_helper.js:190:16)
    at /var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/tests/performance/startup_events_test.js:94:25
    at Object.PerformanceHelper.task (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/tests/performance/performance_helper.js:178:7)
    at /var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/tests/performance/performance_helper.js:164:16
    at Object.Client.waitForSync (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:668:18)
    at Object.Client.waitFor (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:650:60)
    at Object.Helper.delay (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/tests/js-marionette/helper.js:28:12)
    at Object.PerformanceHelper.delay (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/tests/performance/performance_helper.js:186:24)
    at trigger (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/tests/performance/performance_helper.js:163:14)
    at Object.PerformanceHelper.repeatWithDelay (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/tests/performance/performance_helper.js:168:7)
    at Context.<anonymous> (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/tests/performance/startup_events_test.js:89:23)
    at callFn (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:223:21)
    at Test.Runnable.run (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:216:7)
    at Runner.runTest (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:373:10)
    at /var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:451:12
    at next (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:298:14)
    at /var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:308:7
    at next (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:246:23)
    at /var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:270:7
    at done (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:185:5)
    at callFn (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:228:7)
    at Hook.Runnable.run (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:216:7)
    at next (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:258:10)
    at /var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:270:7
    at done (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:185:5)
    at /var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:199:9
    at Object.executeHook (/var/jenkins/1/workspace/flame.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:369:18)
    at process._tickCallback (node.js:419:13)

Comment 1

[Hubert Figuiere [:hub]]

It is a script timeout.

Comment 2

[Hubert Figuiere [:hub]]

I tried a quick gaia bisect on that, and even before, when I know this worked, I get the regression. *sigh*

Comment 3

[:Eli Perelman]

I am going to take this one and try to do some investigation.

Comment 4

[Hubert Figuiere [:hub]]

It actually never worked after implementing bug 1015386

commit
19b93abbe61c1f0a5941cd4ccbd125b9bfd520c1

Even with an alarm created.

Comment 5

[:Eli Perelman]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/22280

This patch adds the performance helper script back to the Clock app so we can run performance tests against it without timeouts. I've verified this patch locally using `APP=clock RUNS=30 make test-perf` and it runs perfectly.

Comment 6

[Carsten Book [:Tomcat]]

master: https://github.com/mozilla-b2g/gaia/commit/abd97f96f8c4631aff3238a917510151e1bc3a5b

Comment 7

[:Eli Perelman]

Requesting uplift to 2.0 as the original implementation has already landed on 2.0 and is non-functional without this patch. These are important for meeting our release performance acceptance criteria.

[Feature/regressing bug #]: bug 1015386
[User impact if declined]: none
[Describe test coverage new/current, TBPL]: Feature only triggers events for testing, no user-facing features or tests
[Risks and why]: Low, as there are no user-perceived changes
[String/UUID change made/needed]: n/a

Comment 8

[:Eli Perelman]

Comment on attachment 8464319 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/22280

See comment 2.

Comment 9

[:Eli Perelman]

Apologies, see comment 7 (not 2).

Comment 10

[Ryan VanderMeulen [:RyanVM]]

v2.0: https://github.com/mozilla-b2g/gaia/commit/5ca7d2a3dfe56ac34d2fe64f4bf1047db9fc31dc