1067625 - [Calendar] Datazilla is missing calendar App when measuring moz-app-visually complete

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::Calendar, defect)

Description

[Tony Chung [:tchung]]

[Blocking Requested - why for this release]:

When analyzing bug 1059349, i noticed that Calendar App has disappeared from datazilla testing results recently.   This is true for master, 2.0, and 2.1 branches at 319MB.  

Spoke to :Hub, and he investigated with the following information:

"It seems that there is a segfault occuring on the PC when running the
test for calendar.  That's what I can see in Jenkins. I have never been able to reproduce
that problem in the past but I have observed it in the wild - only on
Ubuntu.

The crash is NOT on the phone. It is on the PC. It is 'make' on Ubuntu.  Here is the original bug, we worked around it:  https://bugzilla.mozilla.org/show_bug.cgi?id=987383.   Those who have access to the jenkins machines, which exclude me. The A-Team I think."

Going to nominate this bug for detective work and why we arent getting calendar startup results for the Flame.   I can see data results returning if you set the filter to 60 days, but it doesnt appear at 30 days.

Comment 1

[Dylan Oliver [:doliver]]

triage: internal metric, wouldn't block on this but I'm marking it priority and getting it on the schedule for sprint 5.

Comment 2

[Dylan Oliver [:doliver]]

[priority] --> tracking-b2g:+ conversion

Comment 3

[Candice Serran (:cserran)]

James, can you help repro this? Gaye isn't able to repro and we need someone to verify through mtv vpn.

Comment 4

[:Eli Perelman]

Researching.

Comment 5

[:Eli Perelman]

The offending section of the console output:
--------------------------------------------

node_modules installed.
touch -c node_modules
make[1]: Leaving directory `/var/jenkins/2/workspace/flame-kk-319.b2g-inbound.perf.gaia'

/var/jenkins/2/workspace/flame-kk-319.b2g-inbound.perf.gaia/shared/test/integration/profile_builder.js:117
        throw err;
              ^
Error: Command failed: make: *** [app] Segmentation fault (core dumped)

    at ChildProcess.exithandler (child_process.js:647:15)
    at ChildProcess.EventEmitter.emit (events.js:98:17)
    at maybeClose (child_process.js:753:16)
    at Socket.<anonymous> (child_process.js:966:11)
    at Socket.EventEmitter.emit (events.js:95:17)
    at Pipe.close (net.js:465:12)
calendar returned no output.

Comment 6

[:Eli Perelman]

Blowing up at profile_builder.js trying to run what appears to be `make -C $GAIA_ROOT`.

Comment 7

[:Eli Perelman]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/25782

Looks like the test is timing out right before running the Calendar performance test because the profile creation is now longer than 10 seconds. The performance script is a copy of the "gaia-marionette" script and was updated to 60 seconds back in January 2014, but this script was never updated.

Comment 8

[Carsten Book [:Tomcat]]

https://github.com/mozilla-b2g/gaia/commit/905e54e1e9e405d50b1e098708983b5abac1baa4

Comment 9

[:Eli Perelman]

Requesting uplift of *performance-only* TEST-ONLY changes to v2.0 and v2.1. This has no effect on anything else but the timeout of performance tests.

Comment 10

[Ryan VanderMeulen [:RyanVM]]

v2.1: https://github.com/mozilla-b2g/gaia/commit/445656e36d14054d46a8f201dc68d76da8cfa281
v2.0: https://github.com/mozilla-b2g/gaia/commit/e19ba2a049f192a560c6e63a0cc213b0353f1a02

Comment 11

[:Eli Perelman]

While the timeouts fixed the issue I was having locally with this process, it still seems to fail on the Jenkins boxes. I'm currently debugging on the servers themselves to determine where the problem lies.

Comment 12

[:Eli Perelman]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/25906

Could I get one of you gents to review this patch? By ensuring profile creation happens in the bin script (which prior only happened for b2g-desktop-host), we can prevent the segfault that occurs on Linux when trying to run make test-perf without a profile-test directory.

Comment 13

[James Lal [:lightsofapollo]]

Comment on attachment 8518509 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/25906

https://github.com/mozilla-b2g/gaia/commit/5f8206bab97cdd7b547cc2c8953cadb2a80a7e11

Comment 14

[Miller Medeiros [:millermedeiros]]

Looks like it broke again, any idea of what could be wrong?

Comment 15

[:Eli Perelman]

According to the logs:

---

Error: timeout exceeded!
	    at Object.Client.waitForSync (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:693:16)
	    at Object.Client.waitFor (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:661:60)
	    at Object.MarionetteHelper.waitForElement (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/marionette-helper/index.js:142:12)
	    at Object.PerfTestApp.launch (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/tests/performance/app.js:55:24)
	    at /var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/tests/performance/startup_events_test.js:57:11
	    at Object.PerformanceHelper.task (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/tests/performance/performance_helper.js:205:5)
	    at /var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/tests/performance/performance_helper.js:191:14
	    at Object.Client.waitForSync (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:679:18)
	    at Object.Client.waitFor (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:661:60)
	    at Object.Helper.delay (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/tests/js-marionette/helper.js:48:12)
	    at Object.PerformanceHelper.delay (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/tests/performance/performance_helper.js:213:22)
	    at trigger (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/tests/performance/performance_helper.js:190:12)
	    at Object.PerformanceHelper.repeatWithDelay (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/tests/performance/performance_helper.js:195:5)
	    at Context.<anonymous> (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/tests/performance/startup_events_test.js:54:23)
	    at callFn (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:223:21)
	    at Test.Runnable.run (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:216:7)
	    at Runner.runTest (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:373:10)
	    at /var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:451:12
	    at next (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:298:14)
	    at /var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:308:7
	    at next (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:246:23)
	    at /var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:270:7
	    at done (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:185:5)
	    at callFn (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:228:7)
	    at Hook.Runnable.run (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:216:7)
	    at next (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:258:10)
	    at /var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runner.js:270:7
	    at done (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:185:5)
	    at /var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/mocha/lib/runnable.js:199:9
	    at Object.executeHook (/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.perf.gaia/node_modules/marionette-client/lib/marionette/client.js:371:18)
	    at process._tickCallback (node.js:419:13)

---

It appears the test dies during the app.launch() method, specifically on the .waitForElement('body') call here:

```js
var self = this;
this.client.apps.launch(this.origin, this.entryPoint, function(err, app) {
  if (app) {
    self.instance = app;
  }
});
this.client.apps.switchToApp(this.origin);
this.client.helper.waitForElement('body');
```

I find it really weird that it would die at this point, though, waiting for the body element. Any insight as to changes in the past couple months that would affect marionette's ability to find the body element?

Comment 16

[:Eli Perelman]

Datazilla is deprecated, and Calendar data is now displayed in Raptor.