1129687 - [Raptor] Change log parser to support User Timing format changes

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::PerformanceTest, defect)

Description

[:Eli Perelman]

When the User Timing logging [1] is completed, we will need to make some adjustments to the log parser to ensure we can continue to read the information from logcat. Performance testing is blocked until we get the logging and parser changes completed.

[1] Bug 1115130

Comment 1

[:Eli Perelman]

Attachment: Link to Github pull-request: https://github.com/eliperelman/node-mozdevice/pull/6

Comment 2

[:Eli Perelman]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/raptor/pull/14

Comment 3

[GitHub Autolander Bot]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/28274

Comment 4

[Kevin Grandon :kgrandon]

Comment on attachment 8565608 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/28274

This seems fine to me, but please do not bump package.json without updating the node_modules.revision as well. (I assume you will do this after the raptor bits get a review). Thanks for the patch!

Comment 5

[Robert Wood [:rwood]]

(In reply to :Eli Perelman from comment #2)
> Created attachment 8565590 [details] [review]
> Link to Github pull-request: https://github.com/mozilla-b2g/raptor/pull/14

/lib/suite/restart-b2g.js needs to be updated also please (correct?):

https://github.com/mozilla-b2g/raptor/blob/master/lib/suite/restart-b2g.js#L75

Comment 6

[:Eli Perelman]

Yes, you are correct, my mistake.

Comment 7

[Robert Wood [:rwood]]

(In reply to :Eli Perelman from comment #2)
> Created attachment 8565590 [details] [review]
> Link to Github pull-request: https://github.com/mozilla-b2g/raptor/pull/14

Ok, during testing of the launch test I notice some strange behaviour with app priming. I cannot see how this is related to this particular patch; and I'm guessing it's a new problem introduced by upgrading to the latest gecko maybe... but I'll note it here; if you determine it is unrelated for certain pls let me know and I will create a new bug.

So during app priming, the wrong app is often launched! For example, when running the launch test for 'clock', I've seen the priming launch settings, sometimes email, marketplace, etc (not always the same incorrect app). I notice that the homescreen is scrolled down a bit when it happens. Perhaps the swipehack is acting up again...

Btw, I noticed once that priming launched 'settings', and then the first run of the test started and the test launched 'clock' correctly... which brings me to a point, the launch test should probably fail out / not continue if priming launches an incorrect app.

Comment 8

[Robert Wood [:rwood]]

Interesting, I can't seem to reproduce the app priming/swipehack issue without these patches implemented (and hacked so it doesn't wait for homescreen marks, just a timeout). With the patches implemented it is easily reproducible.

Comment 9

[Robert Wood [:rwood]]

Eli: hey another issue, the raptor.log is empty after the launch test, cold-launch.js:handleRun needs to be updated to reflect the new appname (i.e. appName + 'clock.gaiamobile.org' instead of 'Clock').

Comment 10

[Robert Wood [:rwood]]

(In reply to Robert Wood [:rwood] from comment #8)
> Interesting, I can't seem to reproduce the app priming/swipehack issue
> without these patches implemented (and hacked so it doesn't wait for
> homescreen marks, just a timeout). With the patches implemented it is easily
> reproducible.

Ok, first off I was missing the gaia patch from comment 4 above (I had just looked at the PR link and saw mozdevice so thought it was the one I already pulled, note to self to check the attachment hah). However, even after resetting my local gaia, raptor, and node-mozdevice repos to master, then creating new local branches again and pulling all the PRs, I still reproduced the app priming problem on the second try. Email app was launched at priming instead of clock.

Comment 11

[Robert Wood [:rwood]]

Update: I notice when the bug happens, and a wrong app is launched for priming, is always an app in 1, 2, or even 3 rows directly below the app you are testing. i.e. testing clock, but during priming I've seen (and only seen) either email, settings, marketplace, or the mochitest apps incorrectly launched. It looks like the homescreen isn't fully displayed when priming starts; perhaps the 'fullyLoaded' for vertical homescreen is coming too early.

Comment 12

[Robert Wood [:rwood]]

I moved the swipehack so it is called from within app priming (after the 6 second delay in priming) and that fixes it. Ran the launch test on flame 50 times (single run each time) and it always primed the correct app. So it looks like the swipehack is being called too early sometimes, and that somehow causes it to launch an app instead of do the swipe.

Note that my flame was set to 319MB which may be the reason why I could reproduce it on my device. I'll try the same fix with the emulator also.

Comment 13

[Robert Wood [:rwood]]

Comment on attachment 8565590 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/raptor/pull/14

R+. Verified in conjunction with the other patches in this bug. Ran the launch test several times on both the flame-kk (319) and the b2g-emulator-kk (512).

Comment 14

[Robert Wood [:rwood]]

Note: The issue/fix in comment 12 is in a separate bug, bug 1134868.

Comment 15

[:Eli Perelman]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/gaia-node-modules/pull/94

Comment 16

[:Eli Perelman]

Try run: https://treeherder.mozilla.org/#/jobs?repo=gaia-try&revision=01850e9a4f21
In master: https://github.com/mozilla-b2g/gaia/commit/798522e8b4b45afe63fe63eb319043928c603d51