[Raptor] Cache application coordinates to improve application launch variance

RESOLVED FIXED in FxOS-S4 (07Aug)

Status

RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: Eli, Assigned: Eli)

Tracking

unspecified
FxOS-S4 (07Aug)
ARM
Gonk (Firefox OS)

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [systemsfe])

Attachments

(1 attachment)

(Assignee)

Description

3 years ago
When Raptor uses Marionette, we search for and launch the application at every run. During this process, we start a Marionette session, launch the app, and while launch kill the Marionette session. I believe this activity during app launch may be contributing to some variance in times.

Cache the coordinates using Marionette in advance of launch and use those coordinates each time.
(Assignee)

Comment 1

3 years ago
Created attachment 8641239 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/raptor/pull/15
Attachment #8641239 - Flags: review?(rwood)

Comment 2

3 years ago
(In reply to :Eli Perelman from comment #1)
> Created attachment 8641239 [details] [review]
> Link to Github pull-request: https://github.com/mozilla-b2g/raptor/pull/15

I tested launching each of the 12 apps on aries, flame, and emulator and it worked. However retries doesn't work as I expected. If I manually scroll the lcd before an app launch, so that the wrong app is launched, I had expected a retry to scroll the homescreen back into place so the next tap could work, however there's no retry and there's a marionette error instead:

https://gist.github.com/rwood-moz/bb68156d8400b44f75d2
(Assignee)

Comment 3

3 years ago
I'm changing up the way the Marionette connection works, so hopefully I can get this resolved soon.
(Assignee)

Comment 4

3 years ago
OK, PR updated. Please let me know if the retry works better now.

Comment 5

3 years ago
Comment on attachment 8641239 [details] [review]
Link to Github pull-request: https://github.com/mozilla-b2g/raptor/pull/15

LGTM. Retries are working again with latest update. The unrelated marionette polling socket timeout issue (Bug 1188990) prevented me from trying the latest update on the emulator, but I'm satisfied with the patch as it works great on device.
Attachment #8641239 - Flags: review?(rwood) → review+
(Assignee)

Comment 6

3 years ago
In master: https://github.com/mozilla-b2g/raptor/commit/0d2151892b2f8254636b7e02d5e75c3bb8b0e8a0
In npm: @mozilla/raptor v1.4.5
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Whiteboard: [systemsfe]
(Assignee)

Comment 7

3 years ago
As an update, here is a comparison against the Clock app, 30 runs:

[No coordinate cache]

| Metric                           | Mean     | Median   | Min     | Max      | StdDev  | p95      |
| -------------------------------- | -------- | -------- | ------- | -------- | ------- | -------- |
| coldlaunch.navigationLoaded      | 967.200  | 992.500  | 702.000 | 1106.000 | 94.081  | 1078.000 |
| coldlaunch.navigationInteractive | 1057.933 | 1091.000 | 802.000 | 1210.000 | 99.256  | 1165.000 |
| coldlaunch.visuallyLoaded        | 1356.733 | 1404.500 | 975.000 | 1527.000 | 137.964 | 1466.000 |
| coldlaunch.contentInteractive    | 1357.000 | 1404.500 | 975.000 | 1529.000 | 138.021 | 1467.000 |
| coldlaunch.fullyLoaded           | 1357.400 | 1405.500 | 975.000 | 1530.000 | 138.091 | 1467.000 |
| coldlaunch.pss                   | 19.134   | 19.127   | 18.835  | 19.423   | 0.109   | 19.339   |
| coldlaunch.uss                   | 14.501   | 14.475   | 14.406  | 14.773   | 0.079   | 14.676   |
| coldlaunch.rss                   | 31.042   | 31.023   | 30.953  | 31.277   | 0.071   | 31.195   |

---

[Coordinate Cache]

| Metric                           | Mean     | Median   | Min      | Max      | StdDev | p95      |
| -------------------------------- | -------- | -------- | -------- | -------- | ------ | -------- |
| coldlaunch.navigationLoaded      | 1076.467 | 1080.000 | 1010.000 | 1175.000 | 47.993 | 1166.000 |
| coldlaunch.navigationInteractive | 1170.800 | 1174.500 | 1107.000 | 1278.000 | 45.740 | 1261.000 |
| coldlaunch.visuallyLoaded        | 1412.367 | 1400.500 | 1299.000 | 1565.000 | 59.067 | 1540.000 |
| coldlaunch.contentInteractive    | 1412.467 | 1401.000 | 1299.000 | 1565.000 | 59.094 | 1540.000 |
| coldlaunch.fullyLoaded           | 1412.733 | 1401.000 | 1300.000 | 1565.000 | 59.051 | 1540.000 |
| coldlaunch.rss                   | 31.078   | 31.117   | 30.785   | 31.234   | 0.121  | 31.184   |
| coldlaunch.pss                   | 19.217   | 19.271   | 18.936   | 19.378   | 0.134  | 19.336   |
| coldlaunch.uss                   | 14.549   | 14.588   | 14.254   | 14.695   | 0.119  | 14.652   |

---

Big thing to note is the improvement in standard deviation from around 138ms down to 59ms, which is pretty significant. The means and p95s look smaller because greater ranges were produced, not because there were any actual improvements.
(Assignee)

Comment 8

3 years ago
Also, here is a set of runs from when Raptor was in the Gaia tree, Clock, 30 runs:

Metric                            Mean      Median    Min       Max       StdDev  p95
--------------------------------  --------  --------  --------  --------  ------  --------
coldlaunch.navigationLoaded       981.067   993.000   762.000   1108.000  84.570  1076.000
coldlaunch.navigationInteractive  1074.000  1095.000  829.000   1194.000  94.260  1167.000
coldlaunch.visuallyLoaded         1297.867  1318.000  1049.000  1418.000  96.694  1399.000
coldlaunch.contentInteractive     1297.933  1318.000  1049.000  1418.000  96.729  1399.000
coldlaunch.fullyLoaded            1298.367  1318.500  1049.000  1418.000  96.825  1399.000
coldlaunch.pss                    21.139    21.482    18.776    21.578    0.823   21.573
coldlaunch.uss                    16.489    16.830    14.148    16.926    0.818   16.918
coldlaunch.rss                    32.967    33.310    30.551    33.414    0.832   33.414

---

Standard deviation sits around 96ms, so we are even improved from that!
You need to log in before you can comment on or make changes to this bug.