Please report any other irregularities here.
52 bytes, text/plain
46 bytes, text/x-github-pull-request
|Details | Review | Splinter Review|
Our starting moment for make test-perf essentially occurs at document-element-inserted , which is after app touch, iframe instantiation, and splash screen. We need to move this starting moment to occur right after app touch. The timestamp captured at  appears to be the best existing candidate as tracing through the code shows that this would occur close to 0ms after touch.  https://github.com/mozilla-b2g/gaia/blob/master/tests/performance/performance_helper_atom.js#L115  https://mxr.mozilla.org/mozilla-central/source/dom/apps/src/Webapps.js#477
Whiteboard: [c=automation p= s= u=] → [c=automation p= s=2014.07.18 u=]
Whiteboard: [c=automation p= s=2014.07.18 u=] → [c=automation p= s= u=]
Whiteboard: [c=automation p= s= u=] → [c=automation p=3 s=2014.07.18 u=]
Whiteboard: [c=automation p=3 s=2014.07.18 u=] → [c=automation p=3 s= u=]
We currently programmatically launch the app. "App Touch" as you put it is when the homescreen get a tap. That mean instrumenting the homescreen. I'd rather instrument that separately.
The goals  we have defined for the launching of applications is the aggregate of time between the app launch, whether user-initiated or simulated, and the invocation of "moz-app-visually-complete". The aggregate goal is to be within 1000ms, and is what we are basing our release acceptance on . So the only way we can back up these goals with our measurements is to get the start time as close to user-perceived app launch as possible.  https://developer.mozilla.org/en-US/Apps/Build/Performance/Firefox_OS_app_responsiveness_guidelines#Goals  https://wiki.mozilla.org/FirefoxOS/Performance/Release_Acceptance#2.0
Do we launch the apps directly, or do we inject a tap to launch? Also, where does the elapsed-time measurement happen? Typically, for a metric, you'd anchor this sort of timing on the impulse on the script side (i.e. the tap or call to a launch method). But that assumes the test is measuring the elapsed time between impulse and and the event independently. The big reason to do it on the system side is if the events contain their own timing result, but if we're measuring the timeline from the test script then it might not be necessary to change on the system side.
We launch the app directly as said in Comment 1.
Thanks for clarifying. Programmatically could also mean user event injection, which uses a different code path. If the test code intercepts the events directly, might make sense for the timeline to be calculated on the test side, starting at the programmatic call. If nothing else, could be a crutch while we tune the rest.
Whiteboard: [c=automation p=3 s= u=] → [c=automation p=4 s= u=]
Created attachment 8460768 [details] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/22058/files This patch captures the launch creation timestamp and passes it along for use in the tests. The startup tests have been modified to allow overloading of the results to provide a launch time delta.
Attachment #8460768 - Attachment mime type: text/plain → text/x-github-pull-request
Attachment #8460768 - Attachment mime type: text/x-github-pull-request → text/plain
Comment on attachment 8460768 [details] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/22058/files This is expected to slow down the performance result since you move the start point earlier. Please note this.
Attachment #8460768 - Flags: review?(alive) → review+
Alive is correct, this will show an increase in the start time for all applications in the *new* launch events being captured by make test-perf, not the ones currently being used in b2gperf.
Comment on attachment 8460768 [details] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/22058/files Requesting uplift to 2.0 as it is important for meeting our release performance acceptance criteria and providing more accurate numbers for the criteria. [Feature/regressing bug #]: bug 996038 [User impact if declined]: none [Describe test coverage new/current, TBPL]: These changes only affect numbers gathered for a single performance test [Risks and why]: Low, as there are no user-perceived changes [String/UUID change made/needed]: n/a
Attachment #8460768 - Flags: approval-gaia-v2.0?
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
There is a regression, Need a followup.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Created attachment 8461464 [details] [review] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/22114
Attachment #8461464 - Flags: review?(hub)
Comment on attachment 8461464 [details] [review] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/22114 This patch needs the same gaia-approval-v2.0 for the same reason as the other patch.
Attachment #8461464 - Flags: approval-gaia-v2.0?
Comment on attachment 8461464 [details] [review] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/22114 that's it. r=me
Attachment #8461464 - Flags: review?(hub) → review+
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago → 4 years ago
Resolution: --- → FIXED
Attachment #8460768 - Flags: approval-gaia-v2.0? → approval-gaia-v2.0+
Attachment #8461464 - Flags: approval-gaia-v2.0? → approval-gaia-v2.0+
v2.0: https://github.com/mozilla-b2g/gaia/commit/28f807f3f86f98779b40e2fb9ea7f20da00890b6 v2.0: https://github.com/mozilla-b2g/gaia/commit/99a285a504dee269eefb1503599a7705f2529989
status-b2g-v2.0: --- → fixed
status-b2g-v2.1: --- → fixed
Target Milestone: --- → 2.1 S1 (1aug)
You need to log in before you can comment on or make changes to this bug.