Closed
Bug 1320263
Opened 8 years ago
Closed 8 years ago
Feasibility for Fennec performance assessment
Categories
(Firefox for Android Graveyard :: General, defect, P1)
Firefox for Android Graveyard
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: wesley_huang, Assigned: jwu)
References
Details
User Story
We would like to make sure we cover the basics (performance and stability) to make sure users do not leave Firefox to other browsers. HASAL is a tool that allows us to understand performance of different browsers using video recording and analysis techniques, which will provide us with results in Running Time, Speed Index (SI), Perceptual Speed Index (PSI). What are we measuring? App launch time Page load time Perceived page load time Scrolling *reference[1]: We already have such automated but it's only for Fennec https://wiki.mozilla.org/EngineeringProductivity/Autophone#S1S2_Test What other mobile browsers are being tested against Firefox Top 3 mobile browsers globally (excluding Safari) Chrome UC Browser Opera Websites to be tested TBD Reference Test Devices Google Pixel Sony Z3C Google Nexus? “Useful information for users” - (data that user is interested in) Web page test: https://www.webpagetest.org/ Architecture: https://sites.google.com/a/webpagetest.org/docs/system-design/mobile-testing Agent to controll device: https://sites.google.com/a/webpagetest.org/docs/private-instances/node-js-agent/setup Repo: https://github.com/WPO-Foundation/webpagetest/tree/master/agent/js
Attachments
(1 file)
103 bytes,
text/x-google-doc
|
Details |
Hasal[1] team had done efforts on measuring the performance of Firefox desktop. It would be nice if we can integrate Fennec with that as well. Some initial questions as below: a. how to generate screen capture video on android? b. how to automate the user behavior (launch fennec, initiate a page load, scroll ...etc) on android? [1] https://wiki.mozilla.org/Hasal
Comment 1•8 years ago
|
||
Hi Wesley, a. Screen recording can be performed by below command, but this could also impact overall device performance leads to "Uncertainty Principle" ref:https://developer.android.com/studio/command-line/adb.html#screenrecord adb shell screenrecord /sdcard/demo.mp4 b. "adb shell" command can fulfill most requirement(load, tap, scroll, ...etc). Swipe gesture can be simulated by adb command as below. Unfortunately, swipe does not work well on fennec, Google Chrome worked fine. Possible caused by different MotionEvent handling. adb shell am start -a android.intent.action.VIEW -d "https://www.cnet.com/" org.mozilla.fennec_aurora/org.mozilla.fennec_aurora.App adb shell "input swipe 100 960 900 960 200" adb shell "input tap 1020 280" adb shell "input tap 230 710"
Assignee: nobody → max
Flags: needinfo?(whuang)
Reporter | ||
Comment 2•8 years ago
|
||
Thank you Max for the input. One open question we got while chatting with the Hasal team: "how do we determine a web page loading DONE?"
Flags: needinfo?(whuang)
Reporter | ||
Comment 3•8 years ago
|
||
On the requirement side, we would also need to define: a. a list of top sites we would like to measure. I believe it's different from the list for desktop. Google doc is very likely not as important as in Fennec. b. prioritize what else browsers we would like to benchmark.
Flags: needinfo?(jcheng)
Comment 4•8 years ago
|
||
(In reply to Max Liu [:maliu] from comment #1) > b. "adb shell" command can fulfill most requirement(load, tap, scroll, > ...etc). Swipe gesture can be simulated by adb command as below. > Unfortunately, swipe does not work well on fennec, Google Chrome worked > fine. Possible caused by different MotionEvent handling. Yep, bug 1312482 should help. (In reply to Wesley Huang [:wesley_huang] (EPM) (NI me) from comment #2) > Thank you Max for the input. > One open question we got while chatting with the Hasal team: "how do we > determine a web page loading DONE?" Our s1s2 autophone tests do this by looking at the device log. More about that here: https://wiki.mozilla.org/EngineeringProductivity/Autophone#S1S2_Test
Comment 5•8 years ago
|
||
thanks Wesley. let me create a quick doc to answer your questions. leaving the NI so I don't forget
Reporter | ||
Updated•8 years ago
|
User Story: (updated)
Reporter | ||
Comment 6•8 years ago
|
||
As per the meeting together with Hasal and Fennec team, we had aligned on the long-term goal and what we can start with. The living document[1] has more details. It requires permission so I abstract most info to "user story" of this bug. For next step, let's target for creating a simpler PoC (proof of concept) that is capable for a pilot run. With that we should be able to have initial understanding of the page loading, and speed index on Fennec, and other mobile browsers. [1] https://docs.google.com/document/d/1OJi2ZBKrjvvc0vPF_qsIrTCr4s1NLgw7E6wpMT91p34/edit
Reporter | ||
Updated•8 years ago
|
Summary: Feasibility of integrating Fennec into Project Hasal (for performance monitoring) → Feasibility/PoC of integrating Fennec into Project Hasal (for performance monitoring)
Comment 7•8 years ago
|
||
It seems that we are expanding HASAL to measure beyond web content/rendering performance, but also application performance in general. I don't have a strong opinion on that other than we should be clear which part we are measuring and understand what's required to fix a specific performance bottleneck. It's also worthy to note that we could measure application performance through other means, like telemetry marker etc.
Comment 8•8 years ago
|
||
Updating the summary and remove mention of HASAL, as we are talking about app performance, not content performance. We might be able to share their dashboard infra etc but not now...
Summary: Feasibility/PoC of integrating Fennec into Project Hasal (for performance monitoring) → Feasibility/PoC for Fennec performance assessment
Reporter | ||
Updated•8 years ago
|
Priority: -- → P1
Updated•8 years ago
|
Assignee: max → topwu.tw
Updated•8 years ago
|
Flags: needinfo?(jcheng)
Reporter | ||
Updated•8 years ago
|
Summary: Feasibility/PoC for Fennec performance assessment → Feasibility for Fennec performance assessment
Reporter | ||
Comment 10•8 years ago
|
||
Hi Al, Since we changed the scope of this bug to be a "preliminary testing", how about attaching the testing result/report here and then we can close it.
Flags: needinfo?(atsai)
![]() |
||
Comment 11•8 years ago
|
||
Sounds good for me. https://docs.google.com/spreadsheets/d/1DJfcMf1QAP9iM-4_6uH3viP6iwWp73ppUV8fnAiKDBo/edit#gid=1003657603
Flags: needinfo?(atsai)
![]() |
||
Comment 12•8 years ago
|
||
Reporter | ||
Comment 13•8 years ago
|
||
The preliminary testing was done. Follow up will be Bug 1340965 - PoC for Fennec performance assessment
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Updated•4 years ago
|
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•