Last Comment Bug 852744 - Add a cross-browser page loading test to the eideticker dashboard
: Add a cross-browser page loading test to the eideticker dashboard
Status: RESOLVED FIXED
:
Product: Testing
Classification: Components
Component: Eideticker (show other bugs)
: Trunk
: x86_64 Linux
: -- normal (vote)
: ---
Assigned To: William Lachance (:wlach)
:
:
Mentors:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-03-19 15:31 PDT by William Lachance (:wlach)
Modified: 2013-04-10 09:57 PDT (History)
5 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments

Description William Lachance (:wlach) 2013-03-19 15:31:32 PDT
We should have an eideticker test that measures the amount of time to fully render a site like cnn.com in the browser when an instance of the browser is not currently running. This represents the user story of someone clicking a link in another android application (e.g. twitter or gmail).

We should try to generate comparative numbers for the stock browser and chrome for this under android 4.2, to make sure we're competitive. Most existing eideticker tests are hard to do cross browser, but this is one we pretty safely can do, so we should probably do it since there seems to be a lot of interest in getting this kind of data.

Most of the work for this is already done, though a few minor things may have broken since I haven't run this test in a while. The todo list looks something like this:

1. Add support to the dashboard for using a "baseline" number for stock and chrome, so we're not constantly generating new numbers for them every day (which is wasteful)
2. Add a startup variant of cnn.com.
3. Fix any random bugs preventing this from working.
4. Add this test to the dashboard!
Comment 1 William Lachance (:wlach) 2013-04-02 15:40:33 PDT
See bug 758338, which discusses comparing network perf of firefox vs. chrome on mobile.
Comment 2 William Lachance (:wlach) 2013-04-02 16:05:10 PDT
Pushed first round of work on this:

https://github.com/mozilla/eideticker/commit/09efaf11bc34874e69b52adcbb1c79e547b64a00

(more to come)

Dashboard results here:

http://eideticker.wrla.ch/#/samsung-gn/nytimes-load/timetostableframe

The short summary is that we're quite bad in comparison to the competition. Like 3 or 4 times slower. I'll file a bug about this.

I need to at least update the dashboard to show baseline results appropriately, as well as add this test to the nightly schedule. Probably also want to do a profiling run of this on fennec, so we can see where any bottlenecks are.
Comment 3 William Lachance (:wlach) 2013-04-02 16:16:49 PDT
Filed bug 857359 about the bad performance.
Comment 4 William Lachance (:wlach) 2013-04-10 09:57:34 PDT
Added this to the default run for the samsung gn and lg g2x and dashboard now displays baseline results. Calling this done, if anything else is desired please file a follow up bug. :)

Note You need to log in before you can comment on or make changes to this bug.