758338 - Compare network performance between Firefox and Chrome on mobile

Status: RESOLVED DUPLICATE of bug 857359

(Firefox for Android Graveyard :: General, defect, P5)

Description

[Kartikaya Gupta (email:[e-mail])]

Whenever I load a page in Chrome to compare some behaviour against ours, it seems like they load the page much faster than we do, on the same device. Often I find myself waiting for pages to load in Fennec thinking I have a bad network connection but when I switch over to Chrome everything loads super fast. I feel like we should set up a test server to serve some pages and analyze the incoming network requests to see what Chrome is doing differently.

Comment 1

[Mike Hommey [:glandium]]

I wonder how much of this could be due to DNS resolution. It might be better to sniff the network. I guess it's possible to tcpdump on a rooted device.

Comment 2

[William Lachance (:wlach)]

(In reply to Mike Hommey [:glandium] from comment #1)
> I wonder how much of this could be due to DNS resolution. It might be better
> to sniff the network. I guess it's possible to tcpdump on a rooted device.

I've noticed this slowness with eideticker even when loading sites directly from a machine on the local network (via ip) so I don't think that's the cause.

Comment 3

[Kartikaya Gupta (email:[e-mail])]

Just as a preliminary test to get my tools up and running, I installed tcpdump on a rooted Galaxy Tab 10.1 and loaded a "hello world" page over the Wi-Fi LAN in both Firefox and the stock browser. The tcpdump traces showed the stock browser taking 0.145924 seconds from initial tcp SYN to final tcp ACK. The favicon and main page were fetched on the same TCP connection using Connection: keep-alive. Firefox took 0.66468 seconds for the same data, and used two different TCP connections (one for the main page, and one for the favicon). Even if I ignore the favicon stream completely, the main page TCP stream took 0.25664 seconds from start to end, of which 0.244707 seconds was spent between the three-way handshake and sending the HTTP GET request.

I have only run this once so far so these numbers don't really mean anything, but gives me a couple of things to look at further. The separate TCP connection for the favicon comes from the fact that we are fetching the favicon in Java rather than from Gecko, and overall may not really matter that much. The gap between connection setup and issuing the HTTP request, if it is reliably reproducible, is more troubling and will need to be investigated.

Ideally I would like to get a precise timestamp from when I hit enter on the keyboard, because it could just be that we do things in a different order, and so the overall time to render is the same even though the TCP time is different.

Comment 4

[William Lachance (:wlach)]

So I've been coming at this from a different direction and have been modifying Eideticker to collect information on which resources are requested when (it uses an internal web server which is very easy to instrument this way), and dumping it to a log.

I then ran it several times for a few sites that I've been using on the dashboard: CNN, taskjs, and Nightly on Firefox Beta (latest) and Chrome.

Results:

CNN:

http://people.mozilla.com/~wlachance/cnn-chrome-netstats-1.csv
http://people.mozilla.com/~wlachance/cnn-chrome-netstats-2.csv
http://people.mozilla.com/~wlachance/cnn-ffx-beta-netstats-1.csv
http://people.mozilla.com/~wlachance/cnn-ffx-beta-netstats-2.csv

Taskjs: 

http://people.mozilla.com/~wlachance/taskjs-chrome-netstats-1.csv
http://people.mozilla.com/~wlachance/taskjs-ffx-beta-netstats-1.csv

Nightly:

http://people.mozilla.com/~wlachance/nightly-chrome-netstats-1.csv
http://people.mozilla.com/~wlachance/nightly-ffx-beta-netstats-1.csv

Interpretation of results:

CNN: We seem to be competitive with Chrome at first, then (for some reason) seem to wait forever (1.5 seconds) to fetch the "/ep1/cnn/i.cdn.turner.com/cnn/.e/img/3.0/search/btn_search_hp_text.gif" resource. This is reproducible.

Taskjs: Again, we seem to be quite competitive at first, but then mysteriously wait 0.5 seconds to fetch various png resources required by the site's stylesheets.

Nightly: Again, a mysterious long wait (1.6 seconds) to fetch the second web font.

--

My feeling here is that we have some maximum # of open connections at a time, which is causing the download of some resources necessary to fully render the page to be unnecessarily delayed.

Comment 5

[Kartikaya Gupta (email:[e-mail])]

Interesting. Looking at the CNN pages, chrome is also downloading 24 more resources than we are. Four of those are apple-touch-icon files but the rest seem to be legitimate images in the content.

It might a good idea to do this with a simpler test page to get a baseline comparison. On CNN at least all of the resources are downloaded in a different order and it can be hard to compare without a deeper understanding of what's going on under the hood.

Comment 6

[William Lachance (:wlach)]

(In reply to Kartikaya Gupta (:kats) from comment #5)
> Interesting. Looking at the CNN pages, chrome is also downloading 24 more
> resources than we are. Four of those are apple-touch-icon files but the rest
> seem to be legitimate images in the content.

This might be a red herring -- I think the javascript CNN site might randomly cause a different ad to be loaded.

> It might a good idea to do this with a simpler test page to get a baseline
> comparison. On CNN at least all of the resources are downloaded in a
> different order and it can be hard to compare without a deeper understanding
> of what's going on under the hood.

Perhaps the taskjs page makes a good initial point of comparison? It's about the simplest non-trivial test case I can think of, and there also seems to be a real issue with it.

Comment 7

[Mark Finkle (:mfinkle) (use needinfo?)]

Just more data: I looked at http://browserscope.org to compare some network settings between browsers. I looked at two network settings: connections-per-hostname (cph) and max-connections mc).

      stock     chrome   firefox
cph:   6          6        6
mc:   32         32       15

I noticed in our prefs that we set mc=20, but max-connections-per-server=15, so I assume the test is hitting the "per-server" limit.

Our prefs: http://mxr.mozilla.org/mozilla-central/source/mobile/android/app/mobile.js#84

You could tweak these values and see what results you get.

Comment 8

[William Lachance (:wlach)]

(In reply to Mark Finkle (:mfinkle) from comment #7)
> Just more data: I looked at http://browserscope.org to compare some network
> settings between browsers. I looked at two network settings:
> connections-per-hostname (cph) and max-connections mc).
> 
>       stock     chrome   firefox
> cph:   6          6        6
> mc:   32         32       15
> 
> I noticed in our prefs that we set mc=20, but max-connections-per-server=15,
> so I assume the test is hitting the "per-server" limit.
> 
> Our prefs:
> http://mxr.mozilla.org/mozilla-central/source/mobile/android/app/mobile.js#84
> 
> You could tweak these values and see what results you get.

I tried setting max-connections-per-server to 32. No effect. I also tried setting max-persistent-connections-per-server to 16, also no effect.

Perhaps my initial hypothesis was incorrect.

Comment 9

[Kartikaya Gupta (email:[e-mail])]

This isn't specific to a FF/Chrome comparison, but for future reference I filed a couple of bugs for things I found while investigating tcpdump logs from FF on Android:

bug 771507 - When fetching pages sometimes TCP connections are opened and immediately closed with no data transferred
bug 773778 - Allow unused TCP connections to a domain to be reused for a different domain that resolves to the same IP address

Comment 10

[Brad Lassey [:blassey] (use needinfo?)]

Doug, can we get a new assignee?

Comment 11

[Doug Turner (:dougt)]

they are pre-fetching, pre-rending, and using flywheel for http traffic.  There is little further investigation to be done.  We just need to play catch-up fast or we won't have a compelling experience against the default install.

Comment 12

[Brad Lassey [:blassey] (use needinfo?)]

filter on [mass-p5]

Comment 13

[Brad Lassey [:blassey] (use needinfo?)]

(In reply to Doug Turner (:dougt) from comment #11)
> they are pre-fetching, pre-rending, and using flywheel for http traffic. 
> There is little further investigation to be done.  We just need to play
> catch-up fast or we won't have a compelling experience against the default
> install.

Doug, that doesn't explain bug 857359 which is a first run page load over a local net so there is no opportunity for pre-fetching or or pre-rendering. Can we get an assignee here? (or in the other bug if you think that's the more appropriate one)

Comment 14

[Doug Turner (:dougt)]

jason, can you put this in the backlog of things to look into?

Comment 15

[Doug Turner (:dougt)]

jduell?

Comment 16

[Jason Duell]

At this point all the actionable work here is hanging off of bug 857359, which Honza and mcmanus have commented on and which we've split into concrete bugs.

I thought about throwing our summer intern onto this, but we don't have any easy way to compare Chrome + Fennec other than eideticker, and it sounds like :wlach has got a test setup for that already. If eideticker (or anectodal user experience) shows more perf issues once we've fixed bug 857359, let's open a new bug. I'm closing this one.

If I'm missing something else we could be doing here I'm happy to discuss!