544994 - Create an artifically high-latency, low-bandwidth network for regression tests of speculative parser performance.

Status: RESOLVED WONTFIX

(Testing :: General, enhancement)

Description

[Ben Newman (:bnewman) (:benjamn)]

Speculative parsing (bug 482919) allows remote href and src attributes to be fetched before the HTML parser is absolutely sure that those remote resources will be needed.

This pre-fetching is intuitively one of the most significant measures we can take to reduce page load time, but it could theoretically regress load time if too many unneeded resources are requested, or if the gains are outweighed by thread switching overhead.  Furthermore, the implementation of speculative parsing is complicated enough that a high-level analysis of its impact is necessary to prevent bugs.

Since we'd be testing performance, having multiple independent machines running the tests would be good for accuracy, but I think running the entire Talos test suite would be overkill.  I plan to work on compiling a smaller, representative sampling of pages (in a follow-up bug).

With this bug, I'd like to open up a discussion of what kind of hardware would be required, and what sort of time-frame we'd be looking at for getting some version of this idea implemented.

Comment 1

[Ben Newman (:bnewman) (:benjamn)]

Just in case it wasn't clear above, the impact of speculative parsing is greatest (easiest to measure) when the network is slow, and resources that appear earlier on a page delay requests for resources that appear later.

Comment 2

[Ben Newman (:bnewman) (:benjamn)]

cjones notes that qemu may have support for altering network speed, so perhaps this is something that could be implemented at the VM level rather than at the server level?

Comment 3

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

I used this feature 

http://developer.android.com/guide/developing/tools/emulator.html#netdelay

when playing around with the android SDK a couple of years ago.  I don't know how it's implemented.

Comment 4

[Steve Fink [:sfink] [:s:]]

On Linux, this can be done on the "real" network interfaces with traffic control and queuing disciplines, in particular the 'netem' qdisc (NETwork EMulator). It's a bit of a pain to set up and configure, especially for incoming traffic. (When I used it last, I hit a kernel bug that prevented the incoming side from working at all, but that should be fixed in newer kernels.)

The main command you would use is 'tc' in the iproute2 package. See eg http://www.linuxfoundation.org/collaborate/workgroups/networking/netem

Comment 5

[Mark Côté [:mcote]]

Deprecating Testing :: Infrastructure.  Project never went anywhere, and speculative parsing landed a long time ago, so closing out.