747937 - Too much latency in mozhttpd when serving up a site with a bunch of images

Status: RESOLVED FIXED

(Testing Graveyard :: Eideticker, defect)

Description

[William Lachance (:wlach)]

In serving up imgur.com, it seems like the images take a long time to download, which totally messes up the test (i.e. the images are completely downloaded only after the panning is complete).

Possible solutions here:

1. Figure out what's causing the latency and fix it (theory: having mozhttpd run inside the eideticker process is slowing things down)
2. Delay the capture even more.

(2) would make the test take way longer (especially accounting for the additional video that would need to be encoded) and may not even be reliable (hard to set thresholds for this sort of thing). Let's try (1) first.

Comment 1

[William Lachance (:wlach)]

(In reply to William Lachance (:wlach) from comment #0)
> 1. Figure out what's causing the latency and fix it (theory: having mozhttpd
> run inside the eideticker process is slowing things down)

:chmanchester did some raw profiling of imgur on mozhttpd and apache and found little difference between the two:

mozhttpd:

============================================================                                                               
    Page                                    raw                                                                            
  0                                         1158,802,194,150,186,144,139,136,180,136                                       
  1                                         576,392,187,144,142,144,141,187,140,147                                        
  2                                         494,484,186,155,192,191,202,186,142,149                                        
  3                                         353,556,148,149,141,152,143,142,141,142                                        
  4                                         527,153,141,154,142,155,142,142,195,150                                        
Cycle collection: 1167                                                                                                     
============================================================

Doing the same run from an apache 2.2 instance yields:

============================================================                                                               
    Page                                    raw                                                                            
  0                                         1789,504,142,179,178,178,179,140,179,179                                       
  1                                         594,213,182,185,184,189,185,185,185,185                                        
  2                                         572,191,182,187,184,184,192,186,185,184                                        
  3                                         488,198,192,183,184,192,184,194,184,194                                        
  4                                         359,137,186,183,185,183,184,185,194,141                                        
Cycle collection: 1155                                                                                         ============================================================

Anecdotally, when I load the imgur site on my phone without an eideticker capture occurring, everything loads very quickly. So it does seem to be either (1) the capture harness or (2) the way the capture harness is being called that's slowing things down.

Comment 2

[William Lachance (:wlach)]

So it turns out that the problem was that mozhttpd was competing with the part of the code that was grabbing the current frame from the capture subprocess. Moving the latter into a subprocess (and keeping the frame number in shared memory) seems to have fixed the issue:

https://github.com/mozilla/eideticker/commit/02576fea318a0dde33135e31a9c027c199c4196a