1091120 - investigate forget button performance with large amounts of history

Status: NEW

(Firefox :: General, defect)

Description

[:Gavin Sharp [email: [e-mail]]]

History clearing is on the main thread, unfortunately (bug 871908). We're in the process of fixing that (bug 1089695), but in the mean time, it would be good to try to measure how bad this is in some realistic scenarios.

I'm thinking we could provide QA with a test snippet of code to run in the Browser Shell to add many history entries within one of the timespans that the forget button support (the past two hours is probably ideal). Ideally designed to increase the size of the DB as much as possible, but in a somewhat realistic way that might map to actual browsing. Then have them test the impact of that on the button's responsiveness in various configurations (SSD, old crappy spinning disk, etc.).

Comment 1

[:Gavin Sharp [email: [e-mail]]]

(In reply to :Gavin Sharp [email: [e-mail]] from comment #0)
> I'm thinking we could provide QA with a test snippet of code to run in the
> Browser Shell to add many history entries within one of the timespans that
> the forget button support (the past two hours is probably ideal). Ideally
> designed to increase the size of the DB as much as possible, but in a
> somewhat realistic way that might map to actual browsing.

Marco, could you suggest a code snippet to use for this?

Comment 2

[Marco Bonardo [:mak]]

Attachment: genPlaces.txt

Something like this should do. it's very hard to generate real-like data though.

This might help investigation, you can play with PAGES_COUNT and VISITS_PER_PAGE, I'd suggest to keep PAGES_COUNT lower than 500 and rather repeat multiple times to generate many.
Profiles in the wild have around 70K pages, unfortunately we don't have telemetry for avg number of visits per page, so you can only guess. We should probably file a bug to get telemetry for that data.

Comment 3

[:Gavin Sharp [email: [e-mail]]]

Tracy, can you or someone else with access to a variety of test machines run that code in the Browser Console (after setting devtools.chrome.enabled to true to expose the JS input) to generate a big DB, and see what effect that has on "forget" performance?

Comment 4

[Tracy Walker [:tracy]]

We'll take a look at it today and overnight for SV team.  I should have some initial findings today on Mac and Win8.

Comment 5

[Tracy Walker [:tracy]]

Marco,  I get...  'SyntaxError: illegal character' trying to run it in browser console.

Comment 6

[[SV Manager] Florin Mezei, QA (:FlorinMezei)]

Attachment: genPlaces_24hrs.txt

The attached script works for me to generate the History over the past 24 hours (so I can test with the Forget the last 24 hours option).

Comment 7

[Tracy Walker [:tracy]]

With Florins version of the script, I was able to populate with over 100,000 records.  Forget button used for past 24 hours.  The existing window closed quickly then opened the new window.  But it took over a minute for the finished dialog to appear and the browser was 'beach balling' unresponsive during that time.  On Mac 10.9

Comment 8

[[SV Manager] Florin Mezei, QA (:FlorinMezei)]

I did some performance testing today using the following scenario:
1. In Windows 7 x64, I ran the attached script in a modified form 8 times to generate records over the past 5 days. I ended up with a places.sqlite DB containing:
moz_historyvisits = 39620 records
moz_places = 2012 records
2. I used the Forget button to clear all records from the last 24 hours. I ended up with a places.sqlite DB containing:
moz_historyvisits = 31706 records (all older than 24 hours)
moz_places = 2008 records (all older than 24 hours)
3. I ran the attached script in a modified form 14 times to generate records over the past 6 hours. I ended up with a places.sqlite DB containing:
moz_historyvisits = 100965 records (~70k records from the previous 6 hours)
moz_places = 5508 records (~3000 records from the previous 6 hours)
4. I then used this DB for testing by placing it in the profile folder before opening Firefox. On Windows it's enough to overwrite it and use the same profile, on Mac and Linux I had to place it in the folder of a fresh profile then open Firefox.
5. Using a simple stopwatch I made some measurements to determine how much time there is between cllicking the Forget button, until the confirmation displays.

Results:
Win 7 x64         - AVG=7.68s, MIN=6.7s, MAX=9.4s, MEDIAN=7.5s
Win 8.1 x64       - AVG=8.16s, MIN=7.3s, MAX=9.3s, MEDIAN=8.0s
Win 8 x64         - AVG=6.06s, MIN=5.6s, MAX=7.3s, MEDIAN=5.7s
Win Vista x86     - AVG=12.24s, MIN=8.1s, MAX=19.5s, MEDIAN=10.6s
Ubuntu 14.04 x64  - AVG=9.16s, MIN=8.9s, MAX=9.4s, MEDIAN=9.1s
Ubuntu 12.04 x86  - AVG=9.78s, MIN=8.8s, MAX=11s, MEDIAN=9.6s

===== MAC =====
Mac OS X 10.10    - AVG=6.5s, MIN=5.8s, MAX=7.2s, MEDIAN=6.2s
Mac OS X  10.8.5  - AVG=6.22s, MIN=5.9s, MAX=6.8s, MEDIAN=6.1s

Detailed results are at: https://etherpad.mozilla.org/Bug1091120.

I think this is an extreme case that's unlikely to happen, and for that the numbers I've got are actually not bad. If you have any thoughts on real life scenarios (specify number of records over past 24 hours + number of older records) just let me know and I can setup some DBs for that and test.

All testing was done with Firefox 33.1 build 2.

Comment 9

[Marco Bonardo [:mak]]

(In reply to [:tracy] Tracy Walker - QA Mentor from comment #5)
> Marco,  I get...  'SyntaxError: illegal character' trying to run it in
> browser console.

ah sorry, probably you tested in a version of Firefox without template strings.

Comment 10

[Marco Bonardo [:mak]]

so basically, what we should look at here is the difference between MIN (few visits) and MAX (many visits), right?
The WinVista result is puzzling... there must be some very slow IO there.
In the other cases doesn't sound like too bad.

Though I think it would be useful if you could also directly measure the cost of the single call to history clear() here:

http://mxr.mozilla.org/mozilla-central/source/browser/base/content/sanitize.js#244

cause now it's still unclear how much of those 6-7 seconds are due to history.
You can probably use windows.performance.now() there and do a simple diff.

Comment 11

[[SV Manager] Florin Mezei, QA (:FlorinMezei)]

(In reply to Marco Bonardo [::mak] (needinfo? me) from comment #10)
> so basically, what we should look at here is the difference between MIN (few
> visits) and MAX (many visits), right?
> The WinVista result is puzzling... there must be some very slow IO there.
> In the other cases doesn't sound like too bad.

Vista was just slow in everything so I'm guessing it was the main reason for the slowness in Firefox. I'm thinking it might not be very relevant in analyzing the results, and it just tells us that it's likely to be much slower than any other OS.