Last Comment Bug 611837 - Meta: Don't fsync on startup
: Meta: Don't fsync on startup
Status: ASSIGNED
[ts] [Snappy:p2]
: meta, perf
Product: Core
Classification: Components
Component: XPCOM (show other bugs)
: unspecified
: x86 Linux
: -- normal with 11 votes (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
Mentors:
Depends on: 609310 611839 611874 688193 611840 611843 686706 689955 691268
Blocks: 447581 572459
  Show dependency treegraph
 
Reported: 2010-11-12 13:57 PST by (dormant account)
Modified: 2014-06-29 18:35 PDT (History)
18 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
Notes regarding calls to fsync during startup with an empty profile (109.42 KB, text/plain)
2011-09-20 09:11 PDT, David Rajchenbach-Teller [:Yoric] (please use "needinfo")
no flags Details
Tracing fsync during start-up with a non-empty profile. (10.29 KB, text/plain)
2011-09-21 03:27 PDT, David Rajchenbach-Teller [:Yoric] (please use "needinfo")
no flags Details
A list of all SQlite requests executed at startup, based on a non-trivial profile (9.67 KB, text/plain)
2011-10-07 02:41 PDT, David Rajchenbach-Teller [:Yoric] (please use "needinfo")
no flags Details
A list of all SQlite files opened at startup, based on a non-trivial profile on Mac (751 bytes, text/plain)
2011-10-07 03:23 PDT, David Rajchenbach-Teller [:Yoric] (please use "needinfo")
no flags Details
A list of all SQlite requests executed at startup, sorted by database, based on a non-trivial profile (118.84 KB, text/plain)
2011-10-07 05:28 PDT, David Rajchenbach-Teller [:Yoric] (please use "needinfo")
no flags Details
The extension used to parse the output (74.70 KB, application/x-xpinstall)
2011-10-07 05:32 PDT, David Rajchenbach-Teller [:Yoric] (please use "needinfo")
no flags Details

Description (dormant account) 2010-11-12 13:57:30 PST
This is to keep track of startup-killing fsyncers
Comment 1 (dormant account) 2010-11-12 15:55:42 PST
To clarify, at the moment our first startup consists of ~100 fsyncs. I'd like to get that down to 0.
It's really stupid to fsync on first startup since the files being created can not possibly contain useful user data.
Comment 2 David Rajchenbach-Teller [:Yoric] (please use "needinfo") 2011-09-20 09:10:26 PDT
I have traced fsync during first startup on MacOS X. See attachment for details.

A few key informations:
- 89 calls to fsync;
- 64 threads created;
- about 12 threads still active at the end of startup (sorry, I forgot to write down the actual number).
Comment 3 David Rajchenbach-Teller [:Yoric] (please use "needinfo") 2011-09-20 09:11:26 PDT
Created attachment 561211 [details]
Notes regarding calls to fsync during startup with an empty profile
Comment 4 David Rajchenbach-Teller [:Yoric] (please use "needinfo") 2011-09-20 09:34:42 PDT
My tracing session left me thinking that most of the [fsync]s are due to creating empty databases. If so, we could simply ship FF with a set of empty databases and copy the files, which should be much faster.
Comment 5 David Rajchenbach-Teller [:Yoric] (please use "needinfo") 2011-09-21 03:23:49 PDT
Traced fsync during second startup on MacOS X. See attachment for details.

A few key informations:
- 2 calls to fsync on critical path to startup;
- 5 calls to fsync backgrounded that may or may not occur during startup;
- about 30 threads created - actual count seems to vary between runs;
- about 19 threads still active at the end of startup - actual count seems to vary between runs.
Comment 6 David Rajchenbach-Teller [:Yoric] (please use "needinfo") 2011-09-21 03:27:10 PDT
Created attachment 561422 [details]
Tracing fsync during start-up with a non-empty profile.
Comment 7 David Rajchenbach-Teller [:Yoric] (please use "needinfo") 2011-10-07 02:41:01 PDT
Created attachment 565470 [details]
A list of all SQlite requests executed at startup, based on a non-trivial profile

For reference, here's a list of SQLite requests at startup, taken from my active profile, on Mac.
Comment 8 David Rajchenbach-Teller [:Yoric] (please use "needinfo") 2011-10-07 03:23:59 PDT
Created attachment 565482 [details]
A list of all SQlite files opened at startup, based on a non-trivial profile on Mac
Comment 9 David Rajchenbach-Teller [:Yoric] (please use "needinfo") 2011-10-07 05:28:26 PDT
Created attachment 565505 [details]
A list of all SQlite requests executed at startup, sorted by database, based on a non-trivial profile

Here's a refined list of requests executed at startup. Thanks, mak, for the suggestions.
Comment 10 David Rajchenbach-Teller [:Yoric] (please use "needinfo") 2011-10-07 05:32:58 PDT
Created attachment 565507 [details]
The extension used to parse the output

In case anybody needs it, here's the code I used to parse the output of mozStorage:5.

Note You need to log in before you can comment on or make changes to this bug.