Closed Bug 627591 Opened 14 years ago Closed 14 years ago

preload dlls on windows

Categories

(Core :: XPCOM, defect)

Product:
Core
Component:
XPCOM
Platform:
x86
All
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
blocking2.0 --- -

People

(Reporter: taras.mozilla, Assigned: taras.mozilla)

References

Details

(Keywords: perf, Whiteboard: [ts] )

Attachments

(2 files, 6 obsolete files)

style fixes
3.21 KB, patch
shaver
: review+
Details | Diff | Splinter Review
telemetry probes
2.64 KB, patch
Details | Diff | Splinter Review
Attached patch wip (obsolete) — — Splinter Review
We page in most of xul.dll and mozjs.dll. So i figured why not try lazy-linking libxul and preloading it. Turned out lazylinking didnt make any difference, but preloading seems to consistently shave up 2seconds of startup on my profile(40%). I'm going to doublecheck on my favourite user's slow computer. Here is the patch in meantime. Basic idea is that the sequential flag + bullshit read tricks windows into reading xul in 2mb chunks instead of stupid 32k(or smaller) ones. Have to do it this way because there is no fadvise() on Windows(that I know of) A big sequential read cuts down on a lot of seeks.
I love you a little.
Maybe a moderate amount. Fine.
Keywords: perf
Comment on attachment 505620 [details] [diff] [review] wip I just confirmed, this also works wonders on slow harddrives. Shaved around 50% off cold startup on my reference slow system. This patch works better than I expected. Not only does it trade random xul.dll io for faster sequential io. It also reduces the amount of seeks later on(ie no need to seek for xul bits), which significantly speeds up io on other files. There are some tricky refinements that can be done here, such as trying to count pagefaults in order to detect warm startup and checking the amount of ram available, but I think this patch as is is simple and low risk so the rest can be done as followups. @shaver, I guess there is a fine line between hatred and love :) https://bugzilla.mozilla.org/show_bug.cgi?id=554421#c11 was regarding a similar trick on linux(the improvement there wasn't as huge on linux as it is on windows)
Attachment #505620 - Flags: review?(benjamin)
Attachment #505620 - Flags: approval2.0?
This patch greatly reduced my cold startup time so requesting blocking. See below for the cold startup numbers with facebook and gmail app tabs loaded along with the about:home mozilla google search page loaded from previous session: Taras build with patch applied: main 5070 sessionRestored 7083 firstPaint 7239 main 5428 sessionRestored 6958 firstPaint 7067 main 5382 sessionRestored 6881 firstPaint 7021 Latest trunk: main 2294 sessionRestored 16179 firstPaint 16787 main 2325 firstPaint 16646 sessionRestored 17614 main 2777 firstPaint 18331 sessionRestored 19455
blocking2.0: --- → ?
Assignee: nobody → tglek
Comment on attachment 505620 [details] [diff] [review] wip Would it work better to preload separately in two threads? I'm of two minds: on the one hand you're making the drive seek more, given that we defragment on update now, but on the other you can sometimes win by keeping the I/O queue full so that the head can pick up data more often during its rotation. Maybe worth trying as an experiment later, though as I type this I think the first hand is the right hand. { /* nothing */ } instead of the hard-to-see semi, and let's land this bad boy for Monday's nightlies.
Attachment #505620 - Flags: review?(benjamin)
Attachment #505620 - Flags: review+
Attachment #505620 - Flags: approval2.0?
Attachment #505620 - Flags: approval2.0+
Actually there is one issue here, the preload happens in firefox and plugin-container. What's the best way to only run this for firefox? Is there a define or a global variable I can use?
I thought it was intentional to do it in plugin-container too, oh.
XRE_GetProcessType is probably what you want?
Comment on attachment 505620 [details] [diff] [review] wip Shaver's flirting and { /* nothing */ } got me into drive-by-nit mode. >+static void preload(const char *dll) { { on new line. >+ HANDLE fd = CreateFileA(dll, GENERIC_READ, FILE_SHARE_READ, >+ NULL, OPEN_EXISTING, FILE_FLAG_SEQUENTIAL_SCAN, NULL); >+ if (fd == INVALID_HANDLE_VALUE) >+ return; >+ >+ char buf[64*1024]; >+ DWORD dwBytesRead; >+ >+ while(ReadFile(fd, buf, sizeof(buf), &dwBytesRead, NULL) && dwBytesRead == sizeof(buf)); Rather than braces and /* nothing */, just .. continue; on its own line. >+ CreateThread( NULL, 0, preload_thread, NULL, 0, NULL); Leading space before first NULL. /be
Small issue: CreateThread is returning a handle that is not freed until the process dies. You should CloseHandle on it. (MSVS2008 at least should pick this up with code analysis)
Status: NEW → ASSIGNED
OS: Windows 7 → All
blocking2.0: ? → -
Comment 13 hidden, due to suspicious executable attachment.
While this isn't a blocker, it's approved and I would be sadfaces if we neglected to check it in and make go faster now please k thx bye.
Keywords: checkin-needed
Whiteboard: [ts] → [ts][can land]
Comment on attachment 505620 [details] [diff] [review] wip withdrawing review given the desire to not have it affect plugin-container, and I guess the thread handle leak too I suppose maybe.
Attachment #505620 - Flags: review+ → review-
Did I miss where it was explained why we don't want this to affect the plugin-container?
I don't think it was explained, but the way I undrestand it, plugin-container.exe is launched by firefox, and if firefox is already running, the dlls are already preloaded, so it would be counterproductive to "preload" them again
Couple of notes: If in ccomment #17 the review was revoked, shouldn't the keyword 'check-in needed' be removed ? In respect to comment #19, the plugin-container.exe is not launched when Firefox starts if you only open a blank tab. Plugin-container.exe only seems to start when there is say something on the page that uses a plugin, Flash of course being the predominate user. So, unless the dll's are loaded even though the 'container' is not running, I don't understand. If Firefox is 'already running' - then the whole preload stuff is moot anyhow, no?
oops, also the 'whiteboard' needs to be changed I would think.
Keywords: checkin-needed
Whiteboard: [ts][can land] → [ts]
What if the homepage contains plugins, would having the plugin-container included in the preload be a good thing in this instance?
No, once it's loaded it's off disk, and reading it again isn't going to help. Also, plugin-container uses much less of libxul than the main process. We should just do it for the main process for now.
Not to drive by, but as I understand from [1] and [2], the patch currently in the bug would help only on Vista and newer. Is that right? [1] http://infrequently.org/2011/01/on-the-care-and-feeding-of-spinning-disks/ [2] Search for function PreReadImage in http://src.chromium.org/viewvc/chrome/trunk/src/base/file_util_win.cc?view=markup
I may be totally wrong, but wouldn't the patch fail to preload if the current directory isn't the Firefox directory? This could cause problems when external applications launch Firefox, for example. The user would see inconsistent startup performance and would have no idea why.
How can I apply the patch in the attachement in order to test it ?
apply: https://developer.mozilla.org/en/Mercurial_FAQ compile: https://developer.mozilla.org/en/build_documentation BTW, you don't have to to this if there is a try build available. So, to the others: is there a try build somewhere with this patch?
(In reply to comment #30) > apply: https://developer.mozilla.org/en/Mercurial_FAQ > compile: https://developer.mozilla.org/en/build_documentation > > BTW, you don't have to to this if there is a try build available. So, to the > others: is there a try build somewhere with this patch? Nothing you'd want to use yet. Thanks for all of the drive-by reviews. Turned a large part of the speed up was depending on remnants in my builds from my previous experiment. I'm reworking the patch at the moment. I'll post a better one soon.
Attached patch new patch (obsolete) — — Splinter Review
This took me way too long. Spent half a week rebooting my windows laptop to find what kind of a change can cut our startup by 50% vs slowing us down by 50%(the difference in code is as little as one line of code). As I understand it, the key to good io perf is to win the race to open xul.dll with sequential_io flag and then preload the file. Normally the linker beats us to that and performance goes to the crapper. Took me a while to realize that linking against xpcom.dll was dragging in xul.dll early. I'm running this patch through try now. Hopefully I got my linker magic right. Oh and turned out preloading on a thread was a bad idea. That gets us into a nondeterministic race with the linker as to who opens xul.dll first. This patch also fixes relative paths, handle leakage. I haven't looked at how this behaves under xp. Had a hard enough time getting numbers out of my win7 box. My approach is different than that of chrome preload, they seem to prefer loading via sequential page faults, whereas file io seems to perform better for me.
Attachment #505620 - Attachment is obsolete: true
Attached patch ready to go (obsolete) — — Splinter Review
Got rid of debug leftovers. Tested, this is indeed 2x faster to start.
Attachment #507271 - Attachment is obsolete: true
Attachment #507298 - Flags: review?(shaver)
(In reply to comment #34) > here is a try build > http://stage.mozilla.org/pub/mozilla.org/firefox/tryserver-builds/tglek@mozilla.com-eb7ffaf8ef72/try-w32/firefox-4.0b10pre.en-US.win32.zip A total bust on XP. My numbers: UNPATCHED --------- 1. about:startup main: 1344 SessionRestored: 10813 firstPaint: 13719 2. manual timing (stop watch) chrome appears: 9 sec chrome painted: 13 sec PATCHED ------- 1. about:startup main: 2265 SessionRestored: 10093 firstPaint: 11453 2. manual timing (stop watch) chrome appears: 9 sec chrome painted: 12 sec
Just to be clear. In my previous post: Unpatched = January 26 nightly build Patched = Try server build of comment 34 All timings are for cold startup after system restarted and hard drive allowed to settle (no more indication of post windows startup activity).
I see no improvement :( Jan 26 build: main 3400 sessionRestored 7130 firstPaint 8534 Patched build: main 3448 firstPaint 7458 sessionRestored 7770
Csaba, what version of Windows are you running?
Just put it in nightly, so everybody can test.
(In reply to comment #38) > Csaba, what version of Windows are you running? Windows 7 x64.
(In reply to comment #34) > here is a try build > http://stage.mozilla.org/pub/mozilla.org/firefox/tryserver-builds/tglek@mozilla.com-eb7ffaf8ef72/try-w32/firefox-4.0b10pre.en-US.win32.zip windows xp sp3 / core 2 duo E6550, I don't see any different on my machine.
Windows 7 64bit, a bit slow HDD, latest nightly vs. try build Cold start (latest nightly): main 2937 sessionRestored 6421 firstPaint 6909 main 2284 sessionRestored 6239 firstPaint 6507 main 2193 sessionRestored 5071 firstPaint 5330 main 1662 sessionRestored 3499 firstPaint 3800 main 1989 sessionRestored 3393 firstPaint 3506 Cold start (try build): main 1700 firstPaint 5612 sessionRestored 9993 main 4015 sessionRestored 5189 firstPaint 5562 main 3274 sessionRestored 4154 firstPaint 4275 main 3457 sessionRestored 4367 firstPaint 4492 main 3575 sessionRestored 4771 firstPaint 4880 main 3400 sessionRestored 4780 firstPaint 4900 Warm start (nightly): main 10 sessionRestored 531 firstPaint 635 main 16 sessionRestored 498 firstPaint 589 main 16 sessionRestored 489 firstPaint 601 main 11 sessionRestored 503 firstPaint 616 main 33 sessionRestored 523 firstPaint 602 Warm start (try build): main 56 sessionRestored 1360 firstPaint 1473 main 58 sessionRestored 797 firstPaint 905 main 27 sessionRestored 495 firstPaint 596 main 37 sessionRestored 613 firstPaint 725 main 39 sessionRestored 494 firstPaint 600
Netbook - Win 7 Pro - tested 2x nightly main 2324 sessionRestored 19033 firstPaint 19392 build with patch main 3011 sessionRestored 14681 firstPaint 15086 (still very bad, Chrome's cold startup time is ~6s) Desktop - Win 7 Ultimate - tested 2x nightly main 2300 sessionRestored 3821 firstPaint 3911 build with patch main 2530 sessionRestored 4201 firstPaint 4281
I can confirm XP SP3 32bit - zip builds - old profile - no session restore - home=(default)about:home - 4 tests average - cold start: Nightly Build 20110127 main 1422 sessionRestored 18235 firstPaint 19094 Try Build main 2531 sessionRestored 18640 firstPaint 19125 No real Cold start improvement seen for WinXp or Win7/Vista users that I can see.
Comment on attachment 507298 [details] [diff] [review] ready to go Seems like we should only do this on Vista and up, like Chrome found too -- can you add that check, Taras? > // we want to use the DLL blocklist if possible > #define XRE_WANT_DLL_BLOCKLIST >+#define XRE_PRELOAD_XUL Comment here. >+static void preload(LPCWSTR dll) { >+ HANDLE fd = CreateFileW(dll, GENERIC_READ, FILE_SHARE_READ, >+ NULL, OPEN_EXISTING, FILE_FLAG_SEQUENTIAL_SCAN, NULL); >+ char buf[64*1024]; > >+ if (fd == INVALID_HANDLE_VALUE) { >+ return; >+ } >+ >+ DWORD dwBytesRead; >+ DWORD total = 0; >+ while(ReadFile(fd, buf, sizeof(buf), &dwBytesRead, NULL) >+ && dwBytesRead == sizeof(buf)) { >+ total+= dwBytesRead; >+ } Why do we want to stop on a short read? >+//Populate the windows page cache >+static DWORD WINAPI preload_libs(LPCWSTR appPath) >+{ >+ wchar_t msg[MAX_PATH]; >+ size_t pathlen = wcslen(appPath); >+ for(;pathlen && appPath[pathlen] != L'\\'; pathlen--); >+ if (pathlen) >+ pathlen++; >+ >+ if (pathlen + 10 > MAX_PATH) { >+ return 0; >+ } >+ >+ wcsncpy(msg, appPath, pathlen); >+ size_t remaining = sizeof(msg)/sizeof(msg[0]) - pathlen - 1; >+ wcsncpy(msg + pathlen, L"xul.dll", remaining); >+ preload(msg); >+ wcsncpy(msg + pathlen, L"mozjs.dll", remaining); >+ preload(msg); >+ return 0; >+} >+#endif > int wmain(int argc, WCHAR **argv) Blank line after #endif > { >+#ifdef XRE_PRELOAD_XUL >+ preload_libs(argv[0]); >+#endif >+ > #ifndef XRE_DONT_PROTECT_DLL_LOAD > mozilla::NS_SetDllDirectory(L""); > #endif
Attachment #507298 - Flags: review?(shaver) → review-
Thanks for the feedback. I'm guessing the optimization gains are cancelled out by losses due to directwrite/d2d/etc. Please try setting: gfx.direct2d.disabled -> true gfx.font_rendering.directwrite.enabled -> false in your normal build. then verify that when you go to about:support direct2d and directwrite are disabled. Then compare this build vs nigtlies in that setting. Sorry, this isn't a magic bullet that can fix all startup issues, this fix is aimed solely at eliminating the overhead of loading our libraries(which is the slowest part of startup unless we are running into d2d bugs) See bug 602792 for the directwrite-related slowdown info. I'm investigating other direct* slowdowns.
(In reply to comment #47) I've already logged the DW fontcache loading times as described in Bug 602792 Comment 110. I'm not hitting the DW fontcache slowdown. But i'll test with DW disabled to be sure.
(In reply to comment #47) > I'm guessing the optimization gains are cancelled out > by losses due to directwrite/d2d/etc. Not on XP. I've long had hardware acceleration disabled.
(In reply to comment #48) > (In reply to comment #47) > > I've already logged the DW fontcache loading times as described in Bug 602792 > Comment 110. I'm not hitting the DW fontcache slowdown. But i'll test with DW > disabled to be sure. Okay, here are the new numbers. I disabled D2D and DW, and waited much longer after reboot to let *every* I/O get finished. Cold startups, Win7 x64, 5 tabs. Jan 27: main 3135 sessionRestored 5523 firstPaint 6256 Patched: main 2761 sessionRestored 6038 firstPaint 6459
(In reply to comment #50) > (In reply to comment #48) > > (In reply to comment #47) > > > > I've already logged the DW fontcache loading times as described in Bug 602792 > > Comment 110. I'm not hitting the DW fontcache slowdown. But i'll test with DW > > disabled to be sure. > > Okay, here are the new numbers. I disabled D2D and DW, and waited much longer > after reboot to let *every* I/O get finished. > > Cold startups, Win7 x64, 5 tabs. > > Jan 27: > > main 3135 Your main numbers are beyond bad, esp for win7. It would be very helpful if you could do an xperf profile of your nightly cold startup and email it to me. See instructions in. http://blog.mozilla.com/tglek/2010/10/04/diagnosing-slow-startup/ Sounds like something is hurting your startup more than xul loading.
(In reply to comment #51) > (In reply to comment #50) > > (In reply to comment #48) > > > (In reply to comment #47) > > > > > > I've already logged the DW fontcache loading times as described in Bug 602792 > > > Comment 110. I'm not hitting the DW fontcache slowdown. But i'll test with DW > > > disabled to be sure. > > > > Okay, here are the new numbers. I disabled D2D and DW, and waited much longer > > after reboot to let *every* I/O get finished. > > > > Cold startups, Win7 x64, 5 tabs. > > > > Jan 27: > > > > main 3135 > > Your main numbers are beyond bad, esp for win7. It would be very helpful if you > could do an xperf profile of your nightly cold startup and email it to me. See > instructions in. > http://blog.mozilla.com/tglek/2010/10/04/diagnosing-slow-startup/ > > Sounds like something is hurting your startup more than xul loading. It would be interesting to see if the results from others using Win7/Vista following Csaba Kozák [:WonderCsabo] testing process with disabling DirectWrite or HWA with both todays Nightly Build and the TryServer Build. To just to rule out bug 602792 has the culprit of small improvements seen by bug 627591.
(In reply to comment #47) > Thanks for the feedback. I'm guessing the optimization gains are cancelled out > by losses due to directwrite/d2d/etc. > Please try setting: > gfx.direct2d.disabled -> true > gfx.font_rendering.directwrite.enabled -> false > in your normal build. > then verify that when you go to about:support direct2d and directwrite are > disabled. > > Then compare this build vs nigtlies in that setting. Sorry, this isn't a magic > bullet that can fix all startup issues, this fix is aimed solely at eliminating > the overhead of loading our libraries(which is the slowest part of startup > unless we are running into d2d bugs) > > See bug 602792 for the directwrite-related slowdown info. I'm investigating > other direct* slowdowns. tried again with these settings: patched: main 4844 sessionRestored 21453 firstPaint 22657 unpatched: （better） main 2281 sessionRestored 16906 firstPaint 18015 I don't know why my firefox is so slow..
I'm on Windows 7 64bit and my numbers are very close to my original comment 4. I think some of you might not be testing this right so here is detailed description on how to test this. 1) Install latest nightly and start it up a few times (Make sure about:startup is not loaded when you shutdown firefox...causes slow startup times for some reason) 2) Shutdown the system and then restart 3) Wait a minute, two or three minutes after windows starts then start Firefox (Time it so you start firefox at the same point for the test build) 4) load about:Startup and save the numbers somewhere 5) Get the build that Taras linked to in comment 34 6) Install it and start the build a few times with the same profile as before VERY IMPORTANT 7) repeat steps 2-4 (Again make sure you don't have about:Startup loaded before rebooting
I spent some time with Taras yesterday profiling my startup with xperf and testing his variants of the patch. I was consistently getting rather poor results (~3500-4500 for main) no matter from which build (nightly or the one from taras). Since the profile was showing a lot of loading from c:\$MFT I did some googling and found out that it's a Master File Table that stores data on all files on the hard drive (sec privs, settings, etc.). Which probably means that the more files we load, the more MFT is accessed. I did several things: * I checked my hard drive for errors - none found * I defragmented it * I installed Puran Defrag Free which has an on-boot defrag option that also defrags MFT table. * I booted from Samsung HD tools CD, checked it (one again, it's all fine), and tuned it to be less noisy. Since then I'm getting significantly better results. My average main dropped to ~1000-1500ms. My total Disk utilization time before the defrag (numbers are from launching try build): Service time: 4702288 Size: 18524672 After: Service time: 3354041 Size: 14980608 Chrome Canary build: Service time: 2897334 Size: 11442176 So I do see a major improvement and the perceived startup time of chrome and fx is now very close. I'll try to compare 5 cold starts of try vs nightly now. Also, during testing, we had a problem to trigger readahead and it only started working after Taras suggested me to delete data from c:\windows\prefetch.
I did another round of tests. Windows 7 64bit, Direct2d turned on (yeah, forgot to turn it off). I decided to follow the path of prefetch and test both builds with and without prefetch. First scenario (removed prefetch) means that I deleted a content of the c:\windows\prefetch directory before each reboot. In the second scenario, I did not touch the prefetch directory at all. Normal nightly: Prefetch deleted: main 729 sessionRestored 3398 firstPaint 4091 main 754 sessionRestored 3476 firstPaint 4127 main 680 sessionRestored 3331 firstPaint 3946 Prefetch preserved: main 1476 sessionRestored 2063 firstPaint 2207 main 1830 sessionRestored 2390 firstPaint 2536 main 1577 sessionRestored 2162 firstPaint 2297 =============== Latest build from Taras (from his people.mozilla.org directory): Prefetch deleted: main 573 sessionRestored 2950 firstPaint 3315 main 644 firstPaint 3212 sessionRestored 3245 main 680 firstPaint 3183 sessionRestored 3224 main 619 firstPaint 4123 sessionRestored 4165 Prefetch preserved: main 2090 sessionRestored 3264 firstPaint 3411 main 2257 sessionRestored 2928 firstPaint 3071 main 2336 sessionRestored 3032 firstPaint 3175
So what's going on with this, is it still the plan to get this working for the majority and get it in?
(In reply to comment #57) > So what's going on with this, is it still the plan to get this working for the > majority and get it in? Good question. I'm a bit stuck. Been analyzing user profiles for the past week. Seems that this is a significant win for people with non-broken harddrives. Ie if a harddrive can read at > 20mb/s(most modern non-broken drives should start at 50mb/s), then this is a win. Otherwise it is a regression. Seems like the easiest thing to do is to add a commandline -preload option and have users opt-in to that, but that clearly isn't the correct way to go. I could also benchmark startup times and measure preloading throughput, but then that data has to be stored somewhere and there is a risk of measuring wrong (ie mistaking 1 case of slow startup due to an abnormally high disk load). There is also the option of landing this as is and hoping that more users benefit than those who regress from this. But this has the downside of increasing the rift between well-performing firefox and slow firefox. Perhaps the best option would be an escape hatch that measures amount of time spent in the preload() and aborts after a second or two.
(In reply to comment #58) > I could also benchmark startup times and measure preloading throughput, but > then that data has to be stored somewhere and there is a risk of measuring > wrong (ie mistaking 1 case of slow startup due to an abnormally high disk > load). As unhelpful as the Windows Experience Index the rest of the time, how about utilising it in this instance, on just the first run of Fx4, to set a preload=true flag for subsequent instances, if say the primary hard disc score was above whatever value equates to 20mb/s ? It would avoid the issues of Fx trying to benchmark on first run (which as you say, might lead to a false positive if disk load was abnormally high at that moment), as well as the complexities of trying to come up with an accurate benchmarking system. If no WEI scores had been calculated yet, then the true/false flag could say just be rechecked in X days time, once the user has run the OS long enough for the WEI test to run. Obviously the WEI score system only exists for Vista/7 - but if Mike's assessment in comment 46 is correct, then this patch doesn't benefit XP anyway - so the lack of WEI wouldn't matter. Anyway, hope you don't mind the drive-by comments - the above is just something that occurred to me and thought it wouldn't hurt to mention...
The windows experience index crossed my mind. Problem is that at least on gandalf's system above the index is out of date and the harddrive performed significantly worse than indicated. I'll experiment with a few solutions, but given the complexity of heuristics mentioned so far, this will take at least another week to figure out. I checked on XP. The speed doubles there too(from 10s to 5s on Kurt's system). Chrome's use of pagefaults to preload stuff might be why our approaches differ on XP.
Another look at xperf traces (and gandalf's startup measurements above) suggest a bad interaction with windows prefetch. In Csaba's profile a full second is spent on IO before my preload code even runs. So I might just be able to disable preload if prefetch took forever.
Why don't add an option at about:config? this way everybody can switch it on or off at will, no way you can satisfy anybody due to the complexity, even eventually you find a way you think is perfect, you might still need add this option, somenoe at some times just want to turn it off, you never know.
It may happens before FF read prefs, but you can mark it at startupCache or somewhere when people change it at about:config. it's better than -preload because everybody knows how to customize at about:config, it's the first thing you learn as a FF user.
What does the "main" property mean?
.main marks when the platform-independent bits of firefox start running. It's useful for developers to gauge library/other-platform-overhead. Would be useful to have people compare startup of http://people.mozilla.com/~tglek/startup/firefox-4.0b10pre.en-US.win32.zip and http://people.mozilla.com/~tglek/startup/vanilla_ff.zip Also tell me what the messagebox in the first build says. I added a heuristic to disable preload if we seem to be executing with hot cache or if Windows prefetch has spent a long time caching before the preload code runs.
latest nightly: main 680 sessionRestored 8461 firstPaint 8681 latest nightly 2.test: main 480 sessionRestored 8731 firstPaint 9001 build with patch: main 680 sessionRestored 4431 firstPaint 4571 build with patch 2.test: main 990 sessionRestored 4521 firstPaint 4681 D2D/DW/D3D10 - enabled looks good :)
Netbook - Win 7 Pro nightly: main 1482 firstPaint 22107 sessionRestored 25508 build with patch: main 5007 sessionRestored 13822 firstPaint 14165 D2D/DW/D3D - disabled
Thanks for the tests, keep it up please. I need more stats. It's not clear what people are comparing. I specifically need http://people.mozilla.com/~tglek/startup/vanilla_ff.zip compared to http://people.mozilla.com/~tglek/startup/firefox-4.0b10pre.en-US.win32.zip I would also like to knows what the messagebox on startup of firefox-4.0b10pre.en-US.win32.zip says.
Notebook, Win 7 Pro 32bit, D2D/DW/D3D - disabled, clean new profile with patch(firefox-4.0b10pre.en-US.win32.zip): main 2855 sessionRestored 5289 firstPaint 5508 with patch2: main 3651 firstPaint 6538 sessionRestored 6647 with patch3: main 3994 firstPaint 6007 sessionRestored 6085 ------- vanilla: main 1669 firstPaint 7692 sessionRestored 7785 vanilla2: main 2637 firstPaint 7989 sessionRestored 8082 vanilla2: main 3651 firstPaint 6818 sessionRestored 6912 With patch it's better but not significantly... err, I see no messagebox on startup of firefox-4.0b10pre.en-US.win32.zip
Eddward, looks like firefox-4.0b10pre.en-US.win32.zip was a slightly stale version. Fixed that now, sorry & thanks for catching the problem. Can you retest it?
Okay, so this is now correct link http://people.mozilla.com/~tglek/startup/firefox-4.0b11pre.en-US.win32.zip yes? results: 1. cold startup 2900ms since start. Things seems to be cached. main 3151 sessionRestored 5991 firstPaint 6428 2. cold startup 2900ms since start. Things seems to be cached. main 3260 firstPaint 6958 sessionRestored 7083
Ok thanks Eddward, looks like on your system windows prefetch does a lot of io similarly to gandalf's. Anyone else?
Results from another system (desktop) Win 7 Ultimate 32bit D2D/DW/D3D10 - enabled. firefox-4.0b11pre.en-US.win32.zip (with patch) 1. cold startup 800ms since start. Optimizing main 1280 sessionRestored 2551 firstPaint 2741 2. cold startup 1400ms since start. Things seems to be cached. main 1540 sessionRestored 2491 firstPaint 2611 3. cold startup 1300ms since start. Things seems to be cached. main 1430 sessionRestored 2811 firstPaint 2961 vanilla: results very similar; first paint ~2600-2700ms
System: laptop Vista Home 32bit - D2D/DW blocked/disabled Test: 3 runs / only tab about:home / working offline Vanilla version: main 46 sessionRestored 2060 firstPaint 2153 main 62 sessionRestored 2060 firstPaint 2154 main 156 sessionRestored 2201 firstPaint 2294 Patched version firefox-4.0b11pre.en-US.win32: 0 ms since start. Things seem to be cached main 94 sessionRestored 2136 firstPaint 2370 0 ms since start. Things seem to be cached main 63 sessionRestored 2123 firstPaint 2388 0 ms since start. Things seem to be cached main 125 sessionRestored 2190 firstPaint 2439
Sorry for delays. I was triplechecking that prefetch was screwing me. Turns out windows prefetch spends a lot of time prefetching before executable code is even run making any optimization within firefox impossible. Time to give up on approach in the patches. Instead I moved prefetching into a separate executable. Please copy http://people.mozilla.com/~tglek/startup/wrapper.exe into your firefox directory and try running firefox by launching wrapper.exe. Use any nightly build to test. i need about:startup times after reboot(as before) + the message that is shown after firefox exits. I also need about:startup numbers when running firefox.exe directly. Here I see 1.5-2x speeds ups without any/much prefetch slowdown. Would like to see that confirmed.
I was hoping to help test this, but about:startup doesn't exist in this nightly build. When I run your wrapper and let Minefield startup and then exit it, I get 100ms to preload. This is w/o rebooting the computer, which I can't do right now.
(In reply to comment #76) > I was hoping to help test this, but about:startup doesn't exist in this nightly > build. When I run your wrapper and let Minefield startup and then exit it, I > get 100ms to preload. This is w/o rebooting the computer, which I can't do > right now. http://glandium.org/blog/?p=1575 for about:startup extension. I need people to reboot the computer for this.
Okay, here we go... *rebooted* Ran w/ the wrapper: main 3109 sessionRestored 14469 firstPaint 15360 700ms to preload *rebooted again* Ran w/o the wrapper: main 3203 sessionRestored 14704 firstPaint 17063 System specs: Intel i7-920 @ 4.2ghz, 6GB DDR3 @ 1600mhz, Gigabyte X58A-UD7 motherboard, 4x250GB Western Digital Enterprise drives in RAID0+1, etc etc. Build I'm using is built from http://hg.mozilla.org/mozilla-central/rev/fd0817e454fe
Brian, that's horrible given your specs. Can you try deleting files(either all of them or just wrapper* and firefox*) from c:\windows\prefetch and rebooting again?
Ok, deleted the contents of the prefetch directory. Also, those numbers in Comment 78 reflect Minefield loading 4 app tabs and another 3 or 4 regular tabs. These numbers were run after I rebooted, with the wrapper, and only opening about:home. main 523 sessionRestored 15173 firstPaint 16360 500ms to preload
(In reply to comment #80) Brian, your startup is the worst I've ever seen on a high power system. Would you mind rebooting and then recording an xperf trace? See http://blog.mozilla.com/tglek/2010/10/04/diagnosing-slow-startup/ email me a a link to it. Looks to me like wrapper.exe is working great on your system(ie main time is decreased correctly), but the rest of your disk IO is killing startup.
Okay, I disabled direct*, rebooted windows, disabled the realtime scanner on my AV and here is what I get: main 2031 sessionRestored 6016 firstPaint 6298 800ms to preload
Direct* still disabled, av scanner is enabled this time, no wrapper: main 6515 sessionRestored 9970 firstPaint 10673
(In reply to comment #83) > Direct* still disabled, av scanner is enabled this time, no wrapper: > > main 6515 > sessionRestored 9970 > firstPaint 10673 cool so looks like wrapper shaved 4seconds here. Not sure things are still slow with directwrite off. Perhaps a defrag will take care of that.
Attached file wrapper.exe source (obsolete) —
source for wrapper.exe
Attachment #507298 - Attachment is obsolete: true
Ok, just to make sure there were no filesystem errors, I rebooted, forced a chkdsk on C:, ran a boot time defrag and then boot into windows and ran the wrapper again and got this: main 4593 sessionRestored 8469 firstPaint 8579 900 ms to preload Direct* is still disabled, AV was enabled.
(In reply to comment #80) > Ok, deleted the contents of the prefetch directory. Also, those numbers in > Comment 78 reflect Minefield loading 4 app tabs and another 3 or 4 regular > tabs. > > These numbers were run after I rebooted, with the wrapper, and only opening > about:home. > > main 523 > sessionRestored 15173 > firstPaint 16360 > > 500ms to preload Brian, with Direct* enabled, could you attach the info below for your cold startup times? 1. DirectWrite version and font cache size (see about:support) 2. Output of enabling the startup logging described in bug 602792, comment 110 I'm guessing your font cache file is fairly large, I would also suggest testing again after deleting that file. See instructions on bug 602792, comment 103.
I haven't had a chance to get to the logging portion yet, I apologize, however, here is the other info from about:support DirectWrite Enabled true (6.1.7601.17514, font cache 8.05 MB)
Here is the fontlog.txt: 0[82d140]: InitFontList 0[82d140]: Start: 02/10/2011 03:28:53 0[82d140]: Uptime: 30882.671 s 0[82d140]: dwrite version: 6.1.7601.17514 0[82d140]: Total time in InitFontList: 0.018 ms 0[82d140]: --- gfxPlatformFontList init: 0.006 ms 0[82d140]: --- GdiInterop object: 0.012 ms 0[82d140]: DelayedInitFontList 0[82d140]: Start: 02/10/2011 03:28:54 0[82d140]: Uptime: 30882.906 s 0[82d140]: dwrite version: 6.1.7601.17514 0[82d140]: Total time in DelayedInitFontList: 0.960 ms (families: 196, gdi table access) 0[82d140]: --- GetSystemFontCollection: 0.036 ms 0[82d140]: --- iterate over families: 0.923 ms I didn't reboot before running this. If you want me to do that, I will, and then I will also delete the font cache and retest the other scores as well if need be.
And here is the fontlog.txt after removing the fontcache files: 0[72d140]: InitFontList 0[72d140]: Start: 02/10/2011 03:37:54 0[72d140]: Uptime: 31423.343 s 0[72d140]: dwrite version: 6.1.7601.17514 0[72d140]: Total time in InitFontList: 0.019 ms 0[72d140]: --- gfxPlatformFontList init: 0.006 ms 0[72d140]: --- GdiInterop object: 0.013 ms 0[72d140]: DelayedInitFontList 0[72d140]: Start: 02/10/2011 03:37:54 0[72d140]: Uptime: 31423.796 s 0[72d140]: dwrite version: 6.1.7601.17514 0[72d140]: Total time in DelayedInitFontList: 0.956 ms (families: 196, gdi table access) 0[72d140]: --- GetSystemFontCollection: 0.015 ms 0[72d140]: --- iterate over families: 0.941 ms And DirectWrite Enabledtrue (6.1.7601.17514, font cache n/a)
Unfortunately i can no longer test with my laptop, but here are the numbers with my PC (Win7 x64, AMD Phenom X3 8450, 4GB DDR2 RAM, 5.7 WXPI for HDD): Cold startup times with 2011-02-11 build, D2D DW off, 2 tabs restored w/out wrapper main 1232 firstPaint 13027 sessionRestored 14509 w/ wrapper main 1107 firstPaint 6756 sessionRestored 7910 500 ms to preload
Intel Atom N450, 2GB RAM, Win 7 Pro 32bit, D2D,D3D,DW off, clean new profile. Nightly (2011-02-11) without wrapper: main 2043 firstPaint 8050 sessionRestored 8487 main 2637 firstPaint 7989 sessionRestored 8082 Nightly (2011-02-11) with wrapper: main 952 firstPaint 3823 sessionRestored 4073 2200ms to preload main 889 firstPaint 3792 sessionRestored 4088 1900ms to preload
Windows XP laptop, Core2 Duo T7200 @ 2GHz, 2GB RAM, 7200RPM, no AV. D2D, DW off. == No wrapper == main 1093 firstPaint 6062 main 1297 firstPaint 7031 sessionRestored 7281 == Wrapper == 900ms to preload main 2672 firstPaint 7079 sessionRestored 7297 500ms to preload main 1047 firstPaint 5844 sessionRestored 6094
Ok, thanks for the testing guys. Sounds like this approach doesn't get annihilated by windows prefetch as badly. There is a small issue in windows throwing away the cache too soon(due to me indicating sequential IO). Thus there is potential for better perf.
Here are my numbers after a defrag. == No wrapper == main 922 firstPaint 5125 sessionRestored 5672 == Wrapper == 300ms to preload main 953 firstPaint 4750 sessionRestored 4984
I think I fixed the cache issue I was seeing before by mmap()ing the area Here are new versions of wrapper. The first one is threaded, to see if keeping cpu into lower power state helps(ie lower frequency resulting in lower IRQ handling), . http://people.mozilla.com/~tglek/startup/wrapper_threaded.exe This one is the same as before but with mmap. http://people.mozilla.com/~tglek/startup/wrapper.exe Justin could you try these to see if your numbers get better?
Is there any way we can get this in nightlies and/or run A/B testing for the various options (using Test Pilot)? I'm worried that ad-hoc testing may not give us the best outcome across millions of devices. Even running this on all the machines in the QA labs would give us additional clarity via more data points...
(In reply to comment #97) > Is there any way we can get this in nightlies and/or run A/B testing for the > various options (using Test Pilot)? I'm worried that ad-hoc testing may not > give us the best outcome across millions of devices. Even running this on all > the machines in the QA labs would give us additional clarity via more data > points... I'm not too worried about testing this. I'm just making sure that preloading doesn't make things slower for people. As far as testing goes we can turn it on in a nightly(assuming the wrapper approach will fly) and then use our startup reporting mechanism(http://blog.mozilla.com/addons/2011/02/10/add-on-metadata-start-up-time/) to see if our startups decrease.
Sorry, Taras; not much movement. == Wrapper 2 == preload 400ms main 1360 firstPaint 6032 sessionRestored 6704 == wrapper_threaded == preload 500ms main 1625 firstPaint 5594
(In reply to comment #91) With same spec: wrapper_mmap main 796 firstPaint 11998 sessionRestored 14322 400ms to preload wrapper_threaded main 858 firstPaint 9985 sessionRestored 10952 500ms to preload
Using Windows 7 x64 and Firefox latest nightly 20110212(32 bits), the first time I run FF4 once I turn on the laptop, varies from 200ms to 500ms, with no changes to anything, I even have disabled autoupdate of addons, FF and search engines.
Some numbers on my system, an almost 4 years old Intel Q6600@2.8ghz 4gb on Windows 7 64 bits. Prefetch info was cleared before each reboot. Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b12pre) Gecko/20110214 Firefox/4.0b12pre with 19 extensions. D2D and DW enabled GPU Accelerated Windows: 1/1 Direct3D 10 w/o wrapper before deleting Prefetch: main 1320 firstPaint 7061 sessionRestored 7251 w/ wrapper before deleting Prefetch (threaded): 300ms preload main 3910 (weird) firstPaint 6561 sessionRestored 6661 w/ wrapper, before deleting Prefetch (mmap): 200ms preload main 710 firstPaint 6371 sessionRestored 6531 --------------------------------- w/ wrapper, after deleting Prefetch (mmap): 300ms preload main 180 firstPaint 6331 sessionRestored 6531 w/ wrapper, after deleting Prefetch (threaded): 100ms preload main 140 firstPaint 2921 sessionRestored 3011
AMD Athlon 4800+ X2 2.5GHz, 1GB DDR2, Seagate ST3160815A 7200 RPM ATA/100 on Windows 7 Ultimate 32bit. Build identifier: Mozilla/5.0 (Windows NT 6.1; rv:2.0b12pre) Gecko/20110216 Firefox/4.0b12pre with 2 add-ons (AdblockPlus and about:startup) Hardware acceleration on. (installed hotfix from MS - KB2467023) With HWA: main 109 firstPaint 1655 sessionRestored 1717 With HWA and wrapper_mmap main 62 firstPaint 1483 sessionRestored 1545 100 ms to preload With HWA and wrapper_threaded main 62 firstPaint 1655 sessionRestored 1717 All
(In reply to comment #103) > With HWA and wrapper_threaded > main 62 > firstPaint 1655 > sessionRestored 1717 200 ms to preload All with prefetch deleted before. Font cache 1,71 MB
(In reply to comment #104) > (In reply to comment #103) > > With HWA and wrapper_threaded > > main 62 > > firstPaint 1655 > > sessionRestored 1717 > 200 ms to preload > > All with prefetch deleted before. > Font cache 1,71 MB ok thanks. How about with prefetch? For everybody else, thanks for the feedback. Due to having to move to a separate binary to avoid windows prefetch penalty this is too complex to release as part of firefox 4. We are considering options for releasing it this after ff4. My least bad option at the moment is to install a helper that wipes firefox prefetch entries on windows startup and do prefetch within the firefox executable as before. The other option is to teach firefox that it's being launched from a wrapper and deploy the wrapper(this is tricky to get right for integration into windows, etc).
(In reply to comment #105) > ok thanks. How about with prefetch? With HWA: main 109 firstPaint 1685 sessionRestored 1732 With HWA and wrapper_mmap: main 78 firstPaint 1545 sessionRestored 1607 200 ms to preload With HWA and wrapper_threaded main 78 firstPaint 1655 sessionRestored 1733 200 ms to preload Sometimes numbers are much higher like 1500 (main) . But after restart it gets lower. This is without deleting prefetch.
(In reply to comment #105) > My least bad option at the moment is to install a helper that wipes firefox > prefetch entries on windows startup and do prefetch within the firefox > executable as before. One of the ideas I heard floated a while back was to switch to a new background automatic updates system, possibly run as a Windows service. (I think Google Chrome does something like this) If this becomes a reality for Firefox, it would make sense to merge this sort of preloader concept with the background auto-updater into one Firefox background service. You could probably even go further and have it preload a fair bit during idle time if there's enough memory free.
To avoid needing to runas or the UAC prompt the service will be running as the system account (default for services) which is privileged. I would very much prefer not preloading anything since it would add a potential attack vector.
(In reply to comment #108) > To avoid needing to runas or the UAC prompt the service will be running as the > system account (default for services) which is privileged. I would very much > prefer not preloading anything since it would add a potential attack vector. Preloading as in populating the file cache doesn't add any attack vectors.
(In reply to comment #109) > (In reply to comment #108) > > To avoid needing to runas or the UAC prompt the service will be running as the > > system account (default for services) which is privileged. I would very much > > prefer not preloading anything since it would add a potential attack vector. > > Preloading as in populating the file cache doesn't add any attack vectors. The summary "preload dlls" is what threw me off and it is definitely safer from the service's point of view.
(In reply to comment #108) > To avoid needing to runas or the UAC prompt the service will be running as the > system account (default for services) which is privileged. I would very much > prefer not preloading anything since it would add a potential attack vector. is there a bug for implementing this service?
Depends on: 481815
Couple of quick notes about the service. 1. it will start during OS startup before any users logon. 2. not all users will be able to install it since it requires admin privs to install. 3. there will likely be an opt-out option in the installer.
(In reply to comment #107) > (In reply to comment #105) > > My least bad option at the moment is to install a helper that wipes firefox > > prefetch entries on windows startup and do prefetch within the firefox > > executable as before. > > One of the ideas I heard floated a while back was to switch to a new background > automatic updates system, possibly run as a Windows service. (I think Google > Chrome does something like this) If this becomes a reality for Firefox, it > would make sense to merge this sort of preloader concept with the background > auto-updater into one Firefox background service. You could probably even go > further and have it preload a fair bit during idle time if there's enough > memory free. I don't see any windows services for Chrome. Google does have added some Scheduled Tasks for update, but no service or task for preload.
(In reply to comment #114) > I don't see any windows services for Chrome. Google does have added some > Scheduled Tasks for update, but no service or task for preload. I think you need to watch two things: * gpedit.msc * services.msc
Isn't a better idea to simply add this ddl's to SuperPrefetch windows folders ? We can add this on installation and next Firefox will be checking on start that this ddls still exist, because can be deleted by some software. And if not Firefox will add it. Checking need to be wise implanted to not burden slow cold start.
(In reply to comment #116) > Isn't a better idea to simply add this ddl's to SuperPrefetch windows folders ? > We can add this on installation Will this work for portable Fx too? > and next Firefox will be checking on start that this ddls > still exist, because can be deleted by some software. And if not Firefox > will add it. Checking need to be wise implanted to not burden slow cold start. E.g. this dll will be copied to SuperPrefetch after 1st Fx run and won't get loaded for the 1st run? Or it is still planned to be done while Windows is loading?
(In reply to comment #117) > Will this work for portable Fx too? No, because portable is like name means portable. It shouldn't add or leave any files & keys in system (except buggy portable "shells") and as far I remember we don't support officially portable Firefox. (In reply to comment #117) > E.g. this dll will be copied to SuperPrefetch after 1st Fx run and won't get > loaded for the 1st run? Or it is still planned to be done while Windows is > loading? Files should be added to SuperPrefetch folders after installation/upgrade or when Firefox check they're missing, so Windows will be preload this ddls on next startup. Regarding first run with cold-start I have idea, that we can called some process that will be preloaded this ddls, before seeing latest installation screen with "Finish / OK" button or if it's possible preload only ddls to memory without it. So Firefox will be loading much faster with this :) I'm not a programmer so it's only theoretical discussion. What devs thing about it ? It's possible ?
(In reply to comment #118) >... > (In reply to comment #117) > > E.g. this dll will be copied to SuperPrefetch after 1st Fx run and won't get > > loaded for the 1st run? Or it is still planned to be done while Windows is > > loading? > > Files should be added to SuperPrefetch folders after installation/upgrade or > when Firefox check they're missing, so Windows will be preload this ddls on > next startup. > Prefetch is managed by the OS so we can't add files directly. > Regarding first run with cold-start I have idea, that we can called some > process that will be preloaded this ddls, before seeing latest installation > screen with "Finish / OK" button or if it's possible preload only ddls to > memory without it. This would only slightly improve first run after an install since we have to generate our own cache, etc. after installing a different version
I wrote an extension http://people.mozilla.com/~tglek/startup/startup.service.xpi (source http://hg.mozilla.org/users/tglek_mozilla.com/startup-service) to deal with the "slow windows prefetch" problem. The extension is a) wrapper around a service .exe that deletes firefox prefetch files and b) wrapper .exe that preloads firefox in a more efficient manner than windows does by default. The extension installs a "Firefox Service" and removes it on uninstall. It also adds a 'faster firefox' shortcut to the desktop. In theory having this extension installed + launching via the "faster firefox" shortcut should result in the 2x speedup that this bug aims for.
AMD Athlon 4800+ X2 2.5GHz, 1GB DDR2, Seagate ST3160815A 7200 RPM ATA/100 on Windows 7 SP1 Ultimate 32bit. 5 extensions installed (AdblockPLus, GreaseMonkey, F1, startup, startup service) + 1 userscript and active persona. HWA on. Before startup service add-on: main 1638 firstPaint 13354 sessionRestored 13448 With startup service add-on: main 858 firstPaint 7848 sessionRestored 7988
So here are my numbers: Maybe cold/warm? (This is what about:startup showed initially after I installed it. Don't remember if this was for the first time I started Firefox after rebooting the system, the session was left running over the weekend.) main 4661 firstPaint 12132 sessionRestored 14184 Warm (with extension) (After installing the startup service addon, I shut down Firefox and started it with the "faster firefox" shortcut.) main 3631 firstPaint 9751 sessionRestored 13391 Cold (with extension) (I shut down everything and restarted my system completely, then used the "faster firefox" shortcut to launch the browser.) main 3761 firstPaint 10314 sessionRestored 12202 Installed/enabled extensions: Add-on Compatibility Reporter JS Deminifier RT Site Extender Stratiform Feedback Adblock Plus Greasemonkey Status-4-Evar Bugzilla Tweaks Add-on Builder Helper Stylish Echofon NoScript Element Hiding Helper for Adblock Plus DOM Inspector startup startup.service I have 7 open tabs, including Gmail, about:addons and Facebook. (Just out of curiosity, do the startup numbers take the time spent in the profile manager's UI into account?)
I tested extension on other computer on Windows 7 SP 1. AMD Athlon 2800+ 2.0 GHz, 1GB DDR RAM, HDD - Western Digital Caviar Blue WD800JD 7200 RPM SATA150 HWA off. Extensions installed: Adblock Plus, GreaseMonkey (one usersript) , about startup, startup service and active persona theme. Cold without startup service: main 2015 firstPaint 4110 sessionRestored 4235 Cold with startup service: main 688 firstPaint 3251 sessionRestored 3548
I don't know if it's meant to, but the extension doesn't appear to work in Windows XP (at the very least, it complains that CompareStringEx can't be located in kernel32.dll, there may be other issues).
(In reply to comment #125) > I don't know if it's meant to, but the extension doesn't appear to work in > Windows XP (at the very least, it complains that CompareStringEx can't be > located in kernel32.dll, there may be other issues). thanks. I'll fix that in the next rev.
Using: - Win7 x64 ; Minefield x86 opt 2011-03-03 - Core 2 Duo 2.4; WD Raptor 10000 RPM SATA150 WD1500ADFD - H/W accel disabled (both layers and D2D) - Session with 5 tabs open; 8 addons + flash cold: (after waiting 60 secs for win services to finish loading) main 2652 firstPaint 4977 sessionRestored 5164 warm: main 62 firstPaint 1779 sessionRestored 1935 with new wrapper from comment 121... cold: (after waiting 60 secs for win services to finish loading) main 546 firstPaint 4540 sessionRestored 4837 warm: main 172 firstPaint 1920 sessionRestored 2092
Gecko/20110308 Firefox/4.0b13pre Windows 7 sp1 32bit GPU: 1/1 direct3 d9 DirectWrite: off HDD: 5400 rpm laptop drive WDC WD3200BEVT 2 Gb RAM 31 extensions Restored session with: 510 tabs (entry in about:config browser.sessionstore.max_concurrent_tabs=0, see also Bug 632012, still have a hope that Bartab extension will be updated to fully unload tabs in Firefox 4.0 like it worked in Firefox 3.6 = session restore should be much more faster) about one month old profile + database vacuum=after use SpeedyFox from http://www.crystalidea.com/speedyfox + 1Gb readyboost= using old (slow) 1 gb flash drive as readyboost cache in Windows 7 _____________________________ Without Installed startup.service.xpi: _____________________________ Cold boot main 2730 firstPaint 5414 sessionRestored 14946 Hot boot main 125 firstPaint 2076 sessionRestored 11217 + 1 Gb readyboost flash drive Cold boot main 78 firstPaint 3527 sessionRestored 12981 Hot boot main 78 firstPaint 2200 sessionRestored 11389 + database vacuum Cold boot main 93 firstPaint 3776 sessionRestored 13073 Hot boot main 125 firstPaint 2060 sessionRestored 10905 + database vacuum + 1Gb readyboost Cold boot main 93 firstPaint 2793 sessionRestored 12215 Hot boot main 63 firstPaint 1998 sessionRestored 10953 _____________________________ With Installed startup.service.xpi, start by desktop shortcut: _____________________________ Cold boot main 546 firstPaint 4010 sessionRestored 13277 Hot boot main 140 firstPaint 2091 sessionRestored 11389 + 1 Gb readyboost flash drive Cold boot main 484 firstPaint 3480 sessionRestored 12715 Hot boot main 140 firstPaint 2107 sessionRestored 11404
by the way, once http://support.microsoft.com/kb/2505438 is installed, keeping directwrite ON should result in faster startups(ie the opposite of before this microsoft released this patch). Thanks for the benchmarks. I have enough data now.
(In reply to comment #125) > I don't know if it's meant to, but the extension doesn't appear to work in > Windows XP (at the very least, it complains that CompareStringEx can't be > located in kernel32.dll, there may be other issues). Uploaded a new version to https://addons.mozilla.org/en-US/developers/addon/startupservice/ . Would be nice if someone could test it on xp.
(In reply to comment #130) > (In reply to comment #125) > > I don't know if it's meant to, but the extension doesn't appear to work in > > Windows XP (at the very least, it complains that CompareStringEx can't be > > located in kernel32.dll, there may be other issues). > > Uploaded a new version to > https://addons.mozilla.org/en-US/developers/addon/startupservice/ . Would be > nice if someone could test it on xp. I tried to access it using the new link above, but it has me log-in to addons.mozilla.org and then loads a blank page with this Url: https://addons.mozilla.org/en-US/developers/addon/startupservice/edit No download or install of the extension occurs.
(In reply to comment #131) > I tried to access it using the new link above, but it has me log-in to > addons.mozilla.org and then loads a blank page with this Url: > https://addons.mozilla.org/en-US/developers/addon/startupservice/edit > > No download or install of the extension occurs. sorry, https://addons.mozilla.org/en-US/firefox/addon/startupservice/
I'm using the addon on XP and it seems to make startup faster. However, an error shows up on install. svc.exe - Entry Point Not Found The procedure entry point GetUserDefaultLocaleName could not be located in the dynamic link library KERNEL32.dll.
(In reply to comment #130) > > Uploaded a new version to > https://addons.mozilla.org/en-US/developers/addon/startupservice/ . Would be > nice if someone could test it on xp. I think you should write a more detailed description for the extension. Check out review #1 on the addon's page.
- i got this error (picture) http://i56.tinypic.com/2vsp6vo.png and i believe it is related to the followings: [ExtensionDirs] Extension0=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{D4DD63FA-01E4-46a7-B6B1-EDAB7D6AD389}.xpi Extension1=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}.xpi Extension2=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{b9db16a4-6edc-47ec-a1f4-b86292ed211d}.xpi Extension3=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{1018e4d6-728f-4b20-ad56-37578a4de76b} Extension4=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{e4a8a97b-f2ed-450b-b12d-ee082ba24781}.xpi Extension5=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{46551EC9-40F0-4e47-8E18-8E5CF550CFB8}.xpi Extension6=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\compatibility@addons.mozilla.org.xpi Extension7=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{EF522540-89F5-46b9-B6FE-1829E2B572C6}.xpi Extension8=C:\Program Files (x86)\Mozilla Firefox 4.0 Beta 8\extensions\{CAFEEFAC-0016-0000-0022-ABCDEFFEDCBA} [ThemeDirs] Extension0=C:\Program Files (x86)\Mozilla Firefox 4.0 Beta 8\extensions\{972ce4c6-7e08-4474-a285-3208198ce6fd} FYI: I'm not a programmer but i would like to help out, also i can't install this addon anymore after uninstalling it this morning (now it is at night 11ish pm) thanks!
(In reply to comment #135) > - i got this error (picture) http://i56.tinypic.com/2vsp6vo.png and i believe > it is related to the followings: > > [ExtensionDirs] > Extension0=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{D4DD63FA-01E4-46a7-B6B1-EDAB7D6AD389}.xpi > Extension1=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}.xpi > Extension2=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{b9db16a4-6edc-47ec-a1f4-b86292ed211d}.xpi > Extension3=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{1018e4d6-728f-4b20-ad56-37578a4de76b} > Extension4=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{e4a8a97b-f2ed-450b-b12d-ee082ba24781}.xpi > Extension5=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{46551EC9-40F0-4e47-8E18-8E5CF550CFB8}.xpi > Extension6=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\compatibility@addons.mozilla.org.xpi > Extension7=C:\Users\FaTe\AppData\Roaming\Mozilla\Firefox\Profiles\rgn4r093.default\extensions\{EF522540-89F5-46b9-B6FE-1829E2B572C6}.xpi > Extension8=C:\Program Files (x86)\Mozilla Firefox 4.0 Beta > 8\extensions\{CAFEEFAC-0016-0000-0022-ABCDEFFEDCBA} > > [ThemeDirs] > Extension0=C:\Program Files (x86)\Mozilla Firefox 4.0 Beta > 8\extensions\{972ce4c6-7e08-4474-a285-3208198ce6fd} > > FYI: I'm not a programmer but i would like to help out, also i can't install > this addon anymore after uninstalling it this morning (now it is at night 11ish > pm) > > thanks! Win 7 (no SP1) x64 - this error occurs when installation was completed and i try faster firefox still slow i believe it is because the installation was not successful
Guess it's broken on xp and 64bit windows. Thanks
Taras, v0.1 WFM using Win 7 x64 and x86 Minefield; so unless 0.2 regressed x64...
Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:2.0b13pre) Gecko/20110309 Firefox/4.0b13pre with Start Faster 0.2 I get a dialog titled "mozjs.dll" that says "Could not open" when running Faster Firefox. It loads once I hit OK.
(In reply to comment #139) > > I get a dialog titled "mozjs.dll" that says "Could not open" when running > Faster Firefox. It loads once I hit OK. Thanks. That's a bug, I'll fix that in next rev. Shandy, in your screenshot, I can't see the end of the title in the message box. it should end in a number. Would be handy to know what the number is.
Extension activated, running Fx after the OS restart: main 508 sessionRestored 11493 firstPaint 12212 Extension deactivated, running Fx after the OS restart: main 3375 sessionRestored 10407 firstPaint 11079 P4 3.0Ghz, hwa=on.
(In reply to comment #141) > Extension activated, running Fx after the OS restart: > main 508 > sessionRestored 11493 > firstPaint 12212 > > Extension deactivated, running Fx after the OS restart: > main 3375 > sessionRestored 10407 > firstPaint 11079 > > P4 3.0Ghz, hwa=on. Is this via faster firefox shortcut?
tried to install ff4 xp sp3 turned off all addons before, on install get the following: svc.exe entry point not found the proceedure entry point GetUserDefaultLocaleName could not be located in the dynamic link library KERNEL32.dll addon appears to install but no short cut is created on desktop
(In reply to comment #140) > (In reply to comment #139) > > > > I get a dialog titled "mozjs.dll" that says "Could not open" when running > > Faster Firefox. It loads once I hit OK. > > Thanks. That's a bug, I'll fix that in next rev. > > Shandy, in your screenshot, I can't see the end of the title in the message > box. it should end in a number. Would be handy to know what the number is. i can't do anything about it =( caz it doesn't let me expand it >.< sorry & hope you will get this fix soon <3
(In reply to comment #142) > Is this via faster firefox shortcut? Sorry, didn't notice that shortcut. But I've tried it now and it opens new profile, instead of my. I'm using firefox portable, and probably that's the reason. Not going to test not on my profile.
FYI: the addon was ok until http://support.microsoft.com/kb/2505438 is installed Win 7 (no sp1) x64
How about installing SP1 with all patches ? Still the same ?
Hi, My Build identifier: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b13pre) Gecko/20110311 Firefox/4.0b13pre I have win 7 x64 with sp1 with the mentioned patch installed, and the add-on seems to work fine.
(In reply to comment #148) > Hi, > > My Build identifier: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b13pre) > Gecko/20110311 Firefox/4.0b13pre > > I have win 7 x64 with sp1 with the mentioned patch installed, and the add-on > seems to work fine. i'm on Firefox RC1 and installed SP1 (took me long time) reinstalled the addon v0.2 still getting same error
there are some problems with reinstalling/upgrading. I'll see about fixing them in next version.
Depends on: 632404
another 2 weeks of ignorance, nice.
Taras, any news on this bug?
(In reply to comment #152) > Taras, any news on this bug? Nothing atm. I have a good idea of how to implement it, but we need a windows updating service before I can implement it(which is being worked on)
Do you have a bug number we can follow?
wait on progress in bug 481815
I think with the current method to solve this bug being to just delete the firefox.exe prefetch files it would be simpler to just use a scheduled task. Taras, would this be acceptable?
(In reply to comment #156) > I think with the current method to solve this bug being to just delete the > firefox.exe prefetch files it would be simpler to just use a scheduled task. > Taras, would this be acceptable? Yes. I think a scheduled task + code in firefox.exe that checks for the firefox.exe* prefetch files before using our own prefetch logic would solve the after-reboot slowness.
Threw together a hacky vbscript as a proof of concept. You can save it to your filesystem, in the task scheduler create a new task, change the user account to run the task to SYSTEM, set the trigger to At system startup, and set the action to Start a program with the vbs file containing the following for the program / script. On Error Resume Next Dim wmi, fso, shell, prefetchdir, querydir, query, monevts, evt Set fso = CreateObject("Scripting.FileSystemObject") Set shell = WScript.CreateObject("WScript.Shell") prefetchdir = shell.ExpandEnvironmentStrings("%windir%") & "\Prefetch" If fso.FolderExists(prefetchdir) Then fso.DeleteFile prefetchdir & "\FIREFOX.EXE*", True querydir = Replace(prefetchdir, "\", "\\\\") Set wmi = GetObject("winmgmts:{impersonationLevel=impersonate}!\\.\root\cimv2") query = "SELECT * FROM __InstanceCreationEvent WITHIN 10 WHERE " & _ "Targetinstance ISA 'CIM_DirectoryContainsFile' and " & _ "TargetInstance.GroupComponent= " & _ "'Win32_Directory.Name=""" & querydir & """'" Set monevts = wmi.ExecNotificationQuery(query) Do Set evt = monevts.NextEvent If fso.FolderExists(prefetchdir) Then fso.DeleteFile prefetchdir & "\FIREFOX.EXE*", True End If Loop End If
(In reply to comment #158) > Threw together a hacky vbscript as a proof of concept. You can save it to your > filesystem, in the task scheduler create a new task, change the user account to > run the task to SYSTEM, set the trigger to At system startup, and set the > action to Start a program with the vbs file containing the following for the > program / script. > What does the part part "Set wmi" do? Is this to keep deleting firefox.exe* if it appears? The only hard part left is registering this script with the task scheduler during Firefox update/install. Is that something that can be done for Firefox 6?
I assume this will fail gracefully and not crash if the user is not using prefetch? My boot drive in Windows 7 is a 120GB SSD and I've turned off Superfetch and deleted the prefetch directory.
(In reply to comment #159) > (In reply to comment #158) > > Threw together a hacky vbscript as a proof of concept. You can save it to your > > filesystem, in the task scheduler create a new task, change the user account to > > run the task to SYSTEM, set the trigger to At system startup, and set the > > action to Start a program with the vbs file containing the following for the > > program / script. > > > > What does the part part "Set wmi" do? Is this to keep deleting firefox.exe* if > it appears? It is a reference to windows management instrumentation. monevts.NextEvent will wait for the next create file event in the prefetch dir and then delete any FIREFOX.EXE* prefetch files. > The only hard part left is registering this script with the task scheduler > during Firefox update/install. Is that something that can be done for Firefox > 6? Likely (In reply to comment #160) > I assume this will fail gracefully and not crash if the user is not using > prefetch? yes
Whiteboard: [ts]
Whiteboard: [ts]
Here is another WiP. Turns out we can detect prefetch.
Attachment #511223 - Attachment is obsolete: true
Lets land a conservative version of this and track progress via our startup-speed tracking. This uses GetProcessIOCounters as an indicator whether prefetch is ruining our startup or whether we still have room to optimize it ourselves. This only preloads xul.dll and mozjs.dll(if it exists) because each of these has more overhead than the other files combined. This will make startup faster for everybody who has broken windows prefetch(which seems relatively common), is running on ssd or uses another method to opt out (ie Rob's script). Then as step two we can install Rob's script from the installer and enjoy awesomer startup times on every machine.
Attachment #530115 - Attachment is obsolete: true
Attachment #530119 - Flags: review?(shaver)
Comment on attachment 530119 [details] [diff] [review] detect when prefetch is off(for whatever reason) and preload our way Review of attachment 530119 [details] [diff] [review]: ::: toolkit/xre/nsWindowsWMain.cpp @@ -93,1 +93,13 @@ -#endif +#endif + +#ifdef XRE_PRELOAD_XUL +static void ... 9 more ... pick a style for bracing single-line consequents, and stick to it please. (I think the rest of this file doesn't brace single lines, but I'm sure you can tell at a glanc @@ +104,5 @@ + return; + } + + DWORD dwBytesRead; + DWORD total = 0; total appears to not be used other than to accumulate read sizes and then be discarded. @@ +108,5 @@ + DWORD total = 0; + // Do dummy reads to trigger kernel-side readhead via FILE_FLAG_SEQUENTIAL_SCAN. + // Abort when underfilling because during testing the buffers are read fully + // A buffer that's not keeping up would imply that readahead isn't working right + while (ReadFile(fd, buf, sizeof(buf), &dwBytesRead, NULL) this would fit on one line, no? with the total-addition removed, you'd want something like /*nothing*/; as the one-line body I think.
Attached patch style fixes — — Splinter Review
Attachment #530119 - Attachment is obsolete: true
Attachment #530154 - Flags: review?(shaver)
Attachment #530119 - Flags: review?(shaver)
Attached patch telemetry probes — — Splinter Review
These are a bit ugly since we need to capture the stats early during startup(before xul.dll is loaded) and pass em into xul.dll.
Attachment #530687 - Flags: review?
Comment on attachment 530154 [details] [diff] [review] style fixes Review of attachment 530154 [details] [diff] [review]: ----------------------------------------------------------------- I gave you r+ before, didn't need to re-review! But while I'm here... ::: toolkit/xre/nsWindowsWMain.cpp @@ +117,5 @@ > +{ > + wchar_t buf[MAX_PATH]; > + size_t pathlen = wcslen(appPath); > + > + for(;pathlen && appPath[pathlen] != L'\\'; pathlen--); this is pretty opaque, can you switch to something that's not pure side-effect, like: while (pathlen && appPath[pathlen] != L'\\') pathlen--; ? r+ with that, or without if you feel strongly.
Attachment #530154 - Flags: review?(shaver) → review+
Comment on attachment 530687 [details] [diff] [review] telemetry probes Benjamin, requesting your review here since you might have a suggestion on how to better pass the startup io data into xul.dll
Attachment #530687 - Flags: review? → review?(benjamin)
hi, my last build takes ages to start, so I investigated the latest changesets and found this one. Is it possible that this is the reason for the huge delay (~7sec)? My OS is Win7 64bit. I am now building two firefox versions. One short before this patch landed and the other with this patch. How do I compare these versions best? After a reboot with the about-startup extension?
(In reply to comment #170) > hi, my last build takes ages to start, so I investigated the latest > changesets and found this one. Is it possible that this is the reason for > the huge delay (~7sec)? My OS is Win7 64bit. > > I am now building two firefox versions. One short before this patch landed > and the other with this patch. How do I compare these versions best? After a > reboot with the about-startup extension? yes it's possible. That's the way to test.
No, it is not this patch: before patch: main sessionRestored firstPaint 483 2341 2138 477 2759 2493 452 2684 2435 after: main sessionRestored firstPaint 718 2450 2232 702 2387 2169 736 2569 2351 The reason must be somewhere else. My latest build gave me this results: main sessionRestored firstPaint 3508 10566 10359 7399 17492 17088 3591 19871 19510
Florian, Try capturing an xperf profile. See http://blog.mozilla.com/tglek/2010/10/04/diagnosing-slow-startup/ for instructions, then email it to me. I can take a look at it. I wonder if we hit a font-enumeration regression.
Comment on attachment 530687 [details] [diff] [review] telemetry probes As noted in IRC, I'd prefer a little bit that this all live in libxul: we can avoid passing the static from firefox.exe in by moving recordstartupcounters into XRE_main and calling GetProcessIoCounters again. remark r? if that's not a good idea for some reason.
Attachment #530687 - Flags: review?(benjamin) → review-
Comment on attachment 530687 [details] [diff] [review] telemetry probes (In reply to comment #174) > As noted in IRC, I'd prefer a little bit that this all live in libxul: we > can avoid passing the static from firefox.exe in by moving > recordstartupcounters into XRE_main and calling GetProcessIoCounters again. > remark r? if that's not a good idea for some reason. This patch reports data that we base our decision to preload xul.dll on. We can't move the GetProcessIoCounters. It has to happen early on in firefox.exe before other .dlls started loading.
Attachment #530687 - Flags: review- → review?(benjamin)
Comment on attachment 530687 [details] [diff] [review] telemetry probes Looks like the way forward is to incorporate this into Mike Hommey's xpcomglue preload
Attachment #530687 - Flags: review?(benjamin)
Let's say this is closed by the landing. The patch from here will be backed out before landing bug 552864 and bug 632404. The latter will reintroduce the windows dll preloading. I'll file a new bug for the telemetry probes.
Status: ASSIGNED → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
>while (ReadFile(fd, buf, sizeof(buf), &dwBytesRead, NULL) && dwBytesRead == sizeof(buf)) shouldn't be dwBytesRead < sizeof ?
Code like this: + while (ReadFile(fd, buf, sizeof(buf), &dwBytesRead, NULL) && dwBytesRead == sizeof(buf)) + /* Nothing */; Is a very bad practice in Windows development, despite the new way to do threading Windows has. First of all current thread is locked until while loop ends. I do not know if this is done in a separated thread or in main one. If it is in main, it locks all UI and message loop until it finishes. Being in a separated thread it locks the thread and dramatically slow down program performance because CPU goes to 100%. Best way to solve this is using Completion Ports (http://msdn.microsoft.com/en-us/library/windows/desktop/aa365198(v=vs.85).aspx) and/or Overlapped CreateFule way to read a file (http://msdn.microsoft.com/en-us/library/windows/desktop/aa365683(v=vs.85).aspx). Only if code is in a different thread than main one, it is necessary to add a Sleep(0) or Sleep(1) into the loop (or another value, but those have special meaning in Win32). If yo specify Sleep(0) you are telling Windows do only the most urgent tasks and return to the loop. Win Sleep(1) you tell Windows: do all the urgent tasks and process some thread message loop messages. This is a server performance problem. If you intend to preload some DLL into memory, you are locking Firefox until all the files are loaded if this code is in main thread, and dramatically slow down it if it is in a secondary one. This code is inefficient and forces Windows not use its internal DLL and file caching schemes.
(In reply to RFOG from comment #180) > Code like this: > > + while (ReadFile(fd, buf, sizeof(buf), &dwBytesRead, NULL) && dwBytesRead > == sizeof(buf)) > + /* Nothing */; > > Is a very bad practice in Windows development, despite the new way to do > threading Windows has. > > First of all current thread is locked until while loop ends. I do not know > if this is done in a separated thread or in main one. If it is in main, it > locks all UI and message loop until it finishes. Being in a separated thread > it locks the thread and dramatically slow down program performance because > CPU goes to 100%. Please understand the context of the code before commenting on it. This runs right at startup (notice the wmain?) on the main thread before any UI is displayed. > > This is a server performance problem. If you intend to preload some DLL into > memory, you are locking Firefox until all the files are loaded if this code > is in main thread Yes, that is the point of the code. > This code is inefficient and forces Windows not use its internal DLL > and file caching schemes. We've found that Windows' file caching is broken. Please read <https://blog.mozilla.com/tglek/2011/09/20/firefox-7-cheating-the-operating-system-to-start-faster/> and all of Taras's other posts before commenting again.
(In reply to Daniel Gutson from comment #179) > >while (ReadFile(fd, buf, sizeof(buf), &dwBytesRead, NULL) && dwBytesRead == sizeof(buf)) > > > shouldn't be dwBytesRead < sizeof > > ? This is intentional. This loop exists solely to populate the windows filesystem cache. Typically this loop succeeds(ie does full reads for the whole file). Assumption is that if there is an under-read, then something weird is going on and we should abort our heuristic to not risk introducing a slowdown.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: