Closed Bug 8702 Opened 21 years ago Closed 20 years ago
Performance improvement for UTF-8 converter
Below is an excerpt from a message I posted on May 7th. I did not get any response to this message. You can skip the beginning if you want and jump directly to the last paragraph (the first part is about nsJSContext, not Internationalization, but I included it so that you get an idea of the context). It shows that when opening a new window with AppRunner, 40% of the time is spent in nsUnicodeDecoderHelper::ConvertByTable(). Note that at that time, the command "New Browser Window" was opening a blank window. -------------- I profiled Viewer on a Mac while loading "test9.html" and found out that 49% of the time is spent in nsJSContext::InitContext(nsIScriptGlobalObject*). The table below shows that 61.8% of the time was spent inside just 8 calls to DocumentViewerImpl::Init() and the main culprits are 4 calls to nsJSContext::InitContext() that represent 25.1%, and 6 more calls that represent another 24.2%. Function #Calls %Time (...) NET_NGLayoutConverter 12 67.8 nsDocumentBindInfo::OnStartBinding 12 67.6 nsWebShell::Embed 8 66.8 --DocumentViewerImpl::Init 8 61.8 ----GlobalWindowImpl::SetNewDocument 4 32.6 ------nsJSContext::InitContext 4 25.1 <--- ------nsJSContext::GC 4 6.7 ----nsWebShell::GetScriptableObject 8 28.4 ------nsWebShell::CreateScriptEnvironment 8 28.4 --------NS_CreateContext 7 28.4 ----------nsJSContext::InitContext 6 24.2 <--- ----------nsServiceManager::GetService 6 4.2 --DocumentViewerImpl::Release 1 5.0 I did a similar test with "test0.html" which has only one webshell: nsJSContext::InitContext() was called once and represented 11.9% of the CPU time. With "test9b.html" which has 2 webshells, it was called twice for a total of 33.1% (17.0 + 16.1). What about AppRunner? A profile taken when opening a second window shows that we have 2 calls taking 17.5% (8.6 + 8.9). But... we also spend almost 42% of the CPU time inside nsUnicodeDecoderHelper::ConvertByTable(), with especially 48000 calls to uMapCode() and uScan(). It means that without these malicious unicode conversions, nsJSContext::InitContext() would represent 30% (= 17.5*100/(100- 42)). --------------
Maybe you already fixed it. I just saw a checkin in nsUnicodeDecodeHelper.cpp that refers to some caching. Besides I can no longer see a huge amount of time spent in i18n utilities when profiling recent builds although I can't tell whether it's because you fixed the problem or whether it's because I'm overflowing the number of functions that can be traced by the Profiler.
Summary: nsUnicodeDecoderHelper::ConvertByTable() takes 40% of the CPU time on New Window → nsUnicodeDecoderHelper::ConvertByTable() used to take 40% of the CPU time on New Window
Changed the summary from "takes 40%" to "used to take 40%" in order to better reflect the time spent since I did that measurement.
Summary: nsUnicodeDecoderHelper::ConvertByTable() used to take 40% of the CPU time on New Window → [PP]nsUnicodeDecoderHelper::ConvertByTable() used to take 40% of the CPU time on New Window
I have not make the cache work yet. But I did improve the performace by turning some tinyfunction into macro. (which I should do long time ago.) Keep this bug open and mark as M10.
Mark this as M12. I plan to tune the UTF-8 to unicode nsIUnicodeDecoder but I am not sure this is still related.
Status: ASSIGNED → RESOLVED
Closed: 20 years ago
Resolution: --- → WONTFIX
This seems no longer the problem. Mark it as WONTFIX. Please reopen if you can still see the perf problem from profiler.
Reopening, on the basis of some performance data that I collected from a build around 9/29 (i.e. after the macro change was supposed to have been checked in). Profiling data was collected for first window creation. The following i18n routines show up within the routines that take more than 1% of the time (excluding children): function call count % of time (exclusive) uMapCode 94276 3.5 uCheckAndScanByTable 94020 2.5 uScan 94397 2.4 uMapFormate0 94244 1.3 uScanAlways1Byte 94020 1.3 These are all within the top 30 functions
Summary: [PP]nsUnicodeDecoderHelper::ConvertByTable() used to take 40% of the CPU time on New Window → nsUnicodeDecoderHelper::ConvertByTable() performance needs improvement
My previous comments let me think that the performance hit would be much greater when opening subsequent windows. In the tests I made last june, the I18N functions did not appear when opening the first window because I did not focus on anything less than 5 or 10% of the CPU time but when opening a second window they represented 40%. It may not be as bad nowadays but it would still be interesting to check. Franck: you can find more info about the Profiler in news://news.mozilla.org/37573DA1.BC07FE95%40netscape.com
simon's commet make sense. keep it in my M12 list for now. I think rewrite UTF8 to Unicode conversion will solve most of the issue....
Assignee: ftang → bobj
Status: ASSIGNED → NEW
Target Milestone: M12 → M13
We should rewrite UTF-8 converter to not call the helper function but convert by itself. Reassign this to bobj to find someone to own this. Change it to M13
Summary: nsUnicodeDecoderHelper::ConvertByTable() performance needs improvement → [Perf] nsUnicodeDecoderHelper::ConvertByTable() performance needs improvement
Whiteboard: [Perf] HELP WANTED → HELP WANTED
Assignee: bobj → jbetak
Status: ASSIGNED → NEW
Target Milestone: M13 → M14
Reassigned to jbetak for M14
Summary: [Perf] nsUnicodeDecoderHelper::ConvertByTable() performance needs improvement → nsUnicodeDecoderHelper::ConvertByTable() performance needs improvement
Deferring PDT + or - until we sort key tasks for beta1. Is there an absolute time vs a % time available for this?
Adding [NEED INFO] to Status Summary.
Whiteboard: [NEED INFO]
Per emailvcomments from jbetak: "The last measurement indicated significant speed improvement (10- 20x) over the previous version of the decoder. This code is called every time Mozilla opens a file orvacesses a URL... I counted around 100 calls when starting the browser and half as muchvwhen opening a new browser window. The only confusing thing was Pierre's claim thatvwe spend up to 40% of the time in the converter. I haven't been able to verify that, thevchange will result in significant performance improvement, it will be in one-digit % rangevfor browser-start-up and maybe twice that amount when opening a new browser window." Removing [NEED INFO]
Whiteboard: [NEED INFO]
Change to summary from "" to "performance improvemenance for UTF-8 converter"
Summary: nsUnicodeDecoderHelper::ConvertByTable() performance needs improvement → performance improvemenance for UTF-8 converter
Putting on PDT+ radar for beta1.
I'm closing this - the changes should be in the next build. Since the profiler corrupted my system I was not able to quantify the improvement as thoroughly as I wanted to. I'm supplying a large UTF-8 test file though and it seems like we can achieve up to 200% performance improvement over the old version of the converter up to 50% improvement over Navigator 4.7. Also, we need to file a new bug for the decoder buffering. If you open the test file with a new build or Navigator 4.x it renders properly. If you use an old build (e.g. 2000020310), then you will notice that we "eat" some parts of the HTML and the file rendering is disrupted. Although this buffering scheme is not used in UTF-8 anymore, it is still needed for other decoders and should be revisited later (say around M20)
Status: ASSIGNED → RESOLVED
Closed: 20 years ago → 20 years ago
Resolution: --- → FIXED
Summary: performance improvemenance for UTF-8 converter → Performance improvement for UTF-8 converter
I verified this in 2000021008 Mac build.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.