bugzilla.mozilla.org will be intermittently unavailable on Saturday, March 24th, from 16:00 until 20:00 UTC.
# Test Case STR 1. Launch the browser with blank page 2. Open the google doc with 1 page (UTF-8) content (https://goo.gl/QgcVcP) 3. close the browser # Browsers Firefox version: Nightly 50.0a1 (2016/06/28) Chrome version: 50.0.2661.75 # Result Browser | Run time (median value) Firefox | 10,288.89 ms Chrome | 8,188.89 ms # Video https://www.youtube.com/watch?v=RIYaZHq6U64 # Profiler https://cleopatra.io/#report=07e499a626cf296e5ffd4280a08e3b84d95cc9e3 # Test Script: https://github.com/Mozilla-TWQA/Hasal/blob/master/tests/test_firefox_gdoc_read_utf8_txt_1.sikuli/test_firefox_gdoc_read_utf8_txt_1.py # Hardware OS: Ubuntu 14.04 LTS 64-bit CPU: i7-3770 3.4GMhz Memory: 16GB Ram Hard Drive: 1TB SATA HDD Graphics: GK107 [GeForce GT 640]/ GF108 [GeForce GT 440/630]
Jonathan knows about text stuff.
Flags: needinfo?(overholt) → needinfo?(jfkthame)
It's not obvious to me that there's a lot of "text stuff" involved here... but then, maybe I don't know how to find it in the profiler. Can you give some step-by-step guidance as to how to drill into the given profile and see text stuff where we're spending too much time?
Flags: needinfo?(jfkthame) → needinfo?(overholt)
I'm sorry I was just assuming it was text/layout-related due to the content. Is it worth running the same (or I guess similar since that's probably not strictly possible) test with other languages or English text to compare? Cynthia, is that possible?
Yes, it'd be interesting to know whether the performance (or the perf difference from Chrome) is affected by the kind of text in the document -- e.g. Chinese vs an Indic script like Hindi vs simple English text.
Hi Andrew, The test result for reading a google document (English characters): Median value (ms) Average value (ms) Standard Deviation (ms) Firefox | 18,072.22 17,893.50 519.46 Chrome | 17,394.44 17,409.44 114.79 Diff. | 677.78 (3.90%) # Video https://www.youtube.com/watch?v=z6eV-zjrxd8 # Test data https://goo.gl/itZnFM # Browsers Firefox version: Nightly 50.0a1 Chrome version: 50.0.2661.75
Does that mean it's slower for English text (by 8-9 ms (or is it actually 18 *seconds*?) for both Firefox and Chrome)? Am I comparing comment 5 and comment 0 correctly?
Looking at the documents concerned (to the extent that they're visible in the videos), they're not really comparable... the English document has several long paragraphs of small text, which will require lots of line-breaking, while the Chinese one has a lot of single-line paras and only a little bit of line-breaking needed. That might make the English somewhat more expensive to lay out. Nevertheless, I'm surprised at such a big difference, assuming both comment 0 and comment 5 tests were run on the same system. AFAICT from the video, it's not primarily the text content that's taking time; watching the Firefox window in https://www.youtube.com/watch?v=z6eV-zjrxd8, the text of the document appears quite early (though it's still lagging behind Chrome). But other parts of the Google Docs UI take considerably longer to appear. I wonder if there are other factors that affect it more than the content of the document -- e.g., whether the browser is currently signed in to a Google account or not. Is it possible there's some inconsistency between the tests on issues like this?
Created attachment 8770617 [details] Jason file for reading a gDoc (en) Hi Shako, Could you please answer Andrew's question. Thanks! The attachment is the Jason file for testing "test_chrome_gdoc_read_basic_txt_1" and "test_firefox_gdoc_read_basic_txt_1". The running time is more than 17 second. But the video length Original Video: Chrome https://drive.google.com/file/d/0BwkEhia_D6l_TUlRNUJ0SVA2Q0k/view?usp=sharing Firefox https://drive.google.com/file/d/0BwkEhia_D6l_MmZKNUllc0tsbkk/view?usp=sharing
Flags: needinfo?(ctang) → needinfo?(sho)
Please also answer Jonathan's questions in comment 7, Shako. Thanks.
Reply to comment 6, Hi Andrew, Comparing to the result of comment 5 and comment 0, I will say for comment 0 we have significant difference (25%) between Firefox and Chrome, but in comment 5 we didn't see there is any significant difference(~4%). In this case, we will not compare comment 0 and comment 5 in the same base, because they are testing in different machine (even all of our machines have similar spec, we still find some particular machines generate slightly difference result with others. we are doing the test to eliminate the difference between machines). So basically, we expected that we focus on the "difference between different browser at same round" rather than the actual running time, the running time could be slightly affected by network, machine, etc. Reply to Comment 7, Hi Jonathan, You are right about Google Docs UI, according to our testing result, we actually take longer than Chrome to present the UI editor parts. And for the difference between comment 0 and comment 5, we all sign-in to the google account on all our tests, the scripts and contents are all the same. The only different thing is machine. For comment 0, we ran the test on 3 different machine and get the median value of them. For comment 5, we only pick 1 machine to ran the test and the the median value of them. And we are trying to eliminate the difference between machine, sorry for causing the confusion. Just I said before, we should focus on the comparing difference between browser not actual running time. The comparing difference on different environment should not have huge difference, but the running time on different environment could have huge difference.
(In reply to Shako Ho from comment #10) > So basically, we expected that we focus on the > "difference between different browser at same round" rather than the actual > running time, the running time could be slightly affected by network, > machine, etc. I agree. Thanks for clarifying.
Alias: gdoc_read_utf8_txt_1(25.64%) → gdoc_read_utf8_txt_1_ubuntu(25.64%)
This case is same test case with bug 1269698. let's dig more in #1269698 and come back this.
2 years ago
platform-rel: --- → ?
Whiteboard: [platform-rel-Google][platform-rel-GoogleDocs] → [platform-rel-Google][platform-rel-GoogleSuite][platform-rel-GoogleDocs]
Summary: [Perf][google docs] 25.64%(2100 ms) slower than Chrome when opening 1 page UTF8 content → [perf][google suite][google docs] 25.64%(2100 ms) slower than Chrome when opening 1 page UTF8 content
Whiteboard: [platform-rel-Google][platform-rel-GoogleSuite][platform-rel-GoogleDocs] → [qf:investigate][platform-rel-Google][platform-rel-GoogleSuite][platform-rel-GoogleDocs]
You need to log in before you can comment on or make changes to this bug.