1269666 - (gdoc_findr3_ubuntu(284%)) [perf][google suite][google docs] 284.06%(146478ms) slower than Chrome when creating 3 page content and replace the keyword 15 times

Status: RESOLVED WORKSFORME

(Core :: General, defect, P1)

Description

[Shako Ho]

# Test Case
STR
1. Launch the browser with blank page
2. input the blank google doc page url 
3. input 1 page txt (need to contain the keyword)
4. input 1 image
5. input 1 page txt (need to contain the keyword)
6. input 1 table
7. replace the keyword 15 times
8. close the browser

# Hardware
OS: Ubuntu 14.04 LTS 64-bit
CPU: i7-3770 3.4GMhz
Memory: 16GB Ram
Hard Drive: 1TB SATA HDD
Graphics: GK107 [GeForce GT 640]/ GF108 [GeForce GT 440/630]

# Browsers
Firefox version: 45.0.2
Chrome version: 50.0.2661.75

# Result
Browser | Run time (median value) 
Firefox | 198044.444444444 ms
Chrome  | 51566.6666666666 ms

Comment 1

[Askeing Yen[:askeing][:fyen]]

(In reply to Shako Ho from comment #0)
> # Test Case
> STR
> 1. Launch the browser with blank page
> 2. input the blank google doc page url 
> 3. input 1 page txt (need to contain the keyword)
> 4. input 1 image
> 5. input 1 page txt (need to contain the keyword)
> 6. input 1 table
> 7. replace the keyword 15 times
> 8. close the browser
> 
> # Hardware
> OS: Ubuntu 14.04 LTS 64-bit
> CPU: i7-3770 3.4GMhz
> Memory: 16GB Ram
> Hard Drive: 1TB SATA HDD
> Graphics: GK107 [GeForce GT 640]/ GF108 [GeForce GT 440/630]
> 
> # Browsers
> Firefox version: 45.0.2
> Chrome version: 50.0.2661.75
> 
> # Result
> Browser | Run time (median value) 
> Firefox | 198044.444444444 ms
> Chrome  | 51566.6666666666 ms

test_firefox_gdoc_findr3
test_chrome_gdoc_findr3

Comment 2

[Askeing Yen[:askeing][:fyen]]

https://cleopatra.io/#report=455f6924fa977aac35593d2ff1bb1e00e8d8b330&filter=[{%22type%22%3A%22RangeSampleFilter%22,%22start%22%3A39767,%22end%22%3A45374}]

# Gecko Profiler settings:
* Sampling Interval (ms): 3
* Sampling Buffer Size (samples): 10000000

# Run script:
https://github.com/Mozilla-TWQA/Hasal/blob/master/tests/test_firefox_gdoc_findr3.sikuli/test_firefox_gdoc_findr3.py

Comment 3

[Askeing Yen[:askeing][:fyen]]

# Performance Timing:
 * http://goo.gl/Hl0VRX
{
  "navigationStart": 1462775634344,
  "unloadEventStart": 0,
  "unloadEventEnd": 0,
  "redirectStart": 0,
  "redirectEnd": 0,
  "fetchStart": 1462775634347,
  "domainLookupStart": 1462775634347,
  "domainLookupEnd": 1462775634347,
  "connectStart": 1462775634347,
  "connectEnd": 1462775634347,
  "requestStart": 1462775634349,
  "responseStart": 1462775634765,
  "responseEnd": 1462775634765,
  "domLoading": 1462775634767,
  "domInteractive": 1462775635722,
  "domContentLoadedEventStart": 1462775636705,
  "domContentLoadedEventEnd": 1462775636706,
  "domComplete": 1462775639895,
  "loadEventStart": 1462775639895,
  "loadEventEnd": 1462775639902
}

Comment 4

[Askeing Yen[:askeing][:fyen]]

Attachment: test_firefox_gdoc_findr3_1462775593.har

HAR file

Comment 5

[Askeing Yen[:askeing][:fyen]]

Attachment: test_firefox_gdoc_findr3.diff

Script's diff file (for adding the perf mark into profiling data)

Comment 6

[Askeing Yen[:askeing][:fyen]]

Perf Timing domLoading to loadEventEnd: 5558 - 423 = 5135 ms
AddOn Start to End mark: 45293 - 40189 = 5104 ms

From Gecko Profiling data, the Range [40100, 45401]:
 1763  100% Startup::XRE_Main
  565 32.0% ├─ nsInputStreamPump::OnInputStreamReady
  509 28.9% │  ├─ nsInputStreamPump::OnStateStop
> ref bug 1267971
   33  1.9% │  ├─ nsInputStreamPump::OnStateStart
   23  1.3% │  └─ nsInputStreamPump::OnStateTransfer
  271 15.4% ├─ js::RunScript
  267 15.1% ├─ nsRefreshDriver::Tick
> ref bug 1270427
  152  8.6% │  ├─ PresShell::Paint
   72  4.1% │  ├─ PresShell::Flush (Flush_Style)
   31  1.8% │  ├─ PresShell::Flush (Flush_InterruptibleLayout)
            │  └─ ...so on
  260 14.7% ├─ Timer::Fire
  142  8.1% ├─ nsHtml5TreeOpExecutor::RunFlushLoop
   94  5.3% │  ├─ nsJSUtils::EvaluateString
> ref bug 1270351
   40  2.3% │  ├─ EventDispatcher::Dispatch
            │  └─ ...so on
   77  4.4% ├─ nsJSUtils::EvaluateString
            └─ ...so on

Comment 7

[Askeing Yen[:askeing][:fyen]]

Perf Mark #3, script start
Perf Mark #4, type text (one page)
Perf Mark #5, insert image
Perf Mark #6, add new page
Perf Mark #7, open replace dialog
Perf Mark #8, fill the replace dialog done
Perf Mark #9, click replace botton
Perf Mark #0, script end

=====
Type one page text part 1, the Range [57100, 189801]:
Sampling Interval (ms): 3
https://goo.gl/e4X3sh

44223  100% Startup::XRE_Main
11525 26.1% ├─ nsRefreshDriver::Tick
> ref: bug 1271544
 6835 15.5% │  ├─ PresShell::Paint
 3128  7.1% │  ├─ PresShell::Flush (Flush_Style)
  786  1.8% │  ├─ js::GCRuntime::collect
            │  └─ ...so on
11422 25.8% ├─ nsViewManager::DispatchEvent
11230 25.4% │  ├─ EventDispatcher::Dispatch
> ref: bug 1271543
            │  └─ ...so on
10056 22.7% ├─ _poll
 7935 17.9% ├─ Timer::Fire
 1207  2.7% ├─ js::RunScript
  528  1.2% ├─ EventDispatcher::Dispatch
  390  0.9% ├─ nsInputStreamPump::OnInputStreamReady
  214  0.5% │  ├─ nsInputStreamPump::OnStateStop
  101  0.2% │  ├─ nsInputStreamPump::OnStateStart
   74  0.2% │  ├─ nsInputStreamPump::OnStateTransfer
    1  0.0% │  └─ NS_NewLocalFile ??:?
  216  0.5% ├─ IPDL::PCompositor::RecvDidComposite
            └─ ...so on

=====
Type one page text part 2, the Range [189900, 315301]:
Sampling Interval (ms): 3
https://goo.gl/zyA4it

41788  100% Startup::XRE_Main
11359 27.2% ├─ nsRefreshDriver::Tick
> ref: bug 1271544
 5554 13.3% │  ├─ PresShell::Paint
 4344 10.4% │  ├─ PresShell::Flush (Flush_Style)
  577  1.4% │  ├─ js::GCRuntime::collect
            │  └─ ...so on
10527 25.2% ├─ Timer::Fire
 9309 22.3% ├─ nsViewManager::DispatchEvent
 9173 22.0% │  ├─ EventDispatcher::Dispatch
> ref: bug 1271543
            │  └─ ...so on
 7614 18.2% ├─ _poll
 1166  2.8% ├─ js::RunScript
  360  0.9% ├─ EventDispatcher::Dispatch
  323  0.8% ├─ nsInputStreamPump::OnInputStreamReady
  164  0.5% │  ├─ nsInputStreamPump::OnStateStop
  103  0.2% │  ├─ nsInputStreamPump::OnStateStart
   56  0.2% │  └─ nsInputStreamPump::OnStateTransfer
  311  0.7% ├─ IPDL::PCompositor::RecvDidComposite
            └─ ...so on

Comment 8

[Askeing Yen[:askeing][:fyen]]

Replace "OLD" to "NEW" 15 times, the Range [315300, 3337501]:
Sampling Interval (ms): 3
https://goo.gl/uqj8R6

 7392  100% Startup::XRE_Main
 3584 48.5% ├─ nsViewManager::DispatchEvent
 2684 36.3% │  ├─ EventDispatcher::Dispatch
> ref: bug 1272545
  525  7.1% │  ├─ nsLayoutUtils::GetFrameForPoint
            │  └─ ...so on
 1421 19.2% ├─ Timer::Fire
 1296 17.5% ├─ _poll
  652  8.8% ├─ nsRefreshDriver::Tick
  460  6.2% │  ├─ PresShell::Paint
   70  0.9% │  ├─ EventDispatcher::Dispatch
   30  0.4% │  ├─ js::GCRuntime::collect
   29  0.4% │  ├─ PresShell::Flush (Flush_InterruptibleLayout)
            │  └─ ...so on
  103  1.4% ├─ js::RunScript
   60  0.8% ├─ IPDL::PCompositor::RecvDidComposite
   49  0.7% ├─ EventDispatcher::Dispatch
   32  0.4% ├─ nsInputStreamPump::OnInputStreamReady
            └─ ...so on

Comment 9

[Cynthia Tang [:cynthiatang]]

Hi Bobby,

# Browsers
Firefox version: Nightly 50.0a1 (2016/06/28)
Chrome version: 50.0.2661.75

# Result
Browser  | Run time (median value) 
Firefox  | 327,116.67 ms
Chrome   | 325,661.11 ms
Diff(%)  | 0.45

# Video
https://www.youtube.com/watch?v=8eWCpivQdCo 

# Profiler 
https://cleopatra.io/#report=a968d3f59b36d9f90e567b97443a55a49163b8f4

Comment 10

[Walter Chen[:ypwalter][:wachen]]

# Hardware
OS: Windows 7
CPU: i7-3770 3.4GMhz
Memory: 16GB Ram
Hard Drive: 1TB SATA HDD
Graphics: GK107 [GeForce GT 640]/ GF108 [GeForce GT 440/630]

# Browsers
Firefox version: 47
Chrome version: 51.0.2704.103

# Result
Browser | Run time (median value) 
Firefox | 320606 ms
Chrome  | 312511 ms

Comparing to the old 284% slower results, results in win7 shows an much less difference of 2.59%.
However, please note that both browser has a really bad performance on this test comparing to test results in Ubuntu.

Comment 11

[Jenn Chaulk (:jchaulk)]

plat-rel tracked at the meta level

Comment 12

[Shako Ho]

Hi Harald,

Sorry, what additional information do you need for this bug?

Comment 13

[:Harald Kirschner :digitarald]

Looks like I missed to add a comment, sorry about that. Shako, would you advise keeping this open or closing this given the latest results?

Comment 14

[Shako Ho]

I will suggest we can trigger another round of test by using the latest framework code. If the result didn't show the big difference between Firefox and Chrome, I will suggest we can close this bug due to this test case similar to another test case with replacing actions.

Comment 15

[Shako Ho]

The latest result show there is no much difference between Firefox and Chrome, and also this case is similar to another test case, I will suggest we can close this bug.

The test result as below:
             Median Time     Avg Time
Firefox:     320132.4759     318673.4694
Chrome:      312298.7619     310940

Comment 16

[Andrew Overholt [:overholt]]

(In reply to Shako Ho from comment #15)
> The latest result show there is no much difference between Firefox and
> Chrome, and also this case is similar to another test case, I will suggest
> we can close this bug.

Thanks, Shako!