1213432 - [e10s] Scrolling Facebook with D3D11 Warp acceleration is worse than without acceleration

Status: RESOLVED WONTFIX

(Core :: Graphics, defect, P4)

Description

[Vladan Djeric (:vladan)]

+++ This bug was initially created as a clone of Bug #1213429 +++

This was discovered during content-performance testing. 

STR:

1. Start Nightly on Windows in e10s mode, with D3D11 Warp acceleration enabled (layers.d3d11.force-warp = true), D2D disabled (gfx.direct2d.disabled = true) and the laptop in “Power Saver” mode (but plugged in)
2. Open https://www.facebook.com/barackobama without being logged into Facebook
3. Scroll top to bottom using the scrolling bookmarklet attached to bug 894128 (it does not trigger APZ btw)
4. Repeat steps above but with acceleration disabled: gfx.direct2d.disabled = true and layers.acceleration.disabled = true
5. Compare FPS and consistency of scrolling with & without acceleration

We tested with Nightly e10s on Windows 8.1 on the content-perf test laptop: HP Pavillion 14t i3-5010u in “power saver” mode (plugged into power)

Avi notes that without e10s he saw the opposite result, i.e. D3D11 Warp acceleration was benefiting scrolling

Note that our other 2 references pages (Yahoo and Twitter) were not taxing enough to cause meaningful performance differences in this test setup

Comment 1

[Jim Mathies [:jimm]]

scrolling regressions are tier 1 in the release criteria doc. upping priority.

Comment 2

[Benjamin Smedberg]

renominating, this should block

Comment 3

[George Wright (:gw280) (needinfo me!)]

I agree this should block. I just re-ran the test locally and now I can reproduce it again. I'm not sure why I couldn't before.

Anyway, here are the results from the scrolling bookmarklet referenced in the original report:

D3D11 WARP:

Page: https://www.facebook.com/barackobama?_rdr=p
Steps (after ignoring 120): 71
Step: 5 px
Duration: 7035 ms (target: 7000)

Window size: 1687 x 925
Average interval: 37.89 ms
STDDEV intervals: 11.31 ms

intervals histogram:
8.0 - 10.0 ms: 2
18.0 - 22.0 ms: 1
22.0 - 32.0 ms: 9
32.0 - 35.0 ms: 30
35.0 - 50.0 ms: 20
50.0 - 80.0 ms: 9

intervals (sans ignored):
31.08
32.19
21.44
37.70
30.44
46.77
33.55
33.75
33.09
33.70
36.61
33.63
45.43
34.01
33.52
33.16
33.72
33.13
34.31
32.60
33.49
35.19
29.74
9.58
27.50
46.39
34.16
37.83
35.10
29.61
22.21
27.56
48.24
34.34
32.77
33.38
34.21
32.85
30.63
9.94
27.53
46.95
34.64
33.56
32.85
34.39
32.71
33.49
32.82
33.16
33.54
33.61
36.38
45.34
37.10
74.98
53.70
66.42
67.75
55.00
41.94
49.23
58.10
51.40
38.57
48.28
48.67
51.94
46.86
51.06
49.57

Basic (OMTC):

BASIC:

Page: https://www.facebook.com/barackobama
Steps (after ignoring 120): 174
Step: 5 px
Duration: 7009 ms (target: 7000)

Window size: 1687 x 925
Average interval: 25.81 ms
STDDEV intervals: 6.86 ms

intervals histogram:
12.0 - 14.0 ms: 3
14.0 - 15.9 ms: 10
15.9 - 17.3 ms: 2
17.3 - 18.0 ms: 3
18.0 - 22.0 ms: 41
22.0 - 32.0 ms: 76
32.0 - 35.0 ms: 21
35.0 - 50.0 ms: 18

intervals (sans ignored):
19.74
24.03
15.89
19.19
18.76
22.67
23.46
15.03
20.00
18.00
20.34
19.75
23.86
13.75
21.60
18.20
20.11
26.73
14.58
19.57
18.62
21.26
25.76
15.92
18.72
19.62
21.91
24.23
15.35
19.48
18.59
21.63
24.51
16.88
18.39
19.09
21.89
23.91
14.82
19.84
18.96
22.11
24.38
14.60
19.85
18.54
22.62
24.38
15.82
19.01
18.52
21.33
25.29
15.49
19.54
18.45
20.33
27.38
14.06
19.77
18.78
22.58
23.95
14.82
20.08
17.95
21.25
18.27
23.83
13.32
21.18
19.23
17.98
18.25
17.39
26.47
12.99
22.65
28.77
21.49
42.61
36.94
35.91
39.92
37.49
37.72
30.53
28.45
25.67
26.89
25.73
30.31
34.13
35.66
31.48
28.31
27.58
27.01
27.49
26.22
29.25
33.69
34.06
32.72
32.75
34.83
30.26
29.52
25.69
26.52
24.46
26.52
27.38
26.26
27.14
26.20
26.02
22.51
26.92
24.72
30.82
28.54
37.55
28.03
36.38
34.48
39.67
34.79
39.69
33.04
27.88
25.58
27.78
23.99
29.75
28.51
23.97
26.74
23.92
27.60
25.14
25.80
26.42
26.56
27.56
24.48
28.60
27.66
26.39
29.58
30.08
27.78
28.99
29.68
32.19
30.44
33.58
32.32
32.73
33.44
33.72
34.62
36.16
33.50
32.10
33.08
32.86
38.76
40.32
34.92
42.55
38.00
36.58
35.57

Comment 4

[David Anderson [:dvander] - inactive, e-mail if emergency]

Does WARP actually buy us anything? We've had a bunch of bugs with it, and we can't use it for video or content acceleration. Only a small percentage of users end up getting it, too.

Comment 5

[George Wright (:gw280) (needinfo me!)]

My understanding is that we have a significant number of users on WARP, based on a conversation with Milan. If I remember correctly, the figure is around 3 million users?

Anyway, I'm currently getting numbers from non-e10s to see what the damage is for e10s vs. non-e10s.

Comment 6

[Kartikaya Gupta (email:[e-mail])]

I would be very interested to know if the e10s numbers change with the patches from bug 1253860 (assuming APZ is kept enabled).

Comment 7

[George Wright (:gw280) (needinfo me!)]

Huh, so funny story. The figures I posted in comment 3 were the *non-e10s* values. Turns out I'd got it disabled in my e10s profile (long story, I'd done some testing before in non-e10s then forgot to turn it back on).

Here are the equivalent values for e10s:

D3D11 WARP:


Page: https://www.facebook.com/barackobama?_rdr=p
Steps (after ignoring 120): 44
Step: 5 px
Duration: 7022 ms (target: 7000)

Window size: 1687 x 925
Average interval: 50.00 ms
STDDEV intervals: 11.56 ms

intervals histogram:
22.0 - 32.0 ms: 4
32.0 - 35.0 ms: 3
35.0 - 50.0 ms: 20
50.0 - 80.0 ms: 17

intervals (sans ignored):
49.65
27.40
63.36
59.48
53.28
46.57
30.55
69.69
33.38
66.93
48.78
31.41
69.91
48.38
32.90
71.96
46.36
50.06
29.85
70.88
49.75
48.27
40.26
60.78
50.57
47.80
56.15
45.94
49.98
49.35
48.92
49.99
46.99
54.10
50.76
49.53
43.85
60.70
45.15
51.37
45.97
33.00
71.21
48.78

Basic (OMTC):


Page: https://www.facebook.com/barackobama
Steps (after ignoring 120): 236
Step: 5 px
Duration: 7025 ms (target: 7000)

Window size: 1687 x 925
Average interval: 20.51 ms
STDDEV intervals: 10.84 ms

intervals histogram:
8.0 - 10.0 ms: 29
10.0 - 12.0 ms: 2
12.0 - 14.0 ms: 1
14.0 - 15.9 ms: 8
15.9 - 17.3 ms: 61
17.3 - 18.0 ms: 25
18.0 - 22.0 ms: 31
22.0 - 32.0 ms: 59
32.0 - 35.0 ms: 7
35.0 - 50.0 ms: 8
50.0 - 80.0 ms: 4
120.0 - 180.0 ms: 1

intervals (sans ignored):
24.91
8.44
23.48
8.86
25.01
8.49
24.65
8.65
24.19
8.05
25.78
8.12
25.29
8.37
24.60
8.53
24.98
8.43
25.07
8.45
24.57
8.44
25.35
8.29
24.80
8.43
25.01
8.36
25.36
9.26
23.86
9.73
25.28
8.57
22.67
8.51
25.44
8.57
24.24
8.85
25.12
8.38
24.51
8.23
25.37
8.42
25.28
8.84
24.11
8.33
24.66
23.49
36.26
32.20
24.22
26.54
30.70
24.36
28.78
32.53
24.26
29.84
28.99
25.96
33.74
33.35
31.40
30.94
29.65
25.06
8.81
23.21
8.22
25.17
8.29
25.18
9.19
33.07
11.80
23.26
24.70
26.80
32.32
10.52
36.51
12.35
16.63
16.71
16.67
16.39
16.74
17.00
16.85
17.81
15.65
17.50
15.98
16.52
16.08
16.77
16.51
16.69
16.82
16.64
16.54
16.83
17.30
16.58
16.21
17.20
19.95
16.49
16.02
16.80
15.87
15.81
17.31
17.19
16.55
15.04
16.92
16.04
16.27
16.48
16.59
16.81
16.80
16.64
16.51
18.99
15.93
16.00
17.35
15.78
16.64
16.22
16.51
17.57
16.35
16.35
16.53
16.92
16.98
16.01
17.13
16.98
18.82
16.37
15.45
16.06
16.23
16.76
16.72
26.65
31.76
33.44
37.69
58.20
28.60
35.64
35.64
36.95
48.75
47.47
51.10
52.55
52.54
127.70
24.44
14.46
23.01
18.87
17.25
16.06
18.80
17.68
17.82
18.82
18.08
23.83
19.65
18.11
17.85
18.38
17.62
25.32
15.83
17.51
16.76
18.16
19.34
18.53
26.62
16.34
17.53
18.64
17.49
19.13
18.91
18.03
18.25
23.54
17.88
17.32
17.98
18.47
17.97
18.25
24.22
16.53
17.49
17.78
17.60
17.10
19.48
18.68
19.70
23.49
18.06
17.18
17.51
17.55
19.35
20.15
23.13
18.78
17.07
17.93
17.75
19.72
21.04
24.64
18.64
17.55
17.92
18.68

Comment 8

[George Wright (:gw280) (needinfo me!)]

So from these figures it looks like there *is* a regression in WARP performance, but it's not as cut and dry as the original report made out. Interestingly, basic gets faster in e10s vs non-e10s, and WARP gets slower, but in both cases WARP is significantly slower than basic.

Seeing as Basic is faster in both cases and actually gets better with e10s, should we be considering whether we want to enable WARP at all? I mirror dvander's question about whether WARP is actually beneficial to us.

Avi, Vlad's original report states that you didn't see the slow down in non-e10s. Can you confirm this, as I am not seeing those results here.

Comment 9

[Milan Sreckovic [:milan] (needinfo for best results)]

Given our dismal performance with basic compositor and video (see bug 1253062), if WARP improves video performance, we'd have to keep it.  Can somebody time that?

Comment 10

[Jeff Muizelaar [:jrmuizel]]

(In reply to Milan Sreckovic [:milan] (PTO until 3/23) from comment #9)
> Given our dismal performance with basic compositor and video (see bug
> 1253062), if WARP improves video performance, we'd have to keep it.  Can
> somebody time that?

Ethan will do this bug 1254877

Comment 11

[Avi Halachmi (:avih)]

(In reply to George Wright (:gw280) (:gwright) from comment #8)
> Avi, Vlad's original report states that you didn't see the slow down in
> non-e10s. Can you confirm this, as I am not seeing those results here.

TL;DR: I can't find evidence that this statement is correct. Rather the contrary.

Judging by the summaries we've had back then, it seems that Aaron and myself agreed that d3d11_warp is worse than $something, and in my subjective assessment table it shows that at the time I considered d3d11_warp and d3d9 similar and meaningfully worse than other combinations on e10s when scrolling facebook.

In subjective scores for facebook scrolling in e10s, I assessed d3d11_warp and d3d9 as "7", while all others were 8 or 9.

In e10s but on yahoo or twitter, d3d11_warp actually performed among the best (I actually gave it a 10), but most combos were excellent on these pages.

Note, however, that the same tables shows that in non-e10s, d3d11_warp is not worse than others combos.

Also worth mentioning that bug 1253860 which just landed yielded an amazing scroll performance improvement in e10s (which so far seems real), to a degree which makes it meaningfully better than non-e10s, but not sure how it applies to d3d11_warp, or if it's also improved while e10s is enabled but APZ is not.

Comment 12

[Avi Halachmi (:avih)]

(In reply to George Wright (:gw280) (:gwright) from comment #8)
> Avi, Vlad's original report states that you didn't see the slow down in
> non-e10s. Can you confirm this, as I am not seeing those results here.

Sorry, in non-e10s, I indeed didn't see any issue with d3d11_warp:

(In reply to Avi Halachmi (:avih) from comment #11)
> Note, however, that the same tables shows that in non-e10s, d3d11_warp is
> not worse than others combos.

Comment 13

[George Wright (:gw280) (needinfo me!)]

(In reply to Avi Halachmi (:avih) from comment #11)
> Note, however, that the same tables shows that in non-e10s, d3d11_warp is
> not worse than others combos.

Where can I see this table? Curious to see where our results diverge.

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #6)
> I would be very interested to know if the e10s numbers change with the
> patches from bug 1253860 (assuming APZ is kept enabled).

I tried it and didn't see much of a difference. WARP is still quite a bit slower than basic.

Comment 14

[Avi Halachmi (:avih)]

Attachment: gfx-e10s-scroll-perf-table.png

(In reply to George Wright (:gw280) (:gwright) from comment #13)
> Where can I see this table? Curious to see where our results diverge.

A google doc (attached screnshot). Note that Firefox is from July 2015.

Comment 15

[Jim Mathies [:jimm]]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #10)
> (In reply to Milan Sreckovic [:milan] (PTO until 3/23) from comment #9)
> > Given our dismal performance with basic compositor and video (see bug
> > 1253062), if WARP improves video performance, we'd have to keep it.  Can
> > somebody time that?
> 
> Ethan will do this bug 1254877

Jeff, who's Ethan and can he take bug 1254877?

Comment 16

[Jeff Muizelaar [:jrmuizel]]

FWIW, it sounds like we're likely to disable WARP

Comment 17

[Jim Mathies [:jimm]]

This is waiting on a decision from the gfx team in disabling WARP. Should probably end up in m9 if that's the plan.

Comment 18

[Milan Sreckovic [:milan] (needinfo for best results)]

(In reply to Jim Mathies [:jimm] from comment #17)
> This is waiting on a decision from the gfx team in disabling WARP. Should
> probably end up in m9 if that's the plan.

Tracked in bug 1260507

Comment 19

[Milan Sreckovic [:milan] (needinfo for best results)]

George, we still don't automatically go to D3D11 WARP compositor since bug 1260507, right?  If that's the case, we should be able to close this bug.

Comment 20

[George Wright (:gw280) (needinfo me!)]

I believe so, but let me double check.

Comment 21

[George Wright (:gw280) (needinfo me!)]

Yeah, it looks like it. However, we may want to keep this open as it's a legitimate WARP bug.

Comment 22

[Milan Sreckovic [:milan] (needinfo for best results)]

platform-rel minus as this is not a configuration the users can get to without messing with preferences.

Comment 23

[Mayank Bansal]

WARP support doesnt exist in Firefox anymore.