916657 - Android x86 emu reftests run significantly slower

Status: RESOLVED DUPLICATE of bug 929048

(Core :: Graphics, defect)

Description

[Geoff Brown [:gbrown]]

+++ This bug was initially created as a clone of Bug #907351 +++

This bug is very similar to dminor's bug 907351, but applies to the upcoming Android x86 environment (Android emulators running Android 4.2).

I have not identified a regression range, but note that x86 (plain, R1..Rn) reftests are running slower than they were in July.

For the current build (Skia-GL enabled), we can only run the first 650 tests in R1 in one hour.

With gfx.canvas.azure.accelerated=false, there is little change.

With gfx.canvas.azure.accelerated=false and gfx.canvas.azure.backends = "cairo", about 840 R1 tests can be run in one hour -- a significant improvement.


Should we run reftests with gfx.canvas.azure.accelerated=false and gfx.canvas.azure.backends = "cairo" then? Even so, it looks like we will need about 20 test chunks to avoid timeouts -- 5 times worse than our current performance on Tegras.

Comment 1

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

blassey, mfinkle: this bug has significant impact on the androidx86 work in progress. Can you help fix or find an owner?

Comment 2

[Benoit Jacob [:bjacob] (mostly away)]

I am scared by the idea of running reftests on cairo canvas. That would make reftests the only thing that ever runs on cairo canvas, which is scary (we want reftests to be reliable).

(And eventually, cairo might be removed. Though that won't happen very soon, as switching content away from cairo is nontrivial).

If Skia is really much slower than Cairo, given that Skia is what we use for canvas, we should profile it and try to fix that performance bug.

Comment 3

[Brad Lassey [:blassey] (use needinfo?)]

(In reply to Geoff Brown [:gbrown] from comment #0)
> Should we run reftests with gfx.canvas.azure.accelerated=false and
> gfx.canvas.azure.backends = "cairo" then? Even so, it looks like we will
> need about 20 test chunks to avoid timeouts -- 5 times worse than our
> current performance on Tegras.

No, I wouldn't want to test something we're not shipping except as a last option. Assigning to George (though snorp should chime in as well)

Comment 4

[George Wright (:gw280) (needinfo me!)]

Where can I get information on how the emulators are set up?

Comment 5

[Geoff Brown [:gbrown]]

(In reply to George Wright (:gw280) from comment #4)
> Where can I get information on how the emulators are set up?

See bug 894507, especially comments 9 and 17.

Comment 6

[Geoff Brown [:gbrown]]

(In reply to George Wright (:gw280) from comment #4)
> Where can I get information on how the emulators are set up?

You can also see these tests running now on Cedar: https://tbpl.mozilla.org/?tree=Cedar&showall=1. Reftests for "Android 4.2 x86 Opt" run in jobs S3, S4, and S5.

Comment 7

[Geoff Brown [:gbrown]]

I collected some gross timing info in this try run: https://tbpl.mozilla.org/?tree=Try&rev=e4a9d6634d14&showall=1 and ran the same patch on x86. Here's a quick summary:

                      Test count       Test time (s)    Tests/second
Linux x64                 9954             1810             5.5
Android 2.2/Tegra         2591             1480             1.7
Android 4.0/Panda         1430              974             1.5
Android 4.2/x86 emu       1213             3621             0.3

                      Test time (s)    Draw time (s)    Draw time/Test time
Linux x64                 1810              31                2%
Android 2.2/Tegra         1480             876               59%
Android 4.0/Panda          974             687               70%
Android 4.2/x86 emu       3621            2284               63%

"Draw time" here is the time to execute ctx.drawWindow in the reftest DoDrawWindow function. It seems to me that our best opportunity for optimizing reftest run times on x86 emu is reducing drawWindow time.


:gw280 -- Are you making any progress here? What can I do to help?

Comment 8

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

x86 emu has some optional intel acceleration components; are those being used?

Comment 9

[Geoff Brown [:gbrown]]

Attachment: log file for x86 run referenced in Comment 7

Comment 10

[George Wright (:gw280) (needinfo me!)]

I'm currently working on some blocking android/b2g bugs I'm afraid, so no real progress apart from to find that on the Motorola x86 android phone I couldn't reproduce these slow reftest issues

Comment 11

[Geoff Brown [:gbrown]]

(In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #8)
> x86 emu has some optional intel acceleration components; are those being used?

No. Are you thinking of HAXM? That's only available for Windows I believe; we are running on Linux.

Comment 12

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Yeah, HAXM.. it's available on Linux, but it looks like it requires Intel's emulator images.  Not a generic solution, it looks like.  However, we have friends at Intel -- it may be worth reaching out, because it would be a significant performance boost.

Comment 13

[Geoff Brown [:gbrown]]

:vlad's comments reminded me of kvm, so I tried setting that up and testing with kvm -- it works great! With kvm, we can run all of the reftests (not just 1 of 10 chunks) in under an hour -- comparable to desktop tests:

                           Test count       Test time (s)    Tests/second
Linux x64                     9954             1810             5.5
Android 2.2/Tegra             2591             1480             1.7
Android 4.0/Panda             1430              974             1.5
Android 4.2/x86 emu           1213             3621             0.3
Android 4.2/x86 emu+kvm       5430             1348             4.0

:vlad -- thanks so much for reminding me of acceleration options!


I don't think there is any need to do more work on this bug, but I will keep it open until kvm is deployed and rel-eng has verified that it is a viable solution.

Comment 14

[Geoff Brown [:gbrown]]

kvm is deployed and seems to be working fine. In test runs on Cedar, we run all the reftests in just 3 chunks, with each chunk completing in about 20 minutes -- awesome!