610493 - Run reftests on Windows 7 with no hardware acceleration (GDI/BasicLayers only)

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Joe Drew (not getting mail)]

We'd like some form of test coverage with no Direct2D and no Direct3D at all. We think that just having reftests run in this configuration would be enough.

To do this, the following prefs have to be set accordingly:

 * layers.accelerate-none = true
 * gfx.direct2d.disabled = true

Please do the needful!

Comment 1

[Bas Schouten (:bas.schouten)]

I'm not sure, but maybe GDI/Direct3D 9 is a more common(and therefor more useful) setting?

Comment 2

[Armen [:armenzg]]

Bas, Joe what is the final desired setting?
For when do you guys need this to be in production?
Could you guys bring this up on the next Tuesday meeting and get approval that we should duplicate the running reftests?
To be explicit, would this be only for Windows? For optimized and/or debug?
Can we disable both preference in one suite (both pref) or two suites (each suite a different pref disabled)?
BTW how easy or difficult is for users to enable/disable this pref without going to about:config? I believe that hardware acceleration can be disabled in "options..." through the UI (so that one seems easy to disable).

@releng: this bug requires running reftests with acceleration disabled AND with D2D disabled.

In the following code, we added the case for running "with" acceleration because at that time acceleration and d2d were disabled by default (which is not the current state):
http://hg.mozilla.org/build/buildbotcustom/file/tip/steps/unittest.py#l717

To add these new tests we should add them by doing this:
  'win7': {
-   'opt_unittest_suites' : UNITTEST_SUITES['opt_unittest_suites'][:],
+   'opt_unittest_suites' : UNITTEST_SUITES['opt_unittest_suites'][:] +
+        [('reftest-no-d2d-d3d', ['reftest-no-d2d'])]

See bug 595237, bug 549120 and bug 581213.

Comment 3

[Joe Drew (not getting mail)]

We don't need this immediately, but before the final Firefox beta would be perfect.

What we want in the end is to have one extra reftest run on Windows 7, software only, with the settings specified in comment 0.

If we could have all of the things we wanted in the world, we'd take DirectX 9 and GDI on Windows 7 as another reftest run, but that's not necessary.

It's very easy for users to enable/disable this pref without going to about:config, as you noticed; it's in the Advanced preferences dialog. (Specifically, toggling that checkbox is exactly equivalent to the settings in comment 0.)

Comment 4

[Lukas Blakk [:lsblakk] use ?needinfo]

So if we are going to treat this request like a new test suite we should enable it on for Try first and see how it runs there (reliable results) before turning it on for mozilla-central.  If adding them only requires what Armen mentions in comment 2 then it shouldn't take much to land this on Try branch early next week and you can push to Try to check that you're getting what you want.  

Joe: is this for opt _and_ debug or just one of them? Would this test suite become a permanent suite going forward?

Comment 5

[Joe Drew (not getting mail)]

This could be only for opt, but yes, it'd be a permanent test suite.

Comment 6

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: add no-d2d test to try win7

checked in staging that this gets the builder.

Comment 7

[Armen [:armenzg]]

Comment on attachment 495678 [details] [diff] [review]
add no-d2d test to try win7

If the purpose is to have it just for the tryserver, yes, r+

You also need set the prefs in:
http://hg.mozilla.org/build/buildbotcustom/file/tip/steps/unittest.py#l719

You can probably change "reftest-d2d" to "reftest-no-d2d-d3d" or "reftest-no-d2d" (I am not sure which of the top of my head). The naming I gave you might not the best. What do you think of "reftest-no-acceleration"?

Did this work without the patch I think you need? Could you paste the full command to run the actual test?

Comment 8

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: adds reftest-no-d2d-d3d to unittest.py

yup, this is needed - ran it in staging on a win7 slave successfully.

Comment 9

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: config.py add reftest-no-accel test to tryserver win7 opt tests

using this one instead for a nicer public-facing name

Comment 10

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: adds reftest-no-d2d-d3d to unittest.py and test factory

oops - forgot to actually _run_ the test :)

Comment 11

[Lukas Blakk [:lsblakk] use ?needinfo]

> If the purpose is to have it just for the tryserver, yes, r+

Yes, we should only run this on tryserver for a day or two so that we can ensure it's safe to turn this on for mozilla-central (Joe or someone in gfx should push a few try builds to it) 

> Could you paste the full command to run the actual test?

python reftest/runreftest.py --appname=firefox/firefox.exe --utility-path=bin --extra-profile-file=bin/plugins --symbols-path=symbols --setpref=gfx.direct2d.disabled=true --setpref=layers.accelerate-none=true reftest/tests/layout/reftests/reftest.list

Comment 12

[Lukas Blakk [:lsblakk] use ?needinfo]

Joe - here are the results from the test run I did.  Before I land this on tryserver can you check to see if anything else needs to be set or if this is expected orange?  http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1291740230.1291742361.17600.gz

Comment 13

[Joe Drew (not getting mail)]

Yikes!

We didn't expect these oranges, but I doubt you need to do anything - we need to do some debugging.

Comment 14

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 495843 [details] [diff] [review]
config.py add reftest-no-accel test to tryserver win7 opt tests

http://hg.mozilla.org/build/buildbot-configs/rev/531efa39e54d

Comment 15

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 495844 [details] [diff] [review]
adds reftest-no-d2d-d3d to unittest.py and test factory

http://hg.mozilla.org/build/buildbotcustom/rev/59f35f922f09

Comment 16

[Lukas Blakk [:lsblakk] use ?needinfo]

Ok - this is landed on try so you can ask for it with -u reftest-no-accel  and see the results at http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTry&noignore=1 (the column is hidden for others)

Comment 17

[Lukas Blakk [:lsblakk] use ?needinfo]

I'm resolving this bug since you are able to run the reftests.  When the time comes that this is ready for mozilla-central please file a new test suite request bug on Release Engineering.