Closed Bug 610493 Opened 11 years ago Closed 11 years ago

Run reftests on Windows 7 with no hardware acceleration (GDI/BasicLayers only)

Categories

(Release Engineering :: General, defect)

x86
Windows 7
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: joe, Assigned: lsblakk)

References

Details

(Whiteboard: [testing][reftest][gfx])

Attachments

(2 files, 2 obsolete files)

We'd like some form of test coverage with no Direct2D and no Direct3D at all. We think that just having reftests run in this configuration would be enough.

To do this, the following prefs have to be set accordingly:

 * layers.accelerate-none = true
 * gfx.direct2d.disabled = true

Please do the needful!
I'm not sure, but maybe GDI/Direct3D 9 is a more common(and therefor more useful) setting?
Whiteboard: [testing][reftest][gfx]
Bas, Joe what is the final desired setting?
For when do you guys need this to be in production?
Could you guys bring this up on the next Tuesday meeting and get approval that we should duplicate the running reftests?
To be explicit, would this be only for Windows? For optimized and/or debug?
Can we disable both preference in one suite (both pref) or two suites (each suite a different pref disabled)?
BTW how easy or difficult is for users to enable/disable this pref without going to about:config? I believe that hardware acceleration can be disabled in "options..." through the UI (so that one seems easy to disable).

@releng: this bug requires running reftests with acceleration disabled AND with D2D disabled.

In the following code, we added the case for running "with" acceleration because at that time acceleration and d2d were disabled by default (which is not the current state):
http://hg.mozilla.org/build/buildbotcustom/file/tip/steps/unittest.py#l717

To add these new tests we should add them by doing this:
  'win7': {
-   'opt_unittest_suites' : UNITTEST_SUITES['opt_unittest_suites'][:],
+   'opt_unittest_suites' : UNITTEST_SUITES['opt_unittest_suites'][:] +
+        [('reftest-no-d2d-d3d', ['reftest-no-d2d'])]

See bug 595237, bug 549120 and bug 581213.
We don't need this immediately, but before the final Firefox beta would be perfect.

What we want in the end is to have one extra reftest run on Windows 7, software only, with the settings specified in comment 0.

If we could have all of the things we wanted in the world, we'd take DirectX 9 and GDI on Windows 7 as another reftest run, but that's not necessary.

It's very easy for users to enable/disable this pref without going to about:config, as you noticed; it's in the Advanced preferences dialog. (Specifically, toggling that checkbox is exactly equivalent to the settings in comment 0.)
So if we are going to treat this request like a new test suite we should enable it on for Try first and see how it runs there (reliable results) before turning it on for mozilla-central.  If adding them only requires what Armen mentions in comment 2 then it shouldn't take much to land this on Try branch early next week and you can push to Try to check that you're getting what you want.  

Joe: is this for opt _and_ debug or just one of them? Would this test suite become a permanent suite going forward?
This could be only for opt, but yes, it'd be a permanent test suite.
Attached patch add no-d2d test to try win7 (obsolete) — Splinter Review
checked in staging that this gets the builder.
Assignee: nobody → lsblakk
Status: NEW → ASSIGNED
Attachment #495678 - Flags: review?(armenzg)
Comment on attachment 495678 [details] [diff] [review]
add no-d2d test to try win7

If the purpose is to have it just for the tryserver, yes, r+

You also need set the prefs in:
http://hg.mozilla.org/build/buildbotcustom/file/tip/steps/unittest.py#l719

You can probably change "reftest-d2d" to "reftest-no-d2d-d3d" or "reftest-no-d2d" (I am not sure which of the top of my head). The naming I gave you might not the best. What do you think of "reftest-no-acceleration"?

Did this work without the patch I think you need? Could you paste the full command to run the actual test?
yup, this is needed - ran it in staging on a win7 slave successfully.
Attachment #495842 - Flags: review?(armenzg)
using this one instead for a nicer public-facing name
Attachment #495678 - Attachment is obsolete: true
Attachment #495843 - Flags: review?(armenzg)
Attachment #495678 - Flags: review?(armenzg)
oops - forgot to actually _run_ the test :)
Attachment #495842 - Attachment is obsolete: true
Attachment #495844 - Flags: review?(armenzg)
Attachment #495842 - Flags: review?(armenzg)
> If the purpose is to have it just for the tryserver, yes, r+

Yes, we should only run this on tryserver for a day or two so that we can ensure it's safe to turn this on for mozilla-central (Joe or someone in gfx should push a few try builds to it) 

> Could you paste the full command to run the actual test?

python reftest/runreftest.py --appname=firefox/firefox.exe --utility-path=bin --extra-profile-file=bin/plugins --symbols-path=symbols --setpref=gfx.direct2d.disabled=true --setpref=layers.accelerate-none=true reftest/tests/layout/reftests/reftest.list
Attachment #495844 - Flags: review?(armenzg) → review+
Attachment #495843 - Flags: review?(armenzg) → review+
Joe - here are the results from the test run I did.  Before I land this on tryserver can you check to see if anything else needs to be set or if this is expected orange?  http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1291740230.1291742361.17600.gz
Yikes!

We didn't expect these oranges, but I doubt you need to do anything - we need to do some debugging.
Comment on attachment 495843 [details] [diff] [review]
config.py add reftest-no-accel test to tryserver win7 opt tests

http://hg.mozilla.org/build/buildbot-configs/rev/531efa39e54d
Attachment #495843 - Flags: checked-in+
Comment on attachment 495844 [details] [diff] [review]
adds reftest-no-d2d-d3d to unittest.py and test factory

http://hg.mozilla.org/build/buildbotcustom/rev/59f35f922f09
Attachment #495844 - Flags: checked-in+
Ok - this is landed on try so you can ask for it with -u reftest-no-accel  and see the results at http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTry&noignore=1 (the column is hidden for others)
I'm resolving this bug since you are able to run the reftests.  When the time comes that this is ready for mozilla-central please file a new test suite request bug on Release Engineering.
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Depends on: 713422
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.