632895 - Reftests on device cannot require 800x1000 window size

Status: RESOLVED FIXED

(Testing :: Reftest, defect)

Description

[cmtalbert]

Bug 615386 landed, allowing us to support testing cross-process layers in reftest.  While this is a very good thing, it breaks our ability to run reftests on device because our devices can't render a window big enough to successfully run tests. 

This is complicated by the fact that the tests themselves require a large window to run correctly, and simply running them in a smaller window, or running only the tests that "pass" may not actually test anything of note.

There are several ways forward, all of them are thorny.
* Create a mobile-only reftest suite of tests that do test properly but without the window size requirement.  This probably also means that the reftest harness will have to be updated as well so that it can change how it draws its window so that we can inform reftest to draw a "device-sized" window.
* Re-think how reftest works with its image comparisons vs window size. Find a way forward that enables us to do the same comparisons by scaling down/keeping the same aspect ratio/something.  This is more brainstorming, have no idea if that's even possible/worthwhile.
* What else?

I haven't thought about it much until just this morning, so I welcome some other thoughts here.

Comment 1

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

What resolution can our tegra2 boards drive?

Comment 2

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

(Also, to be clear, this is orthogonal to bug 623613 and bug 593874.)

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

from talos code:
http://hg.mozilla.org/build/talos/file/2198b71577c3/getInfo.html#l49

the tegra boards have this resolution:
	Screen width/height:1440/900
	colorDepth:24
	Browser inner width/height: 1024/678

according to SUT agent, we have this:
X:1366 Y:768

Comment 4

[Bob Moss :bmoss]

We have two choices to make the screen size correct. We have to image the tegras for either hdmi or vga output. And then execute the following commands.

To set the resolution to 1680x1050 with an HDMI monitor:
adb wait-for-device
adb shell setprop persist.tegra.dpy5.mode.width 1680
adb shell setprop persist.tegra.dpy5.mode.height 1050
adb shell reboot

To set the resolution to 1680x1050 with an VGA monitor:
adb wait-for-device
adb shell setprop persist.tegra.dpy3.mode.width 1680
adb shell setprop persist.tegra.dpy3.mode.height 1050
adb shell reboot

or if you have the SUTAgentAndroid installed, in a telnet window you can send the following commands.

To set the resolution to 1680x1050 with an HDMI monitor:
exec setprop persist.tegra.dpy5.mode.width 1680
exec setprop persist.tegra.dpy5.mode.height 1050
rebt

To set the resolution to 1680x1050 with an VGA monitor:
exec setprop persist.tegra.dpy3.mode.width 1680
exec setprop persist.tegra.dpy3.mode.height 1050
rebt

This yields an internal windows size 1680 x 1025 according to reftest. Also this resizing is persistent across reboots.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

as a note, I did a fresh install of a crt or hdmi image, then immediately adjusted the resolution as mentioned above.  After doing so I was unable to install fennec due to out of memory errors.  Adjusting the resolution back down (or reinstalling the os image) resolved this problem.

I have been unable to confirm that we can reflash, adjust resolution, install fennec.  My only path to success is: reflash, install fennec, adjust resolution, uninstall fennec.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

lets use this bug to track how we will adjust the resolution.  Since we have issues with the larger resolution eating up a lot of the memory that we need for a user process, we need to adjust the resolution just for reftest.

This is how I see it working:
install script:
 - installs new fennec
 - adjusts screen resolution
 - reboots

cleanup script:
 - removes testroot directory
 - adjusts down screen resolution


reftest script:
 - verifies we have the screen resolution that we want
 - *could possibly adjust it up and down*


We should decide how the install script will know to update the resolution.

Question for releng: Can we run a different install script or add a parameter to the existing script when running reftest?

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

in bug 634691, there are devicemanager.py changes to facilitate resolution changes.

Here is a copy of installApp.py that will adjust the resolution (I don't know where the hg repository is that installApp lives in):
http://people.mozilla.org/~jmaher/installApp.py

http://people.mozilla.org/~jmaher/installApp.py

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: check for invalid resolution when running tests on remote devices (1.0)

simple patch for m-c to fix remotereftests.py to exit with an error if the resolution is too small on the device.

Comment 9

[[e-mail] (:bhearsum)]

Comment on attachment 514019 [details] [diff] [review]
check for invalid resolution when running tests on remote devices (1.0)

This looks fine to me, but I don't think I'm the best reviewer.

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

http://hg.mozilla.org/mozilla-central/pushloghtml?changeset=c3cd62743efe

Comment 11

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

Comment on attachment 514019 [details] [diff] [review]
check for invalid resolution when running tests on remote devices (1.0)

>+    if (width < 1050 or height < 1050):
>+        print "ERROR: Invalid screen resolution %sx%s, please adjust to 1366x1050 or higher" % (width, height)
>+        sys.exit(1)
>+

Either the code or the error message is wrong here. You ask to adjust to 1366x1050, but you check for 1050x1050.

Comment 12

[Joel Maher ( :jmaher ) (UTC -8)]

we need 1050 minimum and the minimum resolution on a tegra device (depending on crt vs hdmi) is 1366x1050 (normally much higher).  I check for what will satisfy the test and the error message is what is possible on the tegra device.