746260 - disable the screen resolution changing on android for jsreftest and crashtest, leave it on for reftest

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P1)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

the jsreftests and crashtests do not require us to compare canvas images that use 800x1000.  Right now in our sut_tools, we do a blanket resolution change for all reftest harness tests.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

when doing this, we would need to add the '--ignore-window-size' argument to the reftest command line arguments.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: sut_tools - only adjust the resolution for reftest (1.0)

simple patch to only adjust/verify resolution for reftest, not crashtest or jsreftest.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

the only other thing needed is to add '--ignore-window-size' to the arguments for jsreftest and crashtest.

Comment 4

[Mike Taylor [:bear]]

Attachment: add --ignore-window-size to jsreftest and creashtest

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 616961 [details] [diff] [review]
add --ignore-window-size to jsreftest and creashtest

Review of attachment 616961 [details] [diff] [review]:
-----------------------------------------------------------------

in general these look like desktop unittests?  Is there a way to get this in the mobile-reftest-mixin?

::: steps/unittest.py
@@ +341,3 @@
>          elif suite == 'crashtest-ipc':
>              return ['--setpref=browser.tabs.remote=true',
> +                    '--ignore-window-size',

we don't run crashtest-ipc on mobile.

@@ +365,4 @@
>                      'jsreftest/tests/jstests.list']
>          elif suite == 'reftest-sanity':
> +            return ['--ignore-window-size',
> +                    'reftest/tests/layout/reftests/reftest-sanity/reftest.list']

we shouldn't change this.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: add --ignore-window-size to remote jsreftest/crashtest commandline (1.0)

my attempt at this.

Comment 7

[Mike Taylor [:bear]]

Comment on attachment 617034 [details] [diff] [review]
add --ignore-window-size to remote jsreftest/crashtest commandline (1.0)

looks good, but what is the piece that is showing just above your patch?

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

oh interesting...maybe I am editing the wrong place.  I don't see --ignore-window-size when looking at the logs.  Maybe next week we can decipher this.

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

in both repositories: buildbotcustom and buildbot-configs, I am unable to figure out how we run remotereftest.py without the --ignore-window-size.  When I look on tbpl, it indicates that we are not running with that cli flag, but in unittest.py it looks like we are:
http://mxr.mozilla.org/build/source/buildbotcustom/steps/unittest.py#965

I suspect either of these two options to explain my confusion:
1) I am not looking in the correct repository
2) What is checked in is NOT running on the foopies

Comment 10

[Phil Ringnalda (:philor)]

catlee pointed out that what matters is not mxr...unittest.py, but http://hg.mozilla.org/build/buildbotcustom/annotate/production-0.8/steps/unittest.py#l965 - apparently --ignore-window-size got backed out on production in http://hg.mozilla.org/build/buildbotcustom/diff/8967863f12b6/steps/unittest.py but not on default.

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: remove global --ignore-window-size and add option for crashtest/jsreftest (1.0)

revert the change to the repo to have --ignore-window-size on by default.  Add this option in for jsreftest and crashtests.

Comment 12

[Armen [:armenzg]]

Comment on attachment 617887 [details] [diff] [review]
remove global --ignore-window-size and add option for crashtest/jsreftest (1.0)

This looks good. Thanks jmaher.

Comment 13

[Armen [:armenzg]]

jmaher, what should I be looking for?
http://dev-master01.build.scl1.mozilla.com:8043/one_line_per_build

I have few left jobs running that were run by bad tegras.

Comment 14

[Joel Maher ( :jmaher ) (UTC -8)]

I am looking for all green jobs.  

It looks like the jobs are failing for other reasons (mostly the updatesut.py step).  All we need is 1 cycle of green to prove this works.  Preferably with an updated mozilla-central build.

Comment 15

[Armen [:armenzg]]

I am re-testing this again together with the sut agent 1.08 upgrade.
I should have results tomorrow morning.

Comment 16

[Armen [:armenzg]]

jmaher is going to look at the green jobs in here:
http://tinderbox.mozilla.org/showbuilds.cgi?tree=MobileTest

Comment 17

[Armen [:armenzg]]

jmaher checked this and I believe he gave clearance to deploy. Can you please confirm?

I would like to deploy this or bug 750744 tomorrow and do the other one on Monday.
I think this one is easier to recover in case anything goes wrong and the other one gives us Tuesday to recover if we need too.

Comment 18

[Joel Maher ( :jmaher ) (UTC -8)]

confirmed!

Comment 19

[Armen [:armenzg]]

reftest [1] did increase the resolution while jsreftest [2] did not.
These two steps differ in 210 seconds which is 3.5 minutes :)

[1]
python /builds/sut_tools/config.py 10.250.49.16 reftest
...
connecting to: 10.250.49.16
reconnecting socket
devroot /mnt/sdcard/tests
results: {'screen': ['X:1024 Y:768']}
current resolution X:1024 Y:768
/builds/tegra-029/test/../proxy.flg
results: {'os': ['harmony-eng 2.2 FRF91 20110202.102810 test-keys']}
results: {'screen': ['X:1024 Y:768']}
INFO: we have a current resolution of 1024, 768
INFO: adjusting screen resolution to 1600, 1200 and rebooting
DEBUG: gCallbackData is:  on port: 50029
Creating server with 10.250.48.200:50029
Calling disconnect on callback server
. Got data back from agent: System rebooted


Shutting down server now
System rebooted


Waiting for tegra to come back...
Try 1
reconnecting socket
devroot /mnt/sdcard/tests
results: {'screen': ['X:1600 Y:1200']}
current resolution X:1600 Y:1200
program finished with exit code 0
elapsedTime=210.825854

[2] 
python /builds/sut_tools/config.py 10.250.49.16 jsreftest
...
connecting to: 10.250.49.16
reconnecting socket
devroot /mnt/sdcard/tests
results: {'screen': ['X:1024 Y:768']}
current resolution X:1024 Y:768
program finished with exit code 0
elapsedTime=0.262667

Comment 21

[Armen [:armenzg]]

Comment on attachment 616757 [details] [diff] [review]
sut_tools - only adjust the resolution for reftest (1.0)

http://hg.mozilla.org/build/tools/rev/5b54bae472ab

Comment 22

[Armen [:armenzg]]

Comment on attachment 617887 [details] [diff] [review]
remove global --ignore-window-size and add option for crashtest/jsreftest (1.0)

http://hg.mozilla.org/build/buildbotcustom/rev/e0c0fd23f72a

Comment 23

[Armen [:armenzg]]

This is now live.

We might see some jobs fail because I had to clobber the tools checkout on the foopies and restore the symlinks.

Comment 24

[Justin Wood (:Callek)]

Armen please confirm this is deployed everywhere, I was checking on foopy23 for an unrelated problem on tegra-266 (verify.py unknown errors) and I noticed that tools/ is outdated here.

I'm not sure if this is one-off or everywhere...

foopy23:~ cltbld$ hg -R /builds/tools in
comparing with http://hg.mozilla.org/build/tools
searching for changes
changeset:   2530:513ae0365f85
user:        Aki Sasaki <[e-mail]>
date:        Wed May 09 17:20:00 2012 -0700
summary:     Added tag FENNEC_14_0b1_RELEASE, FENNEC_14_0b1_BUILD2 for changeset c0122cac3804

changeset:   2531:cfe229dff313
tag:         FENNEC_14_0b1_BUILD3
tag:         FENNEC_14_0b1_RELEASE
user:        Ben Hearsum <[e-mail]>
date:        Fri May 11 03:18:25 2012 +0800
summary:     Remove now-dead sjc1 masters from production-masters.json; fix datacentre attribute on
new schedulers. r=catlee

changeset:   2532:77f3f141ad0b
user:        Aki Sasaki <[e-mail]>
date:        Thu May 10 16:28:56 2012 -0700
summary:     Added tag FENNEC_14_0b1_RELEASE, FENNEC_14_0b1_BUILD3 for changeset cfe229dff313

changeset:   2533:18227ad5ca08
user:        John Hopkins <[e-mail]>
date:        Fri May 11 08:48:19 2012 -0400
summary:     Do thunderbird release builds on bm34 (Bug 748157 - Add Thunderbird to production-*.jso
n). r=catlee

changeset:   2534:573d357258bb
user:        John Hopkins <[e-mail]>
date:        Fri May 11 09:36:20 2012 -0400
summary:     Enable bm34. r=bhearsum

changeset:   2535:5b54bae472ab
tag:         THUNDERBIRD_13_0b2_BUILD1
user:        Joel Maher <[e-mail]>
date:        Fri May 11 10:43:03 2012 -0400
summary:     Bug 746260 - disable the screen resolution changing on android for jsreftest and crasht
est, leave it on for reftest. r=bear

changeset:   2536:e3e998673505
tag:         THUNDERBIRD_13_0b2_RELEASE
user:        John Hopkins <[e-mail]>
date:        Fri May 11 12:12:36 2012 -0400
summary:     Added tag THUNDERBIRD_13_0b2_BUILD1 for changeset 5b54bae472ab

changeset:   2537:8475773f8255
user:        John Hopkins <[e-mail]>
date:        Fri May 11 12:12:41 2012 -0400
summary:     Added tag THUNDERBIRD_13_0b2_RELEASE for changeset e3e998673505

changeset:   2538:f258181830f1
user:        Armen Zambrano Gasparnian <[e-mail]>
date:        Fri May 11 12:26:52 2012 -0400
summary:     Bug 754297. add sys.stdout.flush() to sut_tools' scripts. r=bear

changeset:   2539:fdf190203a10
user:        Justin Wood <[e-mail]>
date:        Fri May 11 15:04:59 2012 -0400
summary:     Bug 747641 - tegra-224 problem tracking. rs+=bear

changeset:   2540:c98cc48b759e
tag:         FENNEC_13_0b2_BUILD2
tag:         FENNEC_13_0b2_RELEASE
user:        Justin Wood <[e-mail]>
date:        Fri May 11 16:52:32 2012 -0400
summary:     No Bug - List tegra-172 in Unassigned block of foopies.sh

changeset:   2541:540c331a82b0
tag:         tip
user:        Aki Sasaki <[e-mail]>
date:        Fri May 11 16:05:31 2012 -0700
summary:     Added tag FENNEC_13_0b2_BUILD2, FENNEC_13_0b2_RELEASE for changeset c98cc48b759e

foopy23:~ cltbld$

Comment 25

[Armen [:armenzg]]

This was my bad.
I always thought we only had up to foopy22.
I updated the tools repo and it is updated to the same revision as the others (f258181830f1).


The only other foopy I did not touch/clobber was foopy05 knowing that it was a staging one and you had changes in there. IIRC I let you know this on IRC.