789301 - Add gesture support to eideticker and get tests running with runtest/update-dashboard

Status: RESOLVED FIXED

(Testing Graveyard :: Eideticker, defect)

Description

[Malini Das [:mdas] - Away, not checking bugmail]

We only have mouse support for b2g right now and we're waiting on a dummy input driver, so it's much easier to send an already crafted sendevent file to drive the actions instead of actions.json for now.

Once we have the gestures, we can then drive the tests in runtest.py with it, and modify the dashboard code so the information is visible.

Comment 1

[Malini Das [:mdas] - Away, not checking bugmail]

Attachment: Add gesture support

This patch adds gestures support and the gesture files required for the tests.

Comment 2

[Malini Das [:mdas] - Away, not checking bugmail]

Attachment: Allow b2g pandaboard tests to be run and viewed with dashboard

This patch adds update-dashboard.sh support for the b2g tests. I've added a run-update-dashboard-b2g.sh script since it must be called with different parameters, and the zoom tests don't work with the mouse (Bug 783023, mwu will focus on getting the test input driver instead of patching the mouse)

fyi, I changed:
", ".join([product["name"] for product in default_products]))

to:
", ".join([p["name"] for p in default_products]))

since the 'product' variable gets leaked outside of the list comprehension, so I changed it to something unused in our script

Comment 3

[Malini Das [:mdas] - Away, not checking bugmail]

These patches depend on the B2GManager landing in mozdevice

Comment 4

[William Lachance (:wlach)]

Comment on attachment 659014 [details] [diff] [review]
Allow b2g pandaboard tests to be run and viewed with dashboard

Ok, so a promising start, but I think we need a few more iterations before we can get this right...

>diff --git a/bin/run-update-dashboard-b2g.sh b/bin/run-update-dashboard-b2g.sh
>new file mode 100755
>index 0000000..5c9b33e
>--- /dev/null
>+++ b/bin/run-update-dashboard-b2g.sh
...

I don't really see the need for this. The whole idea is that you can configure the parameters by setting environment variables. For example, here's what I'm using for 
the galaxy nexus device:

export DEVICE_ID=samsung-gn
export TEST_DEVICE=10.250.120.71
export DM_TRANS=sut
export TESTS="nightly cnn clock wikipedia taskjs"

I store those settings in a file called `~/bin/device-env` then call the run-update-dashboard script using a script like this (also in the ~/bin directory):

#!/bin/sh

. $HOME/bin/device-env

$HOME/src/eideticker/bin/run-update-dashboard.sh

My eventual goal is to have proper configuration for all this stuff (probably using mozharness) but in the mean time this works pretty well for me. ;)

>+    if device_id == 'Panda' and (options.product and options.product == 'b2g'):
>+        b2g_manager = B2GManager(device)
>+        appinfo = b2g_manager.get_appinfo()
>+        appinfo["buildid"] = appinfo["appBuildID"]
>+        appinfo["revision"] = appinfo["version"]
>+        devices[device_id] = { 'name': device.model,
>+                               'version': appinfo['version'] }
>+    else:
>+        devices[device_id] = { 'name': device.model,
>+                                   'version': device.getprop('ro.build.version.release') }

We should not assume anything based on the device_id. The whole idea is that it can be arbitrary. Is there any reason you're checking for it?

>     for product in products:
>-        if product.get('url'):
>-            product_fname = os.path.join(DOWNLOAD_DIR, "%s.apk" % product['name'])
>-            appinfo = eideticker.get_fennec_appinfo(product_fname)
>-            appname = appinfo['appname']
>-            print "Using application name '%s' from apk '%s'" % (appname, product_fname)
>+        if product['name'] == 'b2g':
>             capture_name = "%s %s" % (product['name'], appinfo['date'])
>-        else:
>-            appinfo = { }
>-            appname = product['appname']
>-            capture_name = "%s (taken on %s)" % (product['name'], current_date)
> 
>-        if appinfo.get('appname'):
>-            appname = appinfo['appname']
>+            def call_runtest():
>+                 runtest(device, product, current_date, appinfo, test,
>+                         capture_name + " #%s" % i, outputdir, datafile, data,
>+                         gesture_file=options.gesture_file,
>+                         enable_profiling=options.enable_profiling)
>+
>         else:
>-            appname = product['appname']
>+            if product.get('url'):
>+                product_fname = os.path.join(DOWNLOAD_DIR, "%s.apk" % product['name'])
>+                appinfo = eideticker.get_fennec_appinfo(product_fname)
>+                appname = appinfo['appname']
>+                capture_name = "%s %s" % (product['name'], appinfo['date'])
>+            else:
>+                appinfo = { }
>+                appname = product['appname']
>+                capture_name = "%s (taken on %s)" % (product['name'], current_date)
>+
>+            if appinfo.get('appname'):
>+                appname = appinfo['appname']
>+            else:
>+                appname = product['appname']
>+
>+                def call_runtest():
>+                    # Kill any existing instances of the processes
>+                    device.killProcess(appname)
>+
>+                    # Now run the test
>+                    runtest(device, product, current_date, appinfo, test,
>+                            capture_name + " #%s" % i, outputdir, datafile, data,
>+                            appname=appname, enable_profiling=options.enable_profiling)
>+
>+                    # Kill app after test complete
>+                    device.killProcess(appname)
> 
>         # Run the test the specified number of times
>         for i in range(num_runs):
>-            # Kill any existing instances of the processes
>-            device.killProcess(appname)
>-
>             # Now run the test
>-            runtest(device, product, current_date, appname, appinfo, test,
>-                    capture_name + " #%s" % i, outputdir, datafile, data,
>-                    enable_profiling=options.enable_profiling)
>-
>-            # Kill app after test complete
>-            device.killProcess(appname)
>+            call_runtest()

I'm feeling like this code is way too complicated. Is it really required to define an inline method called call_runtest? Couldn't we just do some kind of conditional? Also, I feel like we should be able to unify the code for setting the capture name somehow.

>diff --git a/src/eideticker/eideticker/device.py b/src/eideticker/eideticker/device.py
>index f81d6f0..8533521 100644
>--- a/src/eideticker/eideticker/device.py
>+++ b/src/eideticker/eideticker/device.py
>@@ -21,7 +21,7 @@ DEVICE_PROPERTIES = {
>     },
>     "Panda": {
>         "hdmiResolution": "720p",
>-        "inputDevice": "/dev/input/event0",
>+        "inputDevice": "/dev/input/event2",
>         "dimensions": (1280, 672)

As discussed on IRC we somehow need to make this work for a Panda both for Android and B2G.

Comment 5

[William Lachance (:wlach)]

Comment on attachment 659010 [details] [diff] [review]
Add gesture support

As discussed on IRC, I think we're going to try to make orangutan do what we need here.

Comment 6

[Malini Das [:mdas] - Away, not checking bugmail]

Attachment: Allow b2g pandaboard tests to be run and viewed with dashboard v2.0

This is the work done so far to get the eideticker tests running with b2g.

You can run it with runtests.py --b2g, but if you'd like to use run-update-dashboard.sh, then you'll need to export the PRODUCT and DEVICE_ID fields:
export PRODUCT=b2g
export DEVICE_ID=Panda

then run 'sh bin/run-update-dashboard.sh'

Comment 7

[William Lachance (:wlach)]

Modified versions of these patches went into Eideticker and the default test set now works with a bunch of caveats. Gonna mark this as fixed.

Comment 8

[William Lachance (:wlach)]

Comment on attachment 670083 [details] [diff] [review]
Allow b2g pandaboard tests to be run and viewed with dashboard v2.0

An updated version of this patch already went in.