822652 - mochitest-remote now fails using on non-rooted devices using adb on /data/local/tests when TEST_PATH is set to a directory or tests require SpecialPowers

Status: NEW

(Testing :: Mochitest, defect)

Description

[Dan Mosedale (:dmosedale, :dmose)]

https://wiki.mozilla.org/Mobile/Fennec/Android#Mochitests suggests that setting TEST_PATH to "content/smil/test" should run all the tests in that directory. Instead, it just displays a generic mochitest results page on Fennec, with Passed/Failed/Todo all set to 0.

My recollection is that the last time I tried this (a month or two ago?) it worked, so I'm marking this as a regression.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

after talking with dmose on irc, the root of this problem is that the profile is pushed to /data/local/tests where there is no permissions to access it from firefox.  A single test will work just fine as we don't load the entire testrunner wrapper, likewise moving the testroot to /mnt/sdcard/tests instead of /data/local/tests will work in this situation.

Comment 2

[(not currently active) Ted Mielczarek]

I know we hit the opposite problem while testing bug 811411, in that we were pushing binaries to /mnt/sdcard/tests and failing when we couldn't chmod them +x.

If we intend to support running tests on non-rooted devices we should make things work consistently in both cases: executables should always go to /data/local/tests, and files that need to be accessible from the app should go to /mnt/sdcard/tests. Alternately, we should add code to detect whether a device is rooted and ensure that it either does the right thing or errors with a clear message if that's impossible.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

another problem is not all devices have /mnt/sdcard available.  I would prefer to not support non rooted devices, but that is not reasonable as more people want to develop, debug, and test on personal devices.  We could say that they must have a device with an available and working sdcard.

Comment 4

[Geoff Brown [:gbrown]]

(In reply to Joel Maher (:jmaher) from comment #1)
> after talking with dmose on irc, the root of this problem is that the
> profile is pushed to /data/local/tests where there is no permissions to
> access it from firefox.  

I think that worked previously. I wonder if this is because devicemanagerADB is no longer using run-as to push files in some circumstances.

Comment 5

[Dan Mosedale (:dmosedale, :dmose)]

The symptom (of the same problem) that happens when attempting to run a single test that requires SpecialPowers is that the test will attempt to execute, but fail when the in-browser JS code is unable to get a reference to the SpecialPowers object.  jmaher suggested that that might be because in the single-test case, the same setup code isn't running.

Comment 6

[Myk Melez [:myk] [@mykmelez]]

I don't see this on a mozilla-central tip build from today when running tests for dom/apps/tests, which is a directory whose tests use SpecialPowers:


  07-17 17:55 > TEST_PATH=dom/apps/tests make -C objdir-droid/ mochitest-remote
  ...
  75 INFO Passed: 70
  76 INFO Failed: 0
  77 INFO Todo:   1
  78 INFO SimpleTest FINISHED

Comment 7

[Myk Melez [:myk] [@mykmelez]]

Erm, never mind, I just realized the tests are using /mnt/sdcard/tests:

  Test root: /mnt/sdcard/tests

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

is this still a problem, I am sure we can fix/hack it to work for you in a reasonable amount of time.