1486004 - Support unrooted Android devices in our Unit tests

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Testing, enhancement, P3)

Description

[Bob Clary [:bc] (inactive)]

adb*.py and friends in mozdevice currently are designed to run tests using rooted devices. With recent work, it may be possible to narrow down the features which require root and to work with our Android developers to implement replacement functionality in Fennec, Geckoview, etc. which would allow us to run our unit tests with an unrooted device.

Edwin: This might be something to get started with. It might not be solvable but it will give you some familiarity with running the unit tests on Android and might actually be possible these days.

One of the features of a Magisk rooted Pixel2 is you can easily turn rooting on and off and test running unit tests on both rooted and unrooted devices using the same device.

Comment 1

[Bob Clary [:bc] (inactive)]

Spoke with snorp and he recommended run-as <app> instead of adding intents to the test apps. Keeping data and programs in the directories owned by the test apps and using run-as might be the solution we need.

Comment 2

[Nick Alexander :nalexander [he/him]]

(In reply to Bob Clary [:bc] from comment #1)

Spoke with snorp and he recommended run-as <app> instead of adding intents to the test apps. Keeping data and programs in the directories owned by the test apps and using run-as might be the solution we need.

If we can make this work, run-as would be my preference as well. However, I have had device-specific interactions with the shell user and the App user under run-as. IIRC, some devices make it nigh on impossible to move files around the filesystem between users -- I often had to resort to catting from stdin to write into /data/app/$PACKAGE (under run-user), which runs into shell issues in older Androids. Hopefully that's all addressed now! Good luck, excited to see better support for unrooted devices!

Comment 3

[Bob Clary [:bc] (inactive)]

I spent some time this weekend looking into this and I have wip for reftest (reftest, crashtest, jstestbrowser), mochitest, xpcshell, jstests and am looking into raptor. Raptor has a number of accesses to system data which will prevent it from running without root unless we can find work arounds for them and it appears to me that raptor doesn't run with the geckoview test app. I am very pleased with the progress so far and hope to have something to show shortly.

Comment 4

[Nick Alexander :nalexander [he/him]]

(In reply to Bob Clary [:bc] from comment #3)

I spent some time this weekend looking into this and I have wip for reftest (reftest, crashtest, jstestbrowser), mochitest, xpcshell, jstests and am looking into raptor.

Yay! Really pleased to hear this, Bob.

Raptor has a number of accesses to system data which will prevent it from running without root unless we can find work arounds for them and it appears to me that raptor doesn't run with the geckoview test app.

Raptor targets many different things: GVE and/or TR (can't recall exactly which), but also Fenix, Fennec, the Reference Browser. In any case, running Raptor unrooted with full functionality shouldn't block the unit/functional tests, because generally these performance tests can't really be relied upon locally anyway. I think that the browsertime replacements to Raptor (confusingly homed in the same harness) can be trusted, but those have never been restricted to rooted devices, AFAIK.

In all: great to hear this!

Comment 5

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - WIP Use run-as to eliminate rooting requirement for devices.

Comment 6

[Bob Clary [:bc] (inactive)]

With the exception of browsertime, invoking tests locally via mach looks good with respect to org.mozilla.geckoview(_example).test on my Pixel 2/Android 10 but not in production with Moto G5/Android 7 and Pixel 2/Android 8.0. Production is using the same version of adb which I have locally so that is not the cause. I am thinking that this may be due to the different versions of run-as and perhaps the storage permission handling in the different versions of Android. I'll investigate using Android emulators and if that doesn't help, I'll begin the process of reflashing my Pixel 2 back to earlier versions of Android. If we can only get this working on more recent versions of Android that will be an improvement but not the win I was hoping to achieve.

Comment 7

[[ex-Mozilla] Agi Sferro | :agi]

Is this going to land soon? The mozregression from pip stopped working too now, so we have no way to bisect on non-rooted devices.

Comment 8

[Bob Clary [:bc] (inactive)]

I'm working through it now. It'll land when it's ready. Hopefully this week.

Comment 9

[Bob Clary [:bc] (inactive)]

I have updated phabricator revision D79650.

This is not yet ready for review but I believe the approach is the final one. I've kept everything in one large patch for now to facilitate development and testing. This way all you need to do is land this one patch in your tree in order to test.

I have tested locally with a Pixel 2 running Android 10 using magisk with root enabled and without root enabled as well as x86 emulators running Android 11 with Google API (with root) and Android 11 with Google Play (without root) for xpcshell-tests jstests jit-test reftest reftest-fission crashtest crashtest-fission jstestbrowser jstestbrowser-fission mochitest mochitest-fission raptor raptor-fission raptor-power raptor-cpu browsertime geckoview-junit gtest.

Note that for Raptor and its access of /data/anr, /data/tombstones and other restricted paths should fail gracefully without root.

On Try:

• try production android em 7.0 android-hw-p2 8.0 both with root enabled
• try bitbar test devices android-hw-p2 9.0 with root enabled

This currently has forced enabled verbose debug logging which I kind of like though I expect most people will not appreciate it. I need to figure out the proper method of controlling the logging via the command line.

Since the test root is no longer freely configurable, I think I will remove all command line arguments which allow for specification of the remote test root for Android devices and convert the existing code to allow me to pass the appropriate log level and verbosity arguments to the logger used in adb.py which has always been lacking.

ahal, gbrown: I would very much appreciate your comments regarding the removal of the remote test root command line arguments as well as the proper means of getting the new log level and verbosity arguments into mozdevice.

I would appreciate it if anyone who has an Android development environment with mk_add_options MOZ_PACKAGE_JSSHELL=1 enabled and could help to test with a variety of emulators or real devices whether rooted or not. The minimum Android version we run in production is Android 4.3 SDK 18 but I believe we will be increasing the minimum soon. The Fenix minimum is Android 5.0 SDK 21. Any device Android 5.0 or higher would be great.

Comment 10

[Greg Mierzwinski [:sparky]]

(In reply to Bob Clary [:bc] from comment #9)

I would appreciate it if anyone who has an Android development environment with mk_add_options MOZ_PACKAGE_JSSHELL=1 enabled and could help to test with a variety of emulators or real devices whether rooted or not. The minimum Android version we run in production is Android 4.3 SDK 18 but I believe we will be increasing the minimum soon. The Fenix minimum is Android 5.0 SDK 21. Any device Android 5.0 or higher would be great.

:acreskey, would you be able to help :bc with the testing and/or find people who can?

Comment 11

[Andrew Creskey [:acreskey]]

(In reply to Greg Mierzwinski [:sparky] from comment #10)

(In reply to Bob Clary [:bc] from comment #9)

I would appreciate it if anyone who has an Android development environment with mk_add_options MOZ_PACKAGE_JSSHELL=1 enabled and could help to test with a variety of emulators or real devices whether rooted or not. The minimum Android version we run in production is Android 4.3 SDK 18 but I believe we will be increasing the minimum soon. The Fenix minimum is Android 5.0 SDK 21. Any device Android 5.0 or higher would be great.

:acreskey, would you be able to help :bc with the testing and/or find people who can?

Yes, happy to help.

My primary use-case for running without root is the ./mach perftest tests.
For which I logged this bug, https://bugzilla.mozilla.org/show_bug.cgi?id=1634531

But I grabbed the latest m-c and I can now run these tests locally on un-rooted devices.

So if there are other tests that you'd like me to try, just let me know.
I have the following unrooted devices: Moto G5, Pixel 3, Pixel 4, and a few Samsungs.

Comment 12

[Bob Clary [:bc] (inactive)]

That's great news. aerickson had problems running reftest on a Samsung S10g.

While considering why I realized while testing and switching back and forth between su enabled or disabled using magisk I was testing without root but with SELinux Permissive. On my Pixel 2 once I setenforce 1and then disable su I can no longer read the state of SELinux and get the same failure aerickson did.

Can you run ./mach reftest --this-chunk 1 --total-chunks 100 with the current version of the patch?

Hopefully I can work around the issue. If I can do so locally, we can check again with the new version of the patch.

Comment 13

[Bob Clary [:bc] (inactive)]

aerickson's issue was that Verify bytecode of debuggable apps was set in Developer options which caused attempts to use run-as to fail. My issue when I set SELinux Enforcing was that I had just previously run a test using root which left root owned files in my /data/local/tmp directory which of course caused me headaches when I attempted to run tests without root. So, we are still "go" for the current version of the patch.

Comment 14

[Ted Campbell [:tcampbell]]

Thanks! I was able to get this working eventually.

ac_add_options --enable-application=mobile/android
ac_add_options --target=aarch64
ac_add_options --enable-js-shell

Configuration 1:

• Pixel 3 -> Win10 host -> VMWare + USB-passthrough -> Ubuntu
• Works successfully.

Configuration 2:

• Xperia Z5 -> Win10 host + adb.exe server -> VMWare + port-forwarding -> Ubuntu
• Works if I change the use_root to always False, otherwise these lock up the adb endpoint on the phone and I need to restart to the phone to unstick it.

Some issues (most pre-existing..):

• Running ./mach jit-test --remote looks up adb on the PATH instead of using the mozbuild version that ./mach install does
• Remote jit-tests are not running parallel jobs. The jit-test harness doesn't have support for this on remote devices yet, but it would reduce the runtime of these tasks.
• As mentioned above, use_root=True is locking some of my devices

SpiderMonkey suites:

• jit-test works as expected
• jstests works as expected
• jsapi-tests it is unclear if we have the right options to make this work. This is pretty low priority, but it is a mechanism we use to run targetted C++ tests if we are investigating specific performance or stability fixes for specific device models.

I was also able to use adb shell to run an interactive js-shell using the version dumped into /data/local/tmp/tests/bin which is useful.

Comment 15

[Bob Clary [:bc] (inactive)]

Thanks! Your environment couldn't get any weirder and is a good test. Can you give me the names of the devices and the version of Android on the ones that lock up when use_root=True as well as capturing the logcat? You can Firefox send the logs and logcats to me. Not sure if we can do anything about it but I might be able to work around it if I can find the commands that cause the issue.

I'll look into the adb path issue. Regarding parallel jobs, I'm not sure how reliable that would be but we can look into it for the future. That might definitely help with our capacity and cost issues. I've never looked into the jsapi-tests but we can also follow up with that later if it is something that would help.

Comment 16

[Ted Campbell [:tcampbell]]

I'll try and get the logcat later today. The command is adb root on a Sony Xperia Z5 Compact (E5823) (original Sony firmware for Android 7.1.1). The device stops enumerating on the USB port entirely, so I'll need to grab the logcat locally from device (any tips here? I'm sure I can find a little app to help me..).

(These devices suffer from jit-bug Bug 1461724 which we've struggled to isolate)

Edit: I should add that the adb server is running on Win10 and I can trigger issue in just powershell by running adb root.

Comment 17

[Bob Clary [:bc] (inactive)]

Ah, knowing that it is adb root is probably sufficient. What is the value of getprop ro.product.model so I could not attempt that command on that device?

Comment 18

[Ted Campbell [:tcampbell]]

(In reply to Bob Clary [:bc] from comment #17)

Ah, knowing that it is adb root is probably sufficient. What is the value of getprop ro.product.model so I could not attempt that command on that device?

> getprop ro.product.model
E5823

Comment 19

[Bob Clary [:bc] (inactive)]

I've updated D79650 with the change for excluding the E5823 as well as adding a check that Verify bytecode of debuggable apps is off. Please try that and let me know how it goes.

Comment 20

[Geoff Brown [:gbrown]]

(In reply to Bob Clary [:bc] from comment #9)

I have updated phabricator revision D79650.

I ran a variety of tests on an unrooted Pixel 2 and an unrooted Galaxy S10. General notes:

• large file pushes seem slow - extra chmod?
• some tests (reftest) warn that the IP address cannot be obtained.

On the Pixel 2, all succeeded: geckoview-junit, mochitest, reftest, xpcshell-test, gtest, jstests, jstestbrowser.
On the S10, all of those succeeded, except jstestbrowser: Test Runner was launched but ran no tests (frequently but intermittently -- fails at least 80% of runs). Will attach logs.

Comment 21

[Bob Clary [:bc] (inactive)]

(In reply to Geoff Brown [:gbrown] from comment #20)

I ran a variety of tests on an unrooted Pixel 2 and an unrooted Galaxy S10. General notes:

• large file pushes seem slow - extra chmod?

I've reduced them in the most recent revision. It should be better now.

• some tests (reftest) warn that the IP address cannot be obtained

The commands require root. I have added a log message about that.

On the Pixel 2, all succeeded: geckoview-junit, mochitest, reftest, xpcshell-test, gtest, jstests, jstestbrowser.

:-)

On the S10, all of those succeeded, except jstestbrowser: Test Runner was launched but ran no tests (frequently but intermittently -- fails at least 80% of runs). Will attach logs.

Thanks!

Comment 22

[Ted Campbell [:tcampbell]]

Some updates..

On this Xperia Z5, if I bind a TCP/IP adb port (adb tcpip 5555) first, then then adb root call still breaks USB enumeration, but I can connect over wifi. Using adb unroot then restores the device to appear on USB again.

Your updated patch seems to detect the phone and avoid calling adb root, but there seems to be a regression with your selinux stuff. In previous patch I would get the same warning, but it would proceed and the tests worked successfully. Now this causes the mach install process to fail for me.

 0:04.76 INFO Native cp support: True
 0:06.07 INFO Native chmod -R support: True
 0:07.08 INFO Native chown -R support: True
 0:10.72 INFO Native normal pidof support: True
 0:13.34 INFO su -c setenforce 0 exitcode 127, stdout: None
 0:14.15 DEBUG Check for su -c failed: args: /home/tcampbell/.mozbuild/android-sdk-linux/platform-tools/adb.168.198.179:5555 wait-for-device shell su -c id, exitcode: 127, stdout: /system/bin/sh: su: not found
 0:15.06 DEBUG Check for su 0 failed: args: /home/tcampbell/.mozbuild/android-sdk-linux/platform-tools/adb.168.198.179:5555 wait-for-device shell su 0 id, exitcode: 127, stdout: /system/bin/sh: su: not found
 0:18.50 WARNING Unable to set SELinux Permissive due to args: /home/tcampbell/.mozbuild/android-sdk-linux/platform-tools/adb.168.198.179:5555 wait-for-device shell setenforce Permissive, exitcode: 1, stdout: setenforce: Couldn't set enforcing status to 'Permissive': Permission denied.
Error running mach:

    ['install']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file install| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

mozdevice.adb.ADBError: Verify bytecode of debuggable apps must be turned off to use run-as

  File "/home/tcampbell/projects/gecko.dev/python/mozbuild/mozbuild/mach_commands.py", line 726, in install
    ret = verify_android_device(self, install=InstallIntent.YES, **kwargs) == 0
  File "/home/tcampbell/projects/gecko.dev/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 239, in verify_android_device
    device = _get_device(build_obj.substs, device_serial)
  File "/home/tcampbell/projects/gecko.dev/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 86, in _get_device
    device = ADBDeviceFactory(adb=adb_path, verbose=verbose_logging, device=device_serial)
  File "/home/tcampbell/projects/gecko.dev/testing/mozbase/mozdevice/mozdevice/adb.py", line 599, in ADBDeviceFactory
    run_as_package=run_as_package)
  File "/home/tcampbell/projects/gecko.dev/testing/mozbase/mozdevice/mozdevice/adb.py", line 943, in __init__
    self.run_as_package = run_as_package
  File "/home/tcampbell/projects/gecko.dev/testing/mozbase/mozdevice/mozdevice/adb.py", line 1193, in run_as_package
    raise ADBError("Verify bytecode of debuggable apps must be turned off to use run-as")

Comment 23

[Geoff Brown [:gbrown]]

(In reply to Geoff Brown [:gbrown] from comment #20)

On the S10, all of those succeeded, except jstestbrowser: Test Runner was launched but ran no tests (frequently but intermittently -- fails at least 80% of runs). Will attach logs.

'mach jstestbrowser' suddenly started working for me on the S10: It failed many times in a row, then I went to collect logs and it succeeded! I stopped collecting logs, and it succeeded again. Now I can't make it fail. Sorry, no logs.

Comment 24

[Bob Clary [:bc] (inactive)]

(In reply to Ted Campbell [:tcampbell] from comment #22)

mozdevice.adb.ADBError: Verify bytecode of debuggable apps must be turned off to use run-as

I added this check to work around the issue that aerickson saw on the samsung S10g. You just need to go to Developer options and turn that off. I'll try to improve the error message for this. If this is a problem I can do device detection to only throw this error on the S10g.

(In reply to Geoff Brown [:gbrown] from comment #23)

All good now?

Comment 25

[Ted Campbell [:tcampbell]]

(In reply to Bob Clary [:bc] from comment #24)

(In reply to Ted Campbell [:tcampbell] from comment #22)

mozdevice.adb.ADBError: Verify bytecode of debuggable apps must be turned off to use run-as

I added this check to work around the issue that aerickson saw on the samsung S10g. You just need to go to Developer options and turn that off. I'll try to improve the error message for this. If this is a problem I can do device detection to only throw this error on the S10g.

I tried toggling the Verify apps over USB setting and could not get past the check.

Comment 26

[Bob Clary [:bc] (inactive)]

Ok. I just commented out that code block. I'll do this in a more controlled fashion for the S10g and figure out how to detect when it might be required for other devices. Try the latest phab revision and maybe if we are lucky you'll get farther.

Comment 27

[Ted Campbell [:tcampbell]]

(In reply to Ted Campbell [:tcampbell] from comment #25)

I tried toggling the Verify apps over USB setting and could not get past the check.

It seems that on this Android 7.1.1 device the Verify bytecode of debuggable apps option does not exist and the settings list global doesn't show the setting as existing either.

On my Pixel 3, I was able to successfully run tests without disabling that option (which does exist on that device..).

Comment 28

[Ted Campbell [:tcampbell]]

(In reply to Bob Clary [:bc] from comment #26)

Ok. I just commented out that code block. I'll do this in a more controlled fashion for the S10g and figure out how to detect when it might be required for other devices. Try the latest phab revision and maybe if we are lucky you'll get farther.

Yep, this works without any local changes now on the Xperia Z5 (and the Pixel 3).

Comment 29

[Andrew Creskey [:acreskey]]

(In reply to Bob Clary [:bc] from comment #12)

That's great news. aerickson had problems running reftest on a Samsung S10g.

While considering why I realized while testing and switching back and forth between su enabled or disabled using magisk I was testing without root but with SELinux Permissive. On my Pixel 2 once I setenforce 1and then disable su I can no longer read the state of SELinux and get the same failure aerickson did.

Can you run ./mach reftest --this-chunk 1 --total-chunks 100 with the current version of the patch?

Hopefully I can work around the issue. If I can do so locally, we can check again with the new version of the patch.

Using the patch at https://phabricator.services.mozilla.com/D79650 and after building I tried this test:

./mach reftest --this-chunk 1 --total-chunks 100

On a non-rooted Pixel 3 (most is MacOS), I ran into this error:

 4:11.37 DEBUG Using cached test_root /data/data/org.mozilla.geckoview.test/tests
 4:11.59 INFO Setting run_as_package to org.mozilla.geckoview.test
 4:11.70 DEBUG Using cached test_root /data/data/org.mozilla.geckoview.test/tests
 4:11.70 WARNING Device not rooted. Can not obtain ip address.
WARNING: unable to get Android device's IP address!
WARNING: tests may fail without network connectivity to the device!
Error running mach:

    ['reftest', '--this-chunk', '1', '--total-chunks', '100']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file reftest| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ImportError: cannot import name _psutil_osx

  File "/Users/acreskey/dev/src/mozilla-central/layout/tools/reftest/mach_commands.py", line 208, in run_reftest
    return self._run_reftest(**kwargs)
  File "/Users/acreskey/dev/src/mozilla-central/layout/tools/reftest/mach_commands.py", line 243, in _run_reftest
    return reftest.run_android_test(**kwargs)
  File "/Users/acreskey/dev/src/mozilla-central/layout/tools/reftest/mach_commands.py", line 101, in run_android_test
    import remotereftest
  File "/Users/acreskey/dev/src/mozilla-central/build/mach_bootstrap.py", line 476, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/Users/acreskey/dev/src/build/obj-release-android/_tests/reftest/remotereftest.py", line 9, in <module>
    import psutil
  File "/Users/acreskey/dev/src/mozilla-central/build/mach_bootstrap.py", line 476, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/Users/acreskey/dev/src/mozilla-central/third_party/python/psutil/psutil/__init__.py", line 161, in <module>
    from . import _psosx as _psplatform
  File "/Users/acreskey/dev/src/mozilla-central/build/mach_bootstrap.py", line 476, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/Users/acreskey/dev/src/mozilla-central/third_party/python/psutil/psutil/_psosx.py", line 15, in <module>
    from . import _psutil_osx as cext

Comment 30

[Andrew Creskey [:acreskey]]

I did try pip install psutil, which gives this log
Requirement already satisfied: psutil in /Users/acreskey/.pyenv/versions/2.7.11/lib/python2.7/site-packages (5.6.3)
but running the test leads to the same error. I had done a ./mach boostrap for Android before running.

Comment 31

[Ted Campbell [:tcampbell]]

One thing I noticed for this Xperia Z5 (E5823) is that if the adb root command is run that on top of disabling the USB connection, the tcp/ip adb connection still is able to load a shell, but the current user is "root"/0 but it does not have any permissions at all and the other steps fail because /data/local/tmp isn't writeable by that user. Using adb unroot does restore the device back to normal operation.

Perhaps we should simply check if we we can mkdir the working directory first before we invoke the various workarounds (eg adb shell mkdir /data/local/tmp/mozdevice + chmod).

As well, it would be nice to have a directly a little more descriptive than "tests". Is something like /data/local/tmp/mozdevice reasonable for this?

Comment 32

[Bob Clary [:bc] (inactive)]

(In reply to Andrew Creskey [:acreskey] [he/him] from comment #29)

ImportError: cannot import name _psutil_osx

I think that is outside the purview of this patch. I pushed an updated revision this morning before heading to the dentist, did you use that one or an older one? Do you get that error without this patch?

I've always had issues with psutil on Fedora as well. If we don't have a bug on file already, please file one and hopefully we can figure it out.

(In reply to Ted Campbell [:tcampbell] from comment #31)

One thing I noticed for this Xperia Z5 (E5823) is that if the adb root command is run that on top of disabling the USB connection, the tcp/ip adb connection still is able to load a shell, but the current user is "root"/0 but it does not have any permissions at all and the other steps fail because /data/local/tmp isn't writeable by that user. Using adb unroot does restore the device back to normal operation.

Perhaps we should simply check if we we can mkdir the working directory first before we invoke the various workarounds (eg adb shell mkdir /data/local/tmp/mozdevice + chmod).

If we don't have a root shell on the device I don't see the benefit since we'll end up either using su if it is rooted or the non-root run-as approach which is the point of the patch. I still want to check for a root shell for use with emulators and aosp builds since there are things we can't do without root.

As well, it would be nice to have a directly a little more descriptive than "tests". Is something like /data/local/tmp/mozdevice reasonable for this?

I suppose. This isn't something new though. We've used tests like since forever even going back to the SUTAgent days. If you can muster enough support I'll go with it but this seems like bikeshedding.

Comment 33

[Bob Clary [:bc] (inactive)]

I've been thinking about it and tests is pretty generic but mozdevice doesn't seem much better. If I saw that would I think that I could or should put my testing content underneath or would I think it has to do with mozdevices internals and that I should put my content elsewhere? I thought of mozilla but that is kind of org centric and has the same issues.

That said, this is turning out to be a major update to mozdevice and if there is any time to make such changes, now is it.

Perhaps the directory name should reflect what we really call it... test_root. We already do create and remove a dummy directory under the root, but perhaps we could also dump a small readme to explain the purpose of the directory for someone spelunking for clues. I'd be willing to go that route if it seems reasonable to folks.

Comment 34

[Ted Campbell [:tcampbell]]

test_root + README actually sounds great, but I agree this was bikeshedding :). I did not realize this was an old pattern we've been using.

Comment 35

[Ted Campbell [:tcampbell]]

Another nice thing about this patch is I was able to build standalone js builds and have tests just work:

//mozconfig
ac_add_options --enable-application=js
ac_add_options --target=aarch64-linux-android

and then running ./mach build && ./mach jit-test --remote.

Comment 36

[Ted Campbell [:tcampbell]]

Additionally, (in another bug), I'm interested in using this similar flow to push the NDK gdbserver (from $HOME/.mozbuild/android-ndk-r20/prebuilt/android-arm64/gdbserver) to the test root to be able to use for directly debugging jsshell crashes.

Comment 37

[Andrew Creskey [:acreskey]]

(In reply to Bob Clary [:bc] from comment #32)

(In reply to Andrew Creskey [:acreskey] [he/him] from comment #29)

ImportError: cannot import name _psutil_osx

I think that is outside the purview of this patch. I pushed an updated revision this morning before heading to the dentist, did you use that one or an older one? Do you get that error without this patch?

I've always had issues with psutil on Fedora as well. If we don't have a bug on file already, please file one and hopefully we can figure it out.

I tried to run the tests again on the latest revision but I bumped into the same _psutil_osx package issue.
Logged here: Bug 1650410

Comment 38

[Andrew Creskey [:acreskey]]

:gbrown provided me with a patch to resolve the _psutil_osx issue I saw in Bug 1650410.

With the patch from this bug, on my unrooted Pixel 3, I can successfully run ./mach reftest --this-chunk 1 --total-chunks 100 .

REFTEST INFO | Result summary:
REFTEST INFO | Successful: 109 (102 pass, 7 load only)
REFTEST INFO | Unexpected: 2 (2 unexpected fail, 0 unexpected pass, 0 unexpected asserts, 0 failed load, 0 exception)

(the unexpected are both image compare differences)

Comment 39

[Andrew Creskey [:acreskey]]

On an unrooted Samsung Galaxy S10 the reftests also run :)

REFTEST INFO | Result summary:
REFTEST INFO | Successful: 111 (104 pass, 7 load only)
REFTEST INFO | Unexpected: 0 (0 unexpected fail, 0 unexpected pass, 0 unexpected asserts, 0 failed load, 0 exception)
REFTEST INFO | Known problems: 39 (20 known fail, 0 known asserts, 2 random, 17 skipped, 0 slow)

logs:
https://paste.mozilla.org/UsceEDjY/raw

Comment 40

[Bob Clary [:bc] (inactive)]

I've updated the phab revision D79650. If this looks good on try I'll split the wip patch into its components and start the process of getting actual reviews.

Please beat up on the patch with whatever devices you have and whatever tests interest you and let me know of any problems.

Comment 41

[Bob Clary [:bc] (inactive)]

I missed a comment from tcampbell and have updated with his suggestion in case you've pulled the patch before now.

Comment 42

[Ted Campbell [:tcampbell]]

This all works great on my Pixel 3. I don't have my Xperia Z5 handy this week, but the patch looks correct to me and I tried similar changes on it when I suggested them.

Thanks for getting this all working!

Comment 43

[Bob Clary [:bc] (inactive)]

Attachment: patch-plan.txt

I've just submitted what I expect will be the final wip version to phabricator. This attachment outlines the series of patches I will create from the wip version. Rather than filing separate bugs for each in the relevent areas, I'm just assigning them to this bug.

I still need to write the new module docstring for mozdevice and the test_root readme is pretty bare bones but I can work on that while we are working through the reviews.

I've updated the in-tree requirements which list mozdevice to require the newest 4.0.0 version. I'm not sure if we will need to do releases for all of the affected packages or who should do so. I expect to defer to the package owners as to their releases.

browsertime will need the updated geckodriver before it can be run on unrooted devices.

web-platform tests work with emulators which are rooted and not rooted but since it only supports running with emulators, it has not been tested with hardware.

I'll do another mach try auto with the full stack of patches when they are done.

If you have any problems using this any of your devices, please let me know soon. I am afk for the later part of today and the later part of tomorrow.

Comment 44

[Geoff Brown [:gbrown]]

nit - This is still a little diagnostic-heavy for my taste. When I run a mach test command, I want to see if the tests pass -- not what the test framework is doing. Here's the start of a mach mochitest run on an unrooted S10:

 0:02.00 adb INFO Using adb 1.0.41
 0:02.48 adb INFO detect ls -1A: args: /home/gbrown/.mozbuild/android-sdk-linux/platform-tools/adb wait-for-device shell /system/bin/ls -1A /system, exitcode: 1, stdout: ls: /system/info.extra: Permission denied
apex
app
bin
build.prop
cameradata
container
etc
fonts
framework
hidden
lib
lib64
media
priv-app
product
recovery-from-boot.p
saiv
tima_measurement_info
tts
usr
vendor
voicebargeindata
vramdiskddata
 0:02.48 adb INFO /system/bin/ls -a supported
 0:02.60 adb INFO Native cp support: True
 0:02.73 adb INFO Native chmod -R support: True
 0:02.85 adb INFO Native chown -R support: True
 0:03.35 adb INFO Native normal pidof support: True
 0:03.85 adb INFO adbd not restarted as root
 0:03.97 adb INFO su -c setenforce 0 exitcode 127, stdout: None
 0:04.70 adb WARNING Unable to set SELinux Permissive due to args: /home/gbrown/.mozbuild/android-sdk-linux/platform-tools/adb wait-for-device shell setenforce Permissive, exitcode: 1, stdout: setenforce: Couldn't set enforcing status to 'Permissive': Permission denied.
 0:06.45 adb INFO Setting test_root to /data/local/tmp/test_root
 0:06.83 adb INFO Setting run_as_package to org.mozilla.geckoview.test
 0:07.29 adb INFO Resetting test root from /data/local/tmp/test_root to /data/data/org.mozilla.geckoview.test/test_root
 0:09.30 adb WARNING Device not rooted. Can not obtain ip address.
WARNING: unable to get Android device's IP address!
WARNING: tests may fail without network connectivity to the device!
 0:13.61 INFO Android sdk version '29'; will use this to filter manifests
 0:13.61 INFO Checking for ssltunnel processes...
 0:13.67 INFO Checking for xpcshell processes...
 0:13.71 SUITE_START: mochitest-plain - running 30 tests
...

Comment 45

[Geoff Brown [:gbrown]]

There is a troubling delay when running gtest with this patch, on my S10; this push loop takes more than 4 minutes:

https://searchfox.org/mozilla-central/rev/7ec7ee4a9bde171ba195ab46ed6077e4baaef34d/testing/gtest/remotegtests.py#100-104

Comment 46

[Geoff Brown [:gbrown]]

Also on my S10, 'mach gtest' ends prematurely (gives up after only a few dozen tests have run) because it finds no top activity. Debugging, I see dumpsys returns

 ...
    mFocusedApp=Token{443fe42 ActivityRecord{f638c8d u0 org.mozilla.geckoview.test/.TestRunnerActivity t89}}
 ...

which doesn't quite match the regex at https://searchfox.org/mozilla-central/rev/7ec7ee4a9bde171ba195ab46ed6077e4baaef34d/testing/mozbase/mozdevice/mozdevice/adb.py#3129-3130. I don't think this has anything to do with this patch; I'll file another bug.

Comment 47

[Bob Clary [:bc] (inactive)]

(In reply to Geoff Brown [:gbrown] from comment #44)

nit - This is still a little diagnostic-heavy for my taste. When I run a mach test command, I want to see if the tests pass -- not what the test framework is doing. Here's the start of a mach mochitest run on an unrooted S10:

This isn't with --log-mach-level debug or anything? Maybe I have it turned on all the time. I'll check.

(In reply to Geoff Brown [:gbrown] from comment #45)

There is a troubling delay when running gtest with this patch, on my S10; this push loop takes more than 4 minutes:

https://searchfox.org/mozilla-central/rev/7ec7ee4a9bde171ba195ab46ed6077e4baaef34d/testing/gtest/remotegtests.py#100-104

I'll look as well. I didn't break things with the last minute change https://phabricator.services.mozilla.com/D79650#inline-470335 ?

(In reply to Geoff Brown [:gbrown] from comment #46)

Also on my S10, 'mach gtest' ends prematurely (gives up after only a few dozen tests have run) because it finds no top activity. Debugging, I see dumpsys returns

 ...
    mFocusedApp=Token{443fe42 ActivityRecord{f638c8d u0 org.mozilla.geckoview.test/.TestRunnerActivity t89}}
 ...

which doesn't quite match the regex at https://searchfox.org/mozilla-central/rev/7ec7ee4a9bde171ba195ab46ed6077e4baaef34d/testing/mozbase/mozdevice/mozdevice/adb.py#3129-3130. I don't think this has anything to do with this patch; I'll file another bug.

Great.

Comment 48

[Ted Campbell [:tcampbell]]

Comment on attachment 9156644 [details]
Bug 1486004 - WIP Use run-as to eliminate rooting requirement for devices.

Current patch works for the cases I've been checking (mach jit-test --remote, mach jstests --remote, application=mobile/android vs application=js) for Xperia Z5 and Pixel 3. Thanks.

Comment 49

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 1 - [mozdevice 4.0.0] Use run-as to work around rooted device requirement, r=gbrown,jmaher

Comment 50

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 2 - Update remoteautomation.py to support unrooted android devices using mozdevice 4.0.0, r=gbrown

Depends on D82744

Comment 51

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 3 - change help prompt for remote test root to show /data/local/tmp/test_root, r=gbrown

Depends on D82745

Comment 52

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 4 - Update jstests, jit-tests to support unrooted android devices using mozdevice 4.0.0, ?

Depends on D82746

Comment 53

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 5 - Update reftests to support unrooted android devices using mozdevice 4.0.0, ?

Depends on D82747

Comment 54

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 6 - Update mach perftest to support unrooted android devices using mozdevice 4.0.0, ?

Depends on D82748

Comment 55

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 7 - Update Conditioned Profiles to support unrooted android devices using mozdevice 4.0.0, ?

Depends on D82749

Comment 56

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 8 - Update gtests to support unrooted android devices using mozdevice 4.0.0, r=gbrown

Depends on D82750

Comment 57

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 9 - Update marionette harness requirements to support unrooted android devices using mozdevice 4.0.0, ?

Depends on D82751

Comment 58

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 10 - Update mochitest to support unrooted android devices using mozdevice 4.0.0, ?

Depends on D82752

Comment 59

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 11 - [mozpower 1.1.2] Update mozpower requirements to support unrooted android devices using mozdevice 4.0.0, ?

Depends on D82753

Comment 60

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 12 - [mozrunner 8.0.2] Update mozrunner requirements to support unrooted android devices using mozdevice 4.0.0, r=gbrown

Depends on D82754

Comment 61

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 13 - Update mozharness configs and scripts to support unrooted android devices using mozdevice 4.0.0, r=gbrown

Depends on D82755

Comment 62

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 14 - Update raptor to support unrooted android devices using mozdevice 4.0.0, ?

Depends on D82756

Comment 63

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 15 - Update remotecppunittests to support unrooted android devices using mozdevice 4.0.0, r=gbrown

Depends on D82757

Comment 64

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 16 - Update web-platform tests to support unrooted android devices using mozdevice 4.0.0, ?

Depends on D82758

Comment 65

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1486004 - Part 17 - Update remote xpcshell-tests to support unrooted android devices using mozdevice 4.0.0, r=gbrown

Depends on D82759

Comment 66

[Bob Clary [:bc] (inactive)]

try runs

Comment 67

[Bob Clary [:bc] (inactive)]

wlach: NI D82754

Comment 68

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/mozregression/pull/684

Comment 69

[Bob Clary [:bc] (inactive)]

I moz-phab submit an updated series of patches corresponding to this try push. moz-phab crashed at the end on part 17 so my tree is horked at the moment. I'm pretty sure that phabricator contains the latest versions since there were no changes for part 17. Not sure what to do next. I've asked in #Conduit on Riot but haven't gotten any help so far.

Comment 70

[Bob Clary [:bc] (inactive)]

Fwiw, I filed bug 1652109 on my moz-phab problems. Not sure where we stand on recovering from this mess. :-(

Comment 71

[Bob Clary [:bc] (inactive)]

I think we are back with a good patchset fwiw. Many thanks to Kwan for the help in recovering.

Comment 72

[Bob Clary [:bc] (inactive)]

I think I have a blocking issue that prevents the mozdevice changes here from landing. I've filed bug 1652339 to work it out but it seems we need to update the requirements on mozdevice to use only mozdevice 3 compatible versions and uplift it to all of our current branches before we can land the changes to update mozdevice to 4.0.0.

Comment 73

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ebe8c2d5432e
Part 1 - [mozdevice 4.0.0] Use run-as to work around rooted device requirement, r=gbrown,jmaher
https://hg.mozilla.org/integration/autoland/rev/a9e73221c471
Part 2 - Update remoteautomation.py to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/38d7def3d425
Part 3 - change help prompt for remote test root to show /data/local/tmp/test_root, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/81427a5f2103
Part 4 - Update jstests, jit-tests to support unrooted android devices using mozdevice 4.0.0, ? r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/677f2129750e
Part 5 - Update reftests to support unrooted android devices using mozdevice 4.0.0, ? r=jmaher
https://hg.mozilla.org/integration/autoland/rev/eb186c92232d
Part 6 - Update mach perftest to support unrooted android devices using mozdevice 4.0.0, ? r=tarek
https://hg.mozilla.org/integration/autoland/rev/88a5a4ab39fa
Part 7 - Update Conditioned Profiles to support unrooted android devices using mozdevice 4.0.0, ? r=tarek
https://hg.mozilla.org/integration/autoland/rev/9d1ce06f6039
Part 8 - Update gtests to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/140a9d2362f3
Part 9 - Update marionette harness requirements to support unrooted android devices using mozdevice 4.0.0, ? r=marionette-reviewers,whimboo
https://hg.mozilla.org/integration/autoland/rev/9730c0303810
Part 10 - Update mochitest to support unrooted android devices using mozdevice 4.0.0, ? r=ahal
https://hg.mozilla.org/integration/autoland/rev/50285a87de6b
Part 11 - [mozpower 1.1.2] Update mozpower requirements to support unrooted android devices using mozdevice 4.0.0, ? r=wlach,sparky
https://hg.mozilla.org/integration/autoland/rev/d0db5eef2343
Part 12 - [mozrunner 8.0.2] Update mozrunner requirements to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/4cba8e6bc515
Part 13 - Update mozharness configs and scripts to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/b487e98320fb
Part 14 - Update raptor to support unrooted android devices using mozdevice 4.0.0, ? r=perftest-reviewers,sparky
https://hg.mozilla.org/integration/autoland/rev/8ab3f1597fde
Part 15 - Update remotecppunittests to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/4fc98067edc5
Part 16 - Update web-platform tests to support unrooted android devices using mozdevice 4.0.0, ? r=jgraham
https://hg.mozilla.org/integration/autoland/rev/86a0855e1e45
Part 17 - Update remote xpcshell-tests to support unrooted android devices using mozdevice 4.0.0, r=gbrown

Comment 74

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/24625 for changes under testing/web-platform/tests

Comment 75

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/43480f194c8b
Backed out 17 changesets for causing bustages in AccessCheck.h

Comment 76

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/49586014bd29
Part 1 - [mozdevice 4.0.0] Use run-as to work around rooted device requirement, r=gbrown,jmaher
https://hg.mozilla.org/integration/autoland/rev/7c23729053f6
Part 2 - Update remoteautomation.py to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/6349ee0bd63f
Part 3 - change help prompt for remote test root to show /data/local/tmp/test_root, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/f20eb5b5a670
Part 4 - Update jstests, jit-tests to support unrooted android devices using mozdevice 4.0.0, ? r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/40763f4afd26
Part 5 - Update reftests to support unrooted android devices using mozdevice 4.0.0, ? r=jmaher
https://hg.mozilla.org/integration/autoland/rev/d1088e178216
Part 6 - Update mach perftest to support unrooted android devices using mozdevice 4.0.0, ? r=tarek
https://hg.mozilla.org/integration/autoland/rev/7c162e1b3a79
Part 7 - Update Conditioned Profiles to support unrooted android devices using mozdevice 4.0.0, ? r=tarek
https://hg.mozilla.org/integration/autoland/rev/582cd190f174
Part 8 - Update gtests to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/1c56cb03ce53
Part 9 - Update marionette harness requirements to support unrooted android devices using mozdevice 4.0.0, ? r=marionette-reviewers,whimboo
https://hg.mozilla.org/integration/autoland/rev/f6f5e7f2bd0a
Part 10 - Update mochitest to support unrooted android devices using mozdevice 4.0.0, ? r=ahal
https://hg.mozilla.org/integration/autoland/rev/e1df276f8a8d
Part 11 - [mozpower 1.1.2] Update mozpower requirements to support unrooted android devices using mozdevice 4.0.0, ? r=wlach,sparky
https://hg.mozilla.org/integration/autoland/rev/2bcca7cf3f1b
Part 12 - [mozrunner 8.0.2] Update mozrunner requirements to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/650c47e81402
Part 13 - Update mozharness configs and scripts to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/cdda7b349cd8
Part 14 - Update raptor to support unrooted android devices using mozdevice 4.0.0, ? r=perftest-reviewers,sparky
https://hg.mozilla.org/integration/autoland/rev/7f186e6bcccc
Part 15 - Update remotecppunittests to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/b10f105b1c36
Part 16 - Update web-platform tests to support unrooted android devices using mozdevice 4.0.0, ? r=jgraham
https://hg.mozilla.org/integration/autoland/rev/120c64ebdea3
Part 17 - Update remote xpcshell-tests to support unrooted android devices using mozdevice 4.0.0, r=gbrown

Comment 77

[Alexandru Michis [:malexandru]]

Comment 75 was an erroneous backout, Comment 76 contains the re-landed changesets!

Comment 78

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2f7ced05c85d
Backed out 20 changesets (bug 1486004, bug 1653094) for wrench bustages. CLOSED TREE

Comment 79

[Bob Clary [:bc] (inactive)]

kats: wrench uses mozbuild which is handling its virtualenv differently that in mozharness. Can you give me some pointers on initializing the virtual environment for use in testing/mozharness/mozharness/mozilla/testing/android.py ?

Comment 80

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot

Comment 81

[Kartikaya Gupta (email:[e-mail])]

(In reply to Bob Clary [:bc] (pto 2020-07-17 - 2020-07-26) from comment #79)

kats: wrench uses mozbuild which is handling its virtualenv differently that in mozharness. Can you give me some pointers on initializing the virtual environment for use in testing/mozharness/mozharness/mozilla/testing/android.py ?

For posterity the discussion for this happened on bug 1653094. I have a patch now that seems to resolve the issues and hopefully we can get everything relanded soon.

Comment 82

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1181ff03bb05
Part 1 - [mozdevice 4.0.0] Use run-as to work around rooted device requirement, r=gbrown,jmaher
https://hg.mozilla.org/integration/autoland/rev/6d2e305f4722
Part 2 - Update remoteautomation.py to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/138c987440c7
Part 3 - change help prompt for remote test root to show /data/local/tmp/test_root, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/a0e2f91eb4d4
Part 4 - Update jstests, jit-tests to support unrooted android devices using mozdevice 4.0.0, ? r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/af6afed0a04a
Part 5 - Update reftests to support unrooted android devices using mozdevice 4.0.0, ? r=jmaher
https://hg.mozilla.org/integration/autoland/rev/473bac8eedd9
Part 6 - Update mach perftest to support unrooted android devices using mozdevice 4.0.0, ? r=tarek
https://hg.mozilla.org/integration/autoland/rev/48947692c176
Part 7 - Update Conditioned Profiles to support unrooted android devices using mozdevice 4.0.0, ? r=tarek
https://hg.mozilla.org/integration/autoland/rev/2f33b309d746
Part 8 - Update gtests to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/234e55153fb6
Part 9 - Update marionette harness requirements to support unrooted android devices using mozdevice 4.0.0, ? r=marionette-reviewers,whimboo
https://hg.mozilla.org/integration/autoland/rev/6ef81c676859
Part 10 - Update mochitest to support unrooted android devices using mozdevice 4.0.0, ? r=ahal
https://hg.mozilla.org/integration/autoland/rev/0a1c8d50ee69
Part 11 - [mozpower 1.1.2] Update mozpower requirements to support unrooted android devices using mozdevice 4.0.0, ? r=wlach,sparky
https://hg.mozilla.org/integration/autoland/rev/5b1c62c4a4d5
Part 12 - [mozrunner 8.0.2] Update mozrunner requirements to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/505747095def
Part 13 - Update mozharness configs and scripts to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/551232749abe
Part 14 - Update raptor to support unrooted android devices using mozdevice 4.0.0, ? r=perftest-reviewers,sparky
https://hg.mozilla.org/integration/autoland/rev/424425ed98c6
Part 15 - Update remotecppunittests to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/e0094d50a315
Part 16 - Update web-platform tests to support unrooted android devices using mozdevice 4.0.0, ? r=jgraham
https://hg.mozilla.org/integration/autoland/rev/cae730f105d7
Part 17 - Update remote xpcshell-tests to support unrooted android devices using mozdevice 4.0.0, r=gbrown

Comment 83

[Narcis Beleuzu [:NarcisB]]

Backed out for wrench bustage on android.py

Backout link: https://hg.mozilla.org/integration/autoland/rev/a7dd00ed82351210265183e18be502b0a578229c
Log link: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=310215441&repo=autoland&lineNumber=600

Comment 84

[Kartikaya Gupta (email:[e-mail])]

Ugh, the try pushes I did apparently didn't have these patches after all. So when I thought it was passing I just wasn't testing the right set of patches. I can look more into this on Monday.

Comment 85

[Bob Clary [:bc] (inactive)]

So only the patches in this bug were backed out but the patches in bug 1653094 were left in?

Comment 86

[Kartikaya Gupta (email:[e-mail])]

(In reply to Bob Clary [:bc] (pto 2020-07-17 - 2020-07-26) from comment #85)

So only the patches in this bug were backed out but the patches in bug 1653094 were left in?

Yes. Is that a problem?

Took a quick look at the error. Looks like on bitbar we don't actually have the full gecko source tree and therefore don't have the mozdevice 4.0.0 code. We have mozharness from a tarball, so presumably we can stuff mozdevice in the tarball (or make a new tarball) and use it for the wrench run.

Comment 87

[Kartikaya Gupta (email:[e-mail])]

https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=e10ce199cce55d2d366dc74616d3af3ed1c25fa1 does what I described in the previous comment - I bundle up the mozdevice folder from the gecko source tree on a dependency job and use it on the wrench job that runs on bitbar. Looks green, patch incoming.

Comment 88

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1486004 - Part 18 - Update wrench to use mozdevice 4.0.0 on bitbar. r?gbrown

Since the system mozdevice version on bitbar is old, and mozharness now requires
mozdevice 4.0.0, we simply have the build job make a tarball of mozdevice 4.0.0
from the gecko source tree and export it as an artifact. The wrench job that
runs on bitbar can unpack the tarball and add it to the python path so it gets
picked up.

Comment 89

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/55f61b412297
Part 1 - [mozdevice 4.0.0] Use run-as to work around rooted device requirement, r=gbrown,jmaher
https://hg.mozilla.org/integration/autoland/rev/76d772af2d88
Part 2 - Update remoteautomation.py to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/c753aef408e4
Part 3 - change help prompt for remote test root to show /data/local/tmp/test_root, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/3a5e274548e1
Part 4 - Update jstests, jit-tests to support unrooted android devices using mozdevice 4.0.0, ? r=tcampbell
https://hg.mozilla.org/integration/autoland/rev/8579adbd1606
Part 5 - Update reftests to support unrooted android devices using mozdevice 4.0.0, ? r=jmaher
https://hg.mozilla.org/integration/autoland/rev/dfd54580ddad
Part 6 - Update mach perftest to support unrooted android devices using mozdevice 4.0.0, ? r=tarek
https://hg.mozilla.org/integration/autoland/rev/e4fa6b309d43
Part 7 - Update Conditioned Profiles to support unrooted android devices using mozdevice 4.0.0, ? r=tarek
https://hg.mozilla.org/integration/autoland/rev/fba8ce5676cc
Part 8 - Update gtests to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/cb414dbfaf5f
Part 9 - Update marionette harness requirements to support unrooted android devices using mozdevice 4.0.0, ? r=marionette-reviewers,whimboo
https://hg.mozilla.org/integration/autoland/rev/6cbc24f8ffac
Part 10 - Update mochitest to support unrooted android devices using mozdevice 4.0.0, ? r=ahal
https://hg.mozilla.org/integration/autoland/rev/b6564bf01757
Part 11 - [mozpower 1.1.2] Update mozpower requirements to support unrooted android devices using mozdevice 4.0.0, ? r=wlach,sparky
https://hg.mozilla.org/integration/autoland/rev/25645c6c3821
Part 12 - [mozrunner 8.0.2] Update mozrunner requirements to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/a9dfb016108e
Part 13 - Update mozharness configs and scripts to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/e91cef14cb72
Part 14 - Update raptor to support unrooted android devices using mozdevice 4.0.0, ? r=perftest-reviewers,sparky
https://hg.mozilla.org/integration/autoland/rev/ee44381808d6
Part 15 - Update remotecppunittests to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/c85a52863665
Part 16 - Update web-platform tests to support unrooted android devices using mozdevice 4.0.0, ? r=jgraham
https://hg.mozilla.org/integration/autoland/rev/4c73375cffad
Part 17 - Update remote xpcshell-tests to support unrooted android devices using mozdevice 4.0.0, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/eb56389efa7b
Part 18 - Update wrench to use mozdevice 4.0.0 on bitbar. r=gbrown

Comment 90

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/55f61b412297
https://hg.mozilla.org/mozilla-central/rev/76d772af2d88
https://hg.mozilla.org/mozilla-central/rev/c753aef408e4
https://hg.mozilla.org/mozilla-central/rev/3a5e274548e1
https://hg.mozilla.org/mozilla-central/rev/8579adbd1606
https://hg.mozilla.org/mozilla-central/rev/dfd54580ddad
https://hg.mozilla.org/mozilla-central/rev/e4fa6b309d43
https://hg.mozilla.org/mozilla-central/rev/fba8ce5676cc
https://hg.mozilla.org/mozilla-central/rev/cb414dbfaf5f
https://hg.mozilla.org/mozilla-central/rev/6cbc24f8ffac
https://hg.mozilla.org/mozilla-central/rev/b6564bf01757
https://hg.mozilla.org/mozilla-central/rev/25645c6c3821
https://hg.mozilla.org/mozilla-central/rev/a9dfb016108e
https://hg.mozilla.org/mozilla-central/rev/e91cef14cb72
https://hg.mozilla.org/mozilla-central/rev/ee44381808d6
https://hg.mozilla.org/mozilla-central/rev/c85a52863665
https://hg.mozilla.org/mozilla-central/rev/4c73375cffad
https://hg.mozilla.org/mozilla-central/rev/eb56389efa7b

Comment 91

[Bob Clary [:bc] (inactive)]

gbrown: We bumped the version information for mozpower in Part 11. Could you do another release for mozpower 1.1.2 ?

Comment 92

[Geoff Brown [:gbrown]]

https://pypi.org/project/mozpower/1.1.2/