Closed Bug 912784 Opened 11 years ago Closed 11 years ago

The wrong utility path is picked up for xpcshell when running mochitest-robocop

Categories

(Testing :: Mochitest, defect)

x86
macOS
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla26

People

(Reporter: kats, Assigned: gbrown)

Details

Attachments

(1 file)

I don't know why this sort of stuff only ever happens to me, but for some reason when I run mochitest-robocop for fennec, the options.utilityPath is set to my fennec objdir/dist/bin, so when the runtestsremote.py looks for the xpcshell, it finds the ARM binary in my fennec objdir, and errors out (bug 912779). I had to comment out the two lines at [1] to get past this error and pick up the correct xpcshell binary from my MOZ_HOST_BIN. [1] http://hg.mozilla.org/mozilla-central/file/99bd249e5a20/testing/mochitest/runtestsremote.py#l299
Assignee: nobody → gbrown
With MOZ_HOST_BIN set correctly, I do not have a problem. mozdev@ubuntu:~/src$ export MOZ_HOST_BIN=/home/mozdev/objdirs/firefox/dist/bin mozdev@ubuntu:~/src$ ls $MOZ_HOST_BIN/xpcshell /home/mozdev/objdirs/firefox/dist/bin/xpcshell mozdev@ubuntu:~/src$ make -C ~/objdirs/droid mochitest-robocop --dry-run make: Entering directory `/home/mozdev/objdirs/droid' if [ ! -f /home/mozdev/objdirs/firefox/dist/bin/xpcshell ]; then \ echo "please prepare your host with the environment variable MOZ_HOST_BIN"; \ elif [ "" = "" -a "adb" != "adb" ]; then \ echo "please prepare your host with the environment variable TEST_DEVICE"; \ else \ rm -f ./mochitest-robocop.log && /home/mozdev/objdirs/droid/_virtualenv/bin/python _tests/testing/mochitest/runtestsremote.py --robocop-apk=./build/mobile/robocop/robocop-debug.apk --robocop-ids=./mobile/android/base/fennec_ids.txt --robocop-ini=./build/mobile/robocop/robocop.ini --console-level=INFO --log-file=./mochitest-robocop.log --file-level=INFO --dm_trans=adb --app=org.mozilla.fennec_mozdev --deviceIP= --xre-path=/home/mozdev/objdirs/firefox/dist/bin --httpd-path=. --symbols-path=./dist/crashreporter-symbols ; \ fi make: Leaving directory `/home/mozdev/objdirs/droid' mozdev@ubuntu:~/src$ make -C ~/objdirs/droid mochitest-robocop make: Entering directory `/home/mozdev/objdirs/droid' Error deleting /data/anr/traces.txt Device info: {} Test root: /mnt/sdcard/tests INFO | automation.py | Launching: /home/mozdev/objdirs/firefox/dist/bin/xpcshell -g /home/mozdev/objdirs/firefox/dist/bin -v 170 -f /home/mozdev/objdirs/droid/_tests/testing/mochitest/httpd.js -e "const _PROFILE_PATH = '/tmp/tmp9GS9p8'; const _SERVER_PORT = '8888'; const _SERVER_ADDR = '192.168.0.50'; const _TEST_PREFIX = undefined; const _DISPLAY_RESULTS = false;" -f ./server.js Mochitest INFO | runtests.py | Server pid: 952 Mochitest INFO | runtests.py | Running tests: start.
ahal -- I'm wondering if this might be related to bug 907552. Do you have any idea how this might be going wrong for kats?
Flags: needinfo?(ahalberstadt)
One thing that may or may not be relevant is that my objdir setup is like so: obj-host // contains the host build obj-android-debug // contains my android build obj-android -> obj-android-debug // obj-android is a symlink to obj-android-debug All my android mozconfigs point to obj-android. I had to also patch mach because it checks the "in-use" objdir against the "mozconfig-specified" objdir and decided that obj-android != obj-android-debug even though in my case they really are. I don't know if the mochitest framework does some similar check but behaves differently.
(In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #3) > All my android mozconfigs point to obj-android. I had to also patch mach > because it checks the "in-use" objdir against the "mozconfig-specified" > objdir and decided that obj-android != obj-android-debug even though in my > case they really are. I don't know if the mochitest framework does some > similar check but behaves differently. FYI that's bug 905824.
See Also: → 914105
This wouldn't be related to bug 907552, but it might be yet more fallout from bug 865349. Though I don't understand why it would work for gbrown and not kats. I believe what is happening is that bug 865349 modified the default utility path to defer to mozbuild [1], which somehow differs from the old default of automation.BIN_DIR on Fennec. The logic for finding the utility path in startWebServer is pretty terrible. It tries to be very magical, searching both xre-path and utility-path, making guesses based on cwd, etc. I may have actually run into this exact same problem in B2G because I simplified the function the equivalent function there greatly [2]. I believe the fix in comment 0 would be ok to land. Alternatively I think we could just add --utility-path MOZ_HOST_BIN to [3]. Though I'm still confused why it would work for gbrown and not kats. Are one of you using an out of date repo (or a different branch)? [1] http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/mochitest_options.py#54 [2] http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/runtestsb2g.py#196 [3] http://mxr.mozilla.org/mozilla-central/source/testing/testsuite-targets.mk#73
Flags: needinfo?(ahalberstadt)
For the record I'd rather add --utility-path to the make target than modify startWebServer at all as it seems safer and I don't really know what I'm doing when it comes to Fennec.
See Also: 914105
(In reply to Andrew Halberstadt [:ahal] from comment #5) > Though I'm still confused > why it would work for gbrown and not kats. Are one of you using an out of > date repo (or a different branch)? I am using a current mozilla-central. As in Comment 1, my desktop build is at /home/mozdev/objdirs/firefox and my android build is at /home/mozdev/objdirs/droid. MOZ_HOST_BIN=/home/mozdev/objdirs/firefox/dist/bin and --xre-path=/home/mozdev/objdirs/firefox/dist/bin is passed on the command line. In startWebServer, I initially [1] find paths = ['/home/mozdev/objdirs/firefox/dist/bin', '/home/mozdev/objdirs/droid/dist/bin', 'fennec', '../fennec'] then [2]: ['/home/mozdev/objdirs/firefox/dist/bin', '/home/mozdev/objdirs/droid/dist/bin', 'fennec', '../fennec', 'bin', '../bin'] and then [3]: ['/mnt/sdcard/tests/fennec/bin', '/home/mozdev/objdirs/firefox/dist/bin', '/home/mozdev/objdirs/droid/dist/bin', 'fennec', '../fennec', 'bin', '../bin'] (That ends up being [options.utilityPath, options.xrePath, localAutomation.DIST_BIN, ...]) After findPaths, utilityPath = /home/mozdev/objdirs/firefox/dist/bin, options.xrePath. [1] http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/runtestsremote.py#290 [2] http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/runtestsremote.py#296 [3] http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/runtestsremote.py#303
In verifyRemoteOptions, my options.utilityPath starts off as /home/mozdev/objdirs/droid/dist/bin but is changed at [4] to /mnt/sdcard/tests/fennec/bin. This setting of utilityPath to a remote (on device) path seems very odd. I would think that if --utility-path is not specified and --xre-path is, we should just use the xre path for the utility path. I wonder why it needs to be more complicated than that. [4] http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/runtestsremote.py#130
(In reply to Andrew Halberstadt [:ahal] from comment #5) > Though I'm still confused > why it would work for gbrown and not kats. Are one of you using an out of > date repo (or a different branch)? Mine is relatively recent m-c code as well. The paths at [1], [2], and [3] from gbrown's comment 7 are: paths1: ['/Users/kats/zspace/mozilla-git/obj-host/dist/bin', '/Users/kats/zspace/mozilla-git/obj-android/dist/bin', 'fennec', '../fennec'] paths2: ['/Users/kats/zspace/mozilla-git/obj-host/dist/bin', '/Users/kats/zspace/mozilla-git/obj-android/dist/bin', 'fennec', '../fennec', 'bin', '../bin'] paths3: [u'/Users/kats/zspace/mozilla-git/obj-android-debug/dist/bin', '/Users/kats/zspace/mozilla-git/obj-host/dist/bin', '/Users/kats/zspace/mozilla-git/obj-android/dist/bin', 'fennec', '../fennec', 'bin', '../bin'] After findPaths, utilityPath = /Users/kats/zspace/mozilla-git/obj-android-debug/dist/bin
utility path at start of verifyRemoteOptions: /Users/kats/zspace/mozilla-git/obj-android-debug/dist/bin utility path later in verifyRemoteOptions: /Users/kats/zspace/mozilla-git/obj-android-debug/dist/bin
This is (mostly) making sense to me now. Utility path handling in remote mochitests is convoluted. If --utility-path is specified, it is used. If it is not specified, build_obj.bindir is used, unless build_obj.bindir == self._automation.DIST_BIN, in which case it is reset to options.remoteTestRoot + "/" + automation._product + "/bin". Then, if the utility path does not exist, xrePath is attempted, and if it does not exist, a few other variations are attempted. Automated tests on tegras and pandas all specify --utility-path, so all is well there. "make mochitest-robocop" does not generate a --utility-path argument. In my environment, build_obj.bindir == self._automation.DIST_BIN, so build_obj.bindir gets reset to a non-existent directory, and I pick up my xre path as a backup -- all is well. :kats is not so lucky. I'll try to simplify all this.
Part of the problem is that mochitest_options.py sets options.utilityPath to build_obj.bindir if --utility-path is not specified -- reasonable for desktop tests, but not for remote. Now runtestsremote.py overrides the option default so that build_obj.bindir is not used. I also removed all references to productRoot -- it doesn't make sense to be considering /mnt/sdcard/tests/... as a location for xre-path or utility-path. Similarly, I cannot think of how paths like ../fennec or ../bin are likely to be useful for utility-path, so the simplified logic comes down to: -- if --utility-path was specified and contains xpcshell, use that for utility-path -- else if xre-path is set and contains xpcshell, use that for utility-path ahal -- any b2g concerns? kats -- please verify that this works in your environment.
Attachment #803284 - Flags: review?(jmaher)
Attachment #803284 - Flags: feedback?(bugmail.mozilla)
Attachment #803284 - Flags: feedback?(ahalberstadt)
Comment on attachment 803284 [details] [diff] [review] simplify utility-path handling Review of attachment 803284 [details] [diff] [review]: ----------------------------------------------------------------- this is all hairy code. If only we didn't need xpcshell for a webserver.
Attachment #803284 - Flags: review?(jmaher) → review+
Comment on attachment 803284 [details] [diff] [review] simplify utility-path handling Review of attachment 803284 [details] [diff] [review]: ----------------------------------------------------------------- This does resolve the problem for me, thanks!
Attachment #803284 - Flags: feedback?(bugmail.mozilla) → feedback+
Comment on attachment 803284 [details] [diff] [review] simplify utility-path handling Review of attachment 803284 [details] [diff] [review]: ----------------------------------------------------------------- A big f+ from me. The way we handled xre-path/utility-path was downright evil and at least this is a slight improvement. Though this could break the workflow of people who are running runtestsremote.py directly. I don't know if this is a common use case or not for Fennec, so I'll let you be the judge of how important it is or isn't.
Attachment #803284 - Flags: feedback?(ahalberstadt) → feedback+
I think there are few direct users of runtestsremote.py. I'll risk annoying them for the sake of long-term sanity! https://hg.mozilla.org/integration/mozilla-inbound/rev/040019b27c8d
(In reply to Geoff Brown [:gbrown] from comment #12) > Similarly, I cannot think of how paths like ../fennec or ../bin are likely > to be useful for utility-path, so the simplified logic comes down to: I assume this was to account for running the script from the test package, where ../bin is probably the utility path, but that probably doesn't actually work since they won't have a host xpcshell anyway.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla26
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: