Android xpcshell failing with Automation Error: error pushing file: Automation Error: Agent Error processing command 'push /mnt/sdcard/tests/xpcshell//netwerk/test/httpserver/test/data/sjs/sjs 72'; err='Push failed!'

RESOLVED WORKSFORME

Status

Testing
XPCShell Harness
RESOLVED WORKSFORME
5 years ago
4 years ago

People

(Reporter: Callek, Assigned: Callek)

Tracking

({intermittent-failure})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Assignee)

Description

5 years ago
So, we just enabled Android XPCShell tests today, and they were promptly hidden on all trees due to high rate of intermittent failure.

error looks to TBPL like:

Automation Error: error pushing file: Automation Error: Agent Error processing command 'push /mnt/sdcard/tests/xpcshell//netwerk/test/httpserver/test/data/sjs/sjs 72'; err='Push failed!'
Automation Error: failed making directory: /mnt
Automation Error: unable to make dirs: /mnt/sdcard/tests/xpcshell//netwerk/test/httpserver/test/data/sjs/sjs
Automation Error: Push File Failed to Validate!
TEST-UNEXPECTED-FAIL | /builds/tegra-210/test/build/tests/xpcshell/tests/gfx/tests/unit/test_nsIScriptableRegion.js | test failed (with xpcshell return code: -1), see following log:


----

The SUTAgent itself spew the following traceback:

--------- beginning of /dev/log/main
04-05 09:54:33.948 W/System.err( 1518): java.io.FileNotFoundException: /mnt/sdcard/tests/xpcshell/netwerk/test/httpserver/test/data/sjs/sjs (No such file or directory)
04-05 09:54:33.948 W/System.err( 1518): 	at org.apache.harmony.luni.platform.OSFileSystem.openImpl(Native Method)
04-05 09:54:33.948 W/System.err( 1518): 	at org.apache.harmony.luni.platform.OSFileSystem.open(OSFileSystem.java:152)
04-05 09:54:33.948 W/System.err( 1518): 	at java.io.FileOutputStream.<init>(FileOutputStream.java:97)
04-05 09:54:33.948 W/System.err( 1518): 	at java.io.FileOutputStream.<init>(FileOutputStream.java:168)
04-05 09:54:33.948 W/System.err( 1518): 	at com.mozilla.SUTAgentAndroid.service.DoCommand.Push(DoCommand.java:1960)
04-05 09:54:33.948 W/System.err( 1518): 	at com.mozilla.SUTAgentAndroid.service.DoCommand.processCommand(DoCommand.java:350)
04-05 09:54:33.948 W/System.err( 1518): 	at com.mozilla.SUTAgentAndroid.service.CmdWorkerThread.run(CmdWorkerThread.java:141)
04-05 09:54:35.628 W/System.err( 1518): java.io.IOException: Error running exec(). Command: [/data/local/xpcb/xpcw, /mnt/sdcard/tests/xpcshell/gfx/tests/unit, -r, /mnt/sdcard/tests/xpcshell/c/httpd.manifest, --greomni, /data/local/xpcb/fennec-23.0a1.en-US.android-arm-armv6.apk, -m, -n, -s, -e, const _HTTPD_JS_PATH = "/mnt/sdcard/tests/xpcshell/c/httpd.js";, -e, const _HEAD_JS_PATH = "/mnt/sdcard/tests/xpcshell/head.js";, -e, const _TESTING_MODULES_DIR = "/mnt/sdcard/tests/xpcshell/m";, -f, /mnt/sdcard/tests/xpcshell/head.js, -e, const _SERVER_ADDR = "localhost", -e, const _HEAD_FILES = [];, -e, const _TAIL_FILES = [];, -e, const _TEST_FILE = ["test_nsIScriptableRegion.js"];, -e, _execute_test(); quit(0);] Working Directory: null Environment: null
04-05 09:54:35.628 W/System.err( 1518): 	at java.lang.ProcessManager.exec(ProcessManager.java:226)
04-05 09:54:35.628 W/System.err( 1518): 	at java.lang.Runtime.exec(Runtime.java:196)
04-05 09:54:35.628 W/System.err( 1518): 	at java.lang.Runtime.exec(Runtime.java:141)
04-05 09:54:35.628 W/System.err( 1518): 	at com.mozilla.SUTAgentAndroid.service.DoCommand.StartPrg(DoCommand.java:3511)
04-05 09:54:35.628 W/System.err( 1518): 	at com.mozilla.SUTAgentAndroid.service.DoCommand.StartPrg2(DoCommand.java:3555)
04-05 09:54:35.628 W/System.err( 1518): 	at com.mozilla.SUTAgentAndroid.service.DoCommand.processCommand(DoCommand.java:689)
04-05 09:54:35.628 W/System.err( 1518): 	at com.mozilla.SUTAgentAndroid.service.CmdWorkerThread.run(CmdWorkerThread.java:141)
04-05 09:54:35.628 W/System.err( 1518): Caused by: java.io.IOException: Permission denied
04-05 09:54:35.628 W/System.err( 1518): 	at java.lang.ProcessManager.exec(Native Method)
04-05 09:54:35.628 W/System.err( 1518): 	at java.lang.ProcessManager.exec(ProcessManager.java:224)
04-05 09:54:35.628 W/System.err( 1518): 	... 6 more

(which was from http://mxr.mozilla.org/comm-central/source/mozilla/build/mobile/sutagent/android/DoCommand.java#2019 )
---------

The test harness traceback is something like

http://mxr.mozilla.org/build/source/tools/sut_tools/mozdevice/devicemanagerSUT.py#408
http://mxr.mozilla.org/build/source/tools/sut_tools/mozdevice/devicemanagerSUT.py#476
http://mxr.mozilla.org/comm-central/source/mozilla/testing/xpcshell/remotexpcshelltests.py#153

Any thoughts accepted.
(Assignee)

Updated

5 years ago
OS: Windows 7 → Android
Hardware: x86_64 → All
I think the problem may be in mkDirs, and occurs when the filename is the same as the directory that contains it. Consider http://mxr.mozilla.org/build/source/tools/sut_tools/mozdevice/devicemanager.py#116 ... and it looks like this is fixed in this version: https://hg.mozilla.org/mozilla-central/annotate/139b6ba547fa/testing/mozbase/mozdevice/mozdevice/devicemanager.py#l213
Comment hidden (Treeherder Robot)
We need the fix from bug 826492.
(Assignee)

Comment 4

5 years ago
Created attachment 734339 [details] [diff] [review]
[tools] v1

Good find gbrown!

Though your example (upstream code) does make things slightly better for official (since we do the rsplit thing, it is still broken in the case of 2 dir's named the same, with last one being the repeated one)

I needed to change the one caller of dm.mkdirs in sut_tools to handle the "doesn't return anything" case properly though

This patch backports the dm.mkDirs change and fixes the sut_tools code.

I've also submitted my enumerate change as a pull request https://github.com/mozilla/mozbase/pull/47
Assignee: nobody → bugspam.Callek
Status: NEW → ASSIGNED
Attachment #734339 - Flags: review?(jmaher)
Attachment #734339 - Flags: review?(gbrown)
Comment hidden (Treeherder Robot)
Comment on attachment 734339 [details] [diff] [review]
[tools] v1

Review of attachment 734339 [details] [diff] [review]:
-----------------------------------------------------------------

I would really really prefer to update the mozdevice code.  I don't see anything wrong with your code here.  Do we use the sut_tools devicemanager for xpcshell?  If we do, I would like us to update to the latest mozdevice
Attachment #734339 - Flags: review?(jmaher) → review-
Comment hidden (Treeherder Robot)
Comment on attachment 734339 [details] [diff] [review]
[tools] v1

Review of attachment 734339 [details] [diff] [review]:
-----------------------------------------------------------------

I'll leave it to you and :jmaher to work this out. I just want xpcshell to work reliably.
Attachment #734339 - Flags: review?(gbrown)
Comment on attachment 734339 [details] [diff] [review]
[tools] v1

Review of attachment 734339 [details] [diff] [review]:
-----------------------------------------------------------------

after looking at this further, it appears we need to modify xpcshell to use the in tree devicemanager instead of the sut_tools version.  Lets land this now and fix xpcshell in the next day or two.
Attachment #734339 - Flags: review- → review+
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Assignee)

Comment 14

5 years ago
Comment on attachment 734339 [details] [diff] [review]
[tools] v1

http://hg.mozilla.org/build/tools/rev/3545fa27fd4e
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
I believe this is a duplicate of bug 826492 and we're seeing this because we're not using the latest version of mozdevice. Updating xpcshell to use in-tree mozbase should fix the issue.
(OrangeWFM bugs not modified in > 2 months)
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.