855863 - disable mozdevice tests on windows and otherwise note mozdevice is unsupported on windows

Status: RESOLVED INVALID

(Testing :: Mozbase, defect)

Description

[Jonathan Griffin (:jgriffin)]

Jeff is trying to mozbase tests run during 'make check', but ran into the problem that the mozdevice tests are causing a crash on Windows:

test_timeout_normal (sut_basic.BasicTest)
Tests DeviceManager timeout, normal case. ... ok
test_timeout_timeout (sut_basic.BasicTest)
Tests DeviceManager timeout, timeout case. ... PROCESS-CRASH | test.py | application crashed [Unknown top frame]
Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpifxiuh\test.dmp
this is some output
PROCESS-CRASH | test.py | application crashed [Unknown top frame]
Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpyroqwp\test.dmp
this is some output
PROCESS-CRASH | test.py | application crashed [Unknown top frame]
Crash dump filename: c:\users\cltbld\appdata\local\temp\tmp0bkrw7\test.dmp
this is some output
PROCESS-CRASH | test.py | application crashed [Unknown top frame]
Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpjh50yw\test.dmp
this is some output
PROCESS-CRASH | test.py | application crashed [Unknown top frame]
Crash dump filename: c:\users\cltbld\appdata\local\temp\tmppanhdj\test.dmp

Full log:
https://tbpl.mozilla.org/php/getParsedLog.php?id=21127486&tree=Try&full=1

Comment 1

[Jonathan Griffin (:jgriffin)]

Those crashes are a red herring.  They are apparently intentional artifacts of test_crash.  The real problem is that test_push_dir times out on Windows:

test_push_dir (sut_push.PushTest) ... 
command timed out: 300 seconds without output, attempting to kill

Comment 2

[Jonathan Griffin (:jgriffin)]

So there are two issues on Windows that cause this to fail.  One is the usual path concatenation issues which are easily addressed.  The second is that the tests use NamedTemporaryFile, which works differently on Windows than on linux:

"Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later)."

This is also not hard to fix.

Comment 3

[Jonathan Griffin (:jgriffin)]

A larger question may be, do we want to run mozdevice tests on Windows at all?  None of our mobile infrastructure is Windows-compliant, so it may be wasted effort trying to keep these tests up-to-date on a platform that the library they're testing will never be used on.

Comment 4

[Jeff Hammel]

Will dig into some of these issues later.  As far as "do we want to run mozdevice tests on Windows?", that depends on if we want to support mozdevice on windows (at least in the long term; we can certainly disable for this bug as an interim solution).  If we don't, documentation, setup.py, etc should be updated to reflect that we do not support mozdevice on windows.

Comment 5

[Jonathan Griffin (:jgriffin)]

I found some additional problems on Windows.  The socket code can get completely out-of-sync on Windows whenever SUTAgent makes a second socket send() command for 'data':

https://github.com/mozilla/mozbase/blob/master/mozdevice/mozdevice/devicemanagerSUT.py#L199

This appears less easy to fix.

Comment 6

[William Lachance (:wlach)]

(In reply to Jeff Hammel [:jhammel] from comment #4)
> Will dig into some of these issues later.  As far as "do we want to run
> mozdevice tests on Windows?", that depends on if we want to support
> mozdevice on windows (at least in the long term; we can certainly disable
> for this bug as an interim solution).  If we don't, documentation, setup.py,
> etc should be updated to reflect that we do not support mozdevice on windows.

Yeah, this strikes me as a WONTFIX. I can do the necessary work on documentation, setup.py work if we're all in agreement.

Comment 7

[Jonathan Griffin (:jgriffin)]

I agree...let's WONTFIX.

Comment 8

[Jeff Hammel]

Actually, if no one minds, I'm going to morph the bug as I'm working on skip-if-ing the tests on windows.  Of course, I assumed this was easy ;)  Feel free to retitle if desired or if we want to WONTFIX this then I can file a different bug (but to me it makes more sense to morph, right now).

Comment 9

[Jeff Hammel]

Attachment: disable tests on windows (+ supporting work)

This *seems* to work for me locally (albeit via skip-if = os == 'linux', not windows).  I'm happy to take requests here, as this is a quick sketch

Comment 10

[Jonathan Griffin (:jgriffin)]

Comment on attachment 735282 [details] [diff] [review]
disable tests on windows (+ supporting work)

Review of attachment 735282 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good to me.

Comment 11

[Jeff Hammel]

pushed: https://github.com/mozilla/mozbase/commit/41c9a1d47390bb6803983395b5be787e144f7605

Comment 12

[Jeff Hammel]

We should still note this in at least mozdevice.py.  A comment in setup.py might be nice.  (It'd be nice if there was a standard way of doing this in the existing python ecosystem, but que sara sara.)  I don't think we should disable installation on windows, although an import from mozdevice could trigger e.g. `print >> sys.stderr, "mozdevice is not supported on windows"`

Comment 13

[William Lachance (:wlach)]

mozdevice should work in Windows these days. If it doesn't, we should file bugs.