818967 - mozdevice unit tests failing

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[William Lachance (:wlach)]

Seeing this:

Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/wlach/src/mozbase/mozdevice/tests/sut.py", line 41, in _serve_thread
    self.tester.assertEqual(data, command)
  File "/usr/lib/python2.7/unittest/case.py", line 511, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python2.7/unittest/case.py", line 504, in _baseAssertEqual
    raise self.failureException(msg)
AssertionError: 'isdir /mnt/sdcard/baz/boop' != 'isdir /mnt'

I suspect this is due to the patch in bug 814767 and us not updating the unit tests accordingly. Oops. Fix should be easy at least if that's the case.

Comment 1

[William Lachance (:wlach)]

We should also look into why this wasn't picked up by http://k0s.org:8010/waterfall

Comment 2

[Jeff Hammel]

(In reply to William Lachance (:wlach) from comment #1)
> We should also look into why this wasn't picked up by
> http://k0s.org:8010/waterfall

So this is easy enough: we exit with 0 despite failures:

"""
...
======================================================================
FAIL: test_push_dir (sut_push.PushTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jhammel/mozilla/src/autobot/bot/slave/mozbase/build/env/src/mozbase/mozdevice/tests/sut_push.py", line 84, in test_push_dir
    self.assertEqual(exceptionThrown, subTest['expectException'])
AssertionError: True != False

----------------------------------------------------------------------
Ran 83 tests in 1036.851s

FAILED (failures=3)
program finished with exit code 0
elapsedTime=1038.528351
"""

We supposedly have code to take care of this, https://github.com/mozilla/mozbase/blob/master/test.py#L66 , but evidently we're not doing it rite

Comment 3

[Jeff Hammel]

Evidently moztest is all kinds of broken:

(Pdb) pp list(results.tests)
[]
(Pdb) results.num_failures
0

Comment 4

[Jeff Hammel]

Attachment: fix test.py to correctly exit 1

It turns out that moztest isn't broken, per se, it just has an API so counter-intuitive that even its author didn't use it correctly

Comment 5

[William Lachance (:wlach)]

Comment on attachment 689453 [details] [diff] [review]
fix test.py to correctly exit 1

Awesome! ty. Now we just need to fix the test. :)

Comment 6

[William Lachance (:wlach)]

Attachment: Adapt to changes in mozdevice internals

Pretty straightforward.

Comment 7

[Jeff Hammel]

(In reply to William Lachance (:wlach) from comment #5)
> Comment on attachment 689453 [details] [diff] [review]
> fix test.py to correctly exit 1
> 
> Awesome! ty. Now we just need to fix the test. :)

pushed: https://github.com/mozilla/mozbase/commit/9cdda125c5fa11239a054ecc54afc9fe29b1d373

Comment 8

[Jeff Hammel]

Comment on attachment 689871 [details] [diff] [review]
Adapt to changes in mozdevice internals

wfm. tested locally

Comment 9

[Jeff Hammel]

pushed: https://github.com/mozilla/mozbase/commit/5f3fa2415884f8c4574d7ddc2ae4dfddfa1f95b5