1595814 - ImportError: cannot import name '_ExpectedFailure'

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect, P3)

Description

[Rishi Gupta (:rgpt)]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0

Steps to reproduce:

Attempt to run marionette-test from python3

$ python3 mach marionette-test

Actual results:

ImportError: cannot import name '_ExpectedFailure'

File "/home/rgpt/Desktop/mozilla/mozilla-unified/python/mach/mach/main.py", line 360, in run
return self._run(argv)
File "/home/rgpt/Desktop/mozilla/mozilla-unified/python/mach/mach/main.py", line 415, in _run
args = parser.parse_args(argv)
File "/usr/lib/python3.5/argparse.py", line 1735, in parse_args
args, argv = self.parse_known_args(args, namespace)
File "/usr/lib/python3.5/argparse.py", line 1767, in parse_known_args
namespace, args = self._parse_known_args(args, namespace)
File "/usr/lib/python3.5/argparse.py", line 1976, in _parse_known_args
stop_index = consume_positionals(start_index)
File "/usr/lib/python3.5/argparse.py", line 1932, in consume_positionals
take_action(action, args)
File "/usr/lib/python3.5/argparse.py", line 1841, in take_action
action(self, namespace, argument_values, option_string)
File "/home/rgpt/Desktop/mozilla/mozilla-unified/python/mach/mach/dispatcher.py", line 179, in call
if handler.parser:
File "/home/rgpt/Desktop/mozilla/mozilla-unified/python/mach/mach/decorators.py", line 77, in parser
self._parser = self._parser()
File "/home/rgpt/Desktop/mozilla/mozilla-unified/testing/marionette/mach_commands.py", line 24, in create_parser_tests
from marionette_harness.runtests import MarionetteArguments
File "/home/rgpt/Desktop/mozilla/mozilla-unified/build/mach_bootstrap.py", line 431, in call
module = self._original_import(name, globals, locals, fromlist, level)
File "/home/rgpt/Desktop/mozilla/mozilla-unified/build/mach_bootstrap.py", line 431, in call
module = self._original_import(name, globals, locals, fromlist, level)
File "/home/rgpt/Desktop/mozilla/mozilla-unified/testing/marionette/harness/marionette_harness/init.py", line 9, in <module>
from .marionette_test import (
File "/home/rgpt/Desktop/mozilla/mozilla-unified/build/mach_bootstrap.py", line 431, in call
module = self._original_import(name, globals, locals, fromlist, level)
File "/home/rgpt/Desktop/mozilla/mozilla-unified/testing/marionette/harness/marionette_harness/marionette_test/init.py", line 27, in <module>
from .testcases import (
File "/home/rgpt/Desktop/mozilla/mozilla-unified/build/mach_bootstrap.py", line 431, in call
module = self._original_import(name, globals, locals, fromlist, level)
File "/home/rgpt/Desktop/mozilla/mozilla-unified/testing/marionette/harness/marionette_harness/marionette_test/testcases.py", line 17, in <module>
from unittest.case import (

Expected results:

marionette tests should have started running seamlessly.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Rishi, thank you for filing this issue! It is indeed a blocker for us to move Marionette over to Python3. Would you also be interested to work on this bug? If yes, what would be your first steps?

Comment 2

[Rishi Gupta (:rgpt)]

Yes whimboo, I am interested in working on this bug. I would try make marionette harness work with py3 so as to keep it compatible with py2 so marionette test on py2 are not affected

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Great. So it's reserved for you. Let me know when you need help.

Comment 4

[Rishi Gupta (:rgpt)]

Based on the search It seems _ExpectedFailure Exception Class is removed from python3 unittest . And, as it was a internal variable it was not mentioned in the py3 porting guide.

_ExpectedFailure is only being used to raise an exception and call expected_failure after catching the exception. (as mentioned in the snippet below) Which could be directly called without raising an exception. The reason we could do that is because _ExpectedFailure is not being raised from anywhere testing module, but this (testcases.py) file

Another Solution would be create our custom Exception to mimic the one python2.7 and follow the same for both py2 and py3

@whimboo what should be the preferred course of action ?

IN : mozilla-unified/testing/marionette/harness/marionette_harness/marionette_test/testcases.py

try:
                if self.expected == "fail":
                    try:
                        self.setUp()
                    except Exception:
                        raise _ExpectedFailure(sys.exc_info())
                else:
                    self.setUp()
            except SkipTest as e:
                self._addSkip(result, str(e))
            except (KeyboardInterrupt, UnresponsiveInstanceException):
                raise
            except _ExpectedFailure as e:
                expected_failure(result, e.exc_info)
            except Exception:
                self._enter_pm()
                result.addError(self, sys.exc_info())
       ```

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Could you please check how _ExpectedFailure is implemented in Python2? If it's just some lines of code we could indeed just add it to our file.

Comment 6

[Rishi Gupta (:rgpt)]

Attachment: Bug 1595814 - [marionette] Don't import private _ExpectedFailure and _UnexpectedSuccess classes from unittest. r=whimboo

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Both newly filed bugs shouldn't actually block this bug from getting landed but bug 1355537 instead.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a3b107922e81
[marionette] Don't import private _ExpectedFailure and _UnexpectedSuccess classes from unittest. r=whimboo,marionette-reviewers

Comment 9

[Andrei Ciure[:aciure]]

Backed out changeset a3b107922e81 (bug 1595814) for causing marionette perma failures

push that caused the backout: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&revision=a3b107922e813e7930c6673095acfd146c305668

failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=fa5d65f365197e89eafc475b577fa5daac2eeef2&failure_classification_id=2

backout: https://hg.mozilla.org/integration/autoland/rev/01d092f87acb2d360caf147c802bb10932734d5e

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Rishi, it looks like this bug got under the wire and I completely missed to reply in regards of the backout. I wonder if you are still interested to work on it? Please let me know. Thanks a lot.

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Hm, ok. I assume not. Opening up this bug for others to pick up.

Comment 12

[rcrahul60]

Hi i would like to work on this if its still open?

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Hi rcrahul60. Thank you for your interest, but I have to admit that I missed to remove this bug from the mentored list given that it will be too hard to get fixed for someone new on the code base. We have plenty of other bugs, so maybe you can find something else?

Otherwise if you are an experienced Python developer and have knowledge with Python3 feel free to pick it up.

Thanks.

Comment 14

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1595814 - work around marionette dependency on private classes in Python2 unittest/case.py, r=whimboo.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/981d3e5d9a00
work around marionette dependency on private classes in Python2 unittest/case.py, r=whimboo,marionette-reviewers,maja_zf.

Comment 16

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/981d3e5d9a00