1597737 - [py3] ImportError: cannot import name 'urlparse'

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect)

Description

[Rishi Gupta (:rgpt)]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0

Steps to reproduce:

Tried to run marionette harness on py3

Actual results:

import urlparse
does not work on py3

Expected results:

should be working on py3 keeping it compatible to py2
ImportError: cannot import name 'urlparse'

Comment 1

[Rishi Gupta (:rgpt)]

in testing/marionette/harness/marionette_harness/runner/httpd.py

Comment 2

[Rishi Gupta (:rgpt)]

Attachment: Detailed Error Log

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Rishi, feel free to add the full stack trace as comment inside of triple-backticks. That way it is formatted as code and is query-able in Bugzilla.

ImportError: No module named 'urlparse'

  File "/home/rgpt/Desktop/mozilla/mozilla-unified/python/mach/mach/main.py", line 360, in run
    return self._run(argv)
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/python/mach/mach/main.py", line 415, in _run
    args = parser.parse_args(argv)
  File "/usr/lib/python3.5/argparse.py", line 1735, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python3.5/argparse.py", line 1767, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib/python3.5/argparse.py", line 1976, in _parse_known_args
    stop_index = consume_positionals(start_index)
  File "/usr/lib/python3.5/argparse.py", line 1932, in consume_positionals
    take_action(action, args)
  File "/usr/lib/python3.5/argparse.py", line 1841, in take_action
    action(self, namespace, argument_values, option_string)
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/python/mach/mach/dispatcher.py", line 179, in __call__
    if handler.parser:
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/python/mach/mach/decorators.py", line 77, in parser
    self._parser = self._parser()
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/testing/marionette/mach_commands.py", line 26, in create_parser_tests
    from marionette_harness.runtests import MarionetteArguments
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/build/mach_bootstrap.py", line 431, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/build/mach_bootstrap.py", line 431, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/testing/marionette/harness/marionette_harness/__init__.py", line 23, in <module>
    from .runner import (
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/build/mach_bootstrap.py", line 431, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/testing/marionette/harness/marionette_harness/runner/__init__.py", line 7, in <module>
    from .base import (
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/build/mach_bootstrap.py", line 431, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/testing/marionette/harness/marionette_harness/runner/base.py", line 34, in <module>
    from . import serve
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/build/mach_bootstrap.py", line 431, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/build/mach_bootstrap.py", line 431, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/testing/marionette/harness/marionette_harness/runner/serve.py", line 21, in <module>
    from . import httpd
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/build/mach_bootstrap.py", line 431, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/build/mach_bootstrap.py", line 431, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/testing/marionette/harness/marionette_harness/runner/httpd.py", line 19, in <module>
    import urlparse
  File "/home/rgpt/Desktop/mozilla/mozilla-unified/build/mach_bootstrap.py", line 431, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)

Fixing the dependency given that it blocks the Marionette harness but not the driver.

Comment 4

[Rishi Gupta (:rgpt)]

I will follow that from next time.

Comment 5

[guow10]

Hi Rishi ~
The python3.x version has merged urllib2, urlparse, and robotparser into the urllib module, which contains 5 submodules, as follows:
urllib.error, urllib.parse, urllib.request, urllib.response, urllib.robotparser.
Solution:
import urlparse
my_url = urlparse.urlparse(url)
change to
from urllib import parse
my_url = parse.urlparse(url)

or you just change the import line base on your codes like
from urllib.parse import urlparse

I hope that helps you!

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Thanks guow10. We are most likely going to make use of six here, which would allow to import the module via: import six.moves.urllib.

Comment 7

[Rishi Gupta (:rgpt)]

Attachment: Bug 1597737 : Importing urlparse from six to keep py2, py3 compatible r=whimboo

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5fe975a91529
Importing urlparse from six to keep py2, py3 compatible r=whimboo,marionette-reviewers

Comment 9

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/5fe975a91529