1242653 - Pulls error out with "abort: No module named simplejson!" on windows

Status: RESOLVED FIXED

Description

[:Gijs (blocked reviews; no response to ni until Nov 2 (PTO))]

Specifically:

    Gijs@hermes /e/dev/mozilla-central
    $ hg pull --traceback
    pulling from https://hg.mozilla.org/mozilla-central/
    searching for changes
    adding changesets
    adding manifests
    adding file changes
    added 5290 changesets with 33091 changes to 14968 files
    added 158 pushes
    Traceback (most recent call last):
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\dispatch.py", line 187, in _runcatch
        return _dispatch(req)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\dispatch.py", line 920, in _dispatch
        cmdpats, cmdoptions)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\dispatch.py", line 679, in runcommand
        ret = _runcommand(ui, options, cmd, d)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\extensions.py", line 183, in closure
        return func(*(args + a), **kw)
      File "e:\mozilla-build\python\Lib\site-packages\hgext\color.py", line 525, in colorcmd
        return orig(ui_, opts, cmd, cmdfunc)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\extensions.py", line 183, in closure
        return func(*(args + a), **kw)
      File "e:\mozilla-build\python\Lib\site-packages\hgext\pager.py", line 139, in pagecmd
        return orig(ui, options, cmd, cmdfunc)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\dispatch.py", line 1051, in _runcommand
        return checkargs()
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\dispatch.py", line 1011, in checkargs
        return cmdfunc()
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\dispatch.py", line 917, in <lambda>
        d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\util.py", line 801, in check
        return func(*args, **kwargs)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\extensions.py", line 183, in closure
        return func(*(args + a), **kw)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\util.py", line 801, in check
        return func(*args, **kwargs)
      File "c:/Users/Gijs/.mozbuild/version-control-tools/hgext\firefoxtree\__init__.py", line 439, in pullcommand
        return orig(ui, repo, source=source, **opts)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\util.py", line 801, in check
        return func(*args, **kwargs)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\extensions.py", line 183, in closure
        return func(*(args + a), **kw)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\util.py", line 801, in check
        return func(*args, **kwargs)
      File "e:\mozilla-build\python\Lib\site-packages\hgext\mq.py", line 3525, in mqcommand
        return orig(ui, repo, *args, **kwargs)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\util.py", line 801, in check
        return func(*args, **kwargs)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\extensions.py", line 183, in closure
        return func(*(args + a), **kw)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\util.py", line 801, in check
        return func(*args, **kwargs)
      File "c:/Users/Gijs/.mozbuild/mutable-history/hgext/evolve.py", line 726, in warnobserrors
        ret = orig(ui, repo, *args, **kwargs)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\util.py", line 801, in check
        return func(*args, **kwargs)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\extensions.py", line 183, in closure
        return func(*(args + a), **kw)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\util.py", line 801, in check
        return func(*args, **kwargs)
      File "c:/Users/Gijs/.mozbuild/mutable-history/hgext/evolve.py", line 698, in wrapmayobsoletewc
        res = origfn(ui, repo, *args, **opts)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\util.py", line 801, in check
        return func(*args, **kwargs)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\extensions.py", line 183, in closure
        return func(*(args + a), **kw)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\util.py", line 801, in check
        return func(*args, **kwargs)
      File "e:\mozilla-build\python\Lib\site-packages\hgext\rebase.py", line 1143, in pullrebase
        orig(ui, repo, *args, **opts)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\util.py", line 801, in check
        return func(*args, **kwargs)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\commands.py", line 5324, in pull
        opargs=pullopargs).cgresult
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\extensions.py", line 183, in closure
        return func(*(args + a), **kw)
      File "c:/Users/Gijs/.mozbuild/version-control-tools/hgext\bundleclone\__init__.py", line 590, in pull
        res = orig(repo, remote, *args, **kwargs)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\extensions.py", line 183, in closure
        return func(*(args + a), **kw)
      File "c:/Users/Gijs/.mozbuild/version-control-tools/hgext\firefoxtree\__init__.py", line 337, in pull
        res = orig(repo, remote, *args, **kwargs)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\extensions.py", line 183, in closure
        return func(*(args + a), **kw)
      File "c:/Users/Gijs/.mozbuild/version-control-tools/hgext\mozext\__init__.py", line 794, in pull
        repo.changetracker.load_pushlog(tree)
      File "c:\Users\Gijs\.mozbuild\version-control-tools\pylib\mozautomation\mozautomation\changetracker.py", line 127, in load_pushlog
        for push_id, push in repo.push_info(start_id=last_push_id + 1):
      File "c:\Users\Gijs\.mozbuild\version-control-tools\pylib\mozautomation\mozautomation\repository.py", line 285, in push_info
        pushes = requests.get(url).json()
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\demandimport.py", line 130, in __getattribute__
        self._load()
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\demandimport.py", line 96, in _load
        mod = _hgextimport(_import, head, globals, locals, None, level)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\demandimport.py", line 53, in _hgextimport
        return importfunc(name, globals, *args, **kwargs)
      File "c:\Users\Gijs\.mozbuild\version-control-tools\pylib\requests\requests\__init__.py", line 59, in <module>
        from .models import Request, Response, PreparedRequest
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\demandimport.py", line 192, in _demandimport
        fromlist, level)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\demandimport.py", line 53, in _hgextimport
        return importfunc(name, globals, *args, **kwargs)
      File "c:\Users\Gijs\.mozbuild\version-control-tools\pylib\requests\requests\models.py", line 49, in <module>
        json_dumps = json.dumps
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\demandimport.py", line 130, in __getattribute__
        self._load()
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\demandimport.py", line 96, in _load
        mod = _hgextimport(_import, head, globals, locals, None, level)
      File "e:\mozilla-build\python\Lib\site-packages\mercurial\demandimport.py", line 53, in _hgextimport
        return importfunc(name, globals, *args, **kwargs)
    ImportError: No module named simplejson
    abort: No module named simplejson!

This happens on every pull. gps says that to workaround, setting HGDEMANDIMPORT=disable should work.

Comment 1

[Gregory Szorc [:gps] (away until ~October 23)]

There are multiple bugs here.

First, this appears to be a regression from bug 1241219. hg'd demand importer doesn't play nice with "requests." We need to disable the demand importer when importing requests or any module that imports requests.

Second, we shouldn't be performing a HTTP request at all. This indicates that bug 1236618 didn't prevent the HTTP fetching at pull time and thus is only a partial fix.

Comment 2

[Gregory Szorc [:gps] (away until ~October 23)]

Created attachment 8711804 [details]
MozReview Request: mozext: disable demand importer when importing mozautomation (bug 1242653); r?dminor

Because "requests" doesn't play nice with the demand importer. This
fixes a regression from 4b953dc90fa0.

Review commit: https://reviewboard.mozilla.org/r/32307/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/32307/

Comment 3

[Gregory Szorc [:gps] (away until ~October 23)]

Created attachment 8711805 [details]
MozReview Request: mozext: don't load pushlog twice (bug 1242653); r?dminor

5410683439c5 added pushlog fetching elsewhere in this file. The
code removed in this patch was redundantly fetching the pushlog data
from the JSON HTTP API after it had already been fetched.

Review commit: https://reviewboard.mozilla.org/r/32309/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/32309/

Comment 4

[Dan Minor [:dminor]]

Comment on attachment 8711804 [details]
MozReview Request: mozext: disable demand importer when importing mozautomation (bug 1242653); r?dminor

https://reviewboard.mozilla.org/r/32307/#review28971

Comment 5

[Dan Minor [:dminor]]

Comment on attachment 8711805 [details]
MozReview Request: mozext: don't load pushlog twice (bug 1242653); r?dminor

https://reviewboard.mozilla.org/r/32309/#review28975

Comment 6

[Gregory Szorc [:gps] (away until ~October 23)]

Twas autolanded.

Thanks for the quick review, dminor!