Closed Bug 1706456 Opened 3 years ago Closed 3 years ago

Mach command returns ImportError: bad magic number in 'requests.packages': b'\x03\xf3\r\n'

Categories

(Firefox Build System :: Mach Core, defect)

defect

Tracking

(firefox-esr78 unaffected, firefox87 unaffected, firefox88 unaffected, firefox89 unaffected, firefox90 fixed)

RESOLVED FIXED
90 Branch
Tracking Status
firefox-esr78 --- unaffected
firefox87 --- unaffected
firefox88 --- unaffected
firefox89 --- unaffected
firefox90 --- fixed

People

(Reporter: nataliaCs, Assigned: mhentges)

References

(Regression)

Details

(Keywords: regression)

After running the command:
mfi browser_tabopen_squeeze.js where mfi == ./mach file-info bugzilla-component <path>
I have received the following error:

Traceback (most recent call last):
  File "./mach", line 153, in <module>
    main(sys.argv[1:])
  File "./mach", line 145, in main
    mach = check_and_get_mach(os.path.dirname(os.path.realpath(__file__)))
  File "./mach", line 132, in check_and_get_mach
    return load_mach(dir_path, mach_path)
  File "./mach", line 120, in load_mach
    return mach_bootstrap.bootstrap(dir_path)
  File "/home/moz/mozilla-unified/build/mach_bootstrap.py", line 222, in bootstrap
    import mach.main
  File "/home/moz/mozilla-unified/python/mach/mach/main.py", line 36, in <module>
    from .sentry import register_sentry, NoopErrorReporter
  File "/home/moz/mozilla-unified/python/mach/mach/sentry.py", line 13, in <module>
    from mach.telemetry import is_telemetry_enabled
  File "/home/moz/mozilla-unified/python/mach/mach/telemetry.py", line 14, in <module>
    import requests
  File "/home/moz/mozilla-unified/third_party/python/requests/requests/__init__.py", line 119, in <module>
    from . import packages
ImportError: bad magic number in 'requests.packages': b'\x03\xf3\r\n'

Same error was received when also running:

  • ./mach bootstrap
  • ./mach clobber

Mentioning that I have made no changes whatsoever to the env.

Regressed by: 1698616
Has Regression Range: --- → yes

Hmm, a little concerning, though I wonder if this is related to *.pyc files.
Can you try find . -name '*.pyc' -delete for me, then try running a ./mach command again?

🤞 Hoping that this doesn't affect too many devs 😅

Flags: needinfo?(ncsoregi)

(In reply to Mitchell Hentges [:mhentges] 🦀 from comment #1)

Hmm, a little concerning, though I wonder if this is related to *.pyc files.
Can you try find . -name '*.pyc' -delete for me, then try running a ./mach command again?

🤞 Hoping that this doesn't affect too many devs 😅

It works!
Thank you

Flags: needinfo?(ncsoregi)

I'm going to leave this open for a bit - if anybody else runs into this, please leave a comment so I can gauge severity.

Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → DUPLICATE
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---

Multiple people are hitting it (also reported on Slack).

Status: REOPENED → NEW
Flags: needinfo?(mhentges)
Keywords: regression
Summary: Mach file-info command returns ImportError: bad magic number in 'requests.packages': b'\x03\xf3\r\n' → Mach command returns ImportError: bad magic number in 'requests.packages': b'\x03\xf3\r\n'

Comment #1 helped me (removing all the *.pyc files), thank you. I am on Win10
Honza

The Bugbug bot thinks this bug should belong to the 'Firefox Build System::Mach Core' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Component: General → Mach Core

Got it too, on Linux.

This should be fixed in the latest central commit. Please update mozilla-central if you hit the issue.

Flags: needinfo?(mhentges)

Resolved by backing out the regressing patch.

Status: NEW → RESOLVED
Closed: 3 years ago3 years ago
Resolution: --- → FIXED
Assignee: nobody → mhentges
Target Milestone: --- → 90 Branch

This was caused by old Python 2 .pyc files hanging around.
It can be reproduced by using Python 2 to run a Mach command that imports requests, updating to b413fc, then using Python 3 to run a Mach command that imports requests.
Interestingly, using Python 2 again after updating to b413fc fails once, then everything works. When it fails once, it has the following backtrace:

Traceback (most recent call last):
  File "./mach", line 153, in <module>
    main(sys.argv[1:])
  File "./mach", line 145, in main
    mach = check_and_get_mach(os.path.dirname(os.path.realpath(__file__)))
  File "./mach", line 132, in check_and_get_mach
    return load_mach(dir_path, mach_path)
  File "./mach", line 120, in load_mach
    return mach_bootstrap.bootstrap(dir_path)
  File "/home/mitch/dev/firefox/build/mach_bootstrap.py", line 221, in bootstrap
    import mach.base
  File "/home/mitch/dev/firefox/build/mach_bootstrap.py", line 468, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/mitch/dev/firefox/python/mach/mach/base.py", line 7, in <module>
    from mach.telemetry import NoopTelemetry
  File "/home/mitch/dev/firefox/build/mach_bootstrap.py", line 468, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/mitch/dev/firefox/python/mach/mach/telemetry.py", line 69, in <module>
    import requests
  File "/home/mitch/dev/firefox/build/mach_bootstrap.py", line 468, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/mitch/dev/firefox/third_party/python/requests/requests/__init__.py", line 119, in <module>
    from . import packages
  File "/home/mitch/dev/firefox/build/mach_bootstrap.py", line 468, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/home/mitch/dev/firefox/third_party/python/requests/requests/packages/__init__.py", line 33, in <module>
  File "/home/mitch/dev/firefox/build/mach_bootstrap.py", line 506, in __call__
    module = self(name, globals, locals, fromlist, level)
  File "/home/mitch/dev/firefox/build/mach_bootstrap.py", line 468, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
SystemError: Parent module 'requests.packages' not loaded, cannot perform relative import

This appears to be because requests/packages/__init__.py was removed as part of this upgrade.
However, Python 3 sees the old Python 2 pyc file and uses it, even though its associated .py file is gone.
When the command is run with Python 2 and fails once, the pyc file is deleted, which is why it works afterwards.

See Also: → 1708547
You need to log in before you can comment on or make changes to this bug.