Windows KeyError build failures in _emit_libs_derived with Python 3.8
Categories
(Firefox Build System :: General, defect)
Tracking
(firefox-esr68 unaffected, firefox74 unaffected, firefox75 unaffected, firefox76 unaffected, firefox77 verified)
Tracking | Status | |
---|---|---|
firefox-esr68 | --- | unaffected |
firefox74 | --- | unaffected |
firefox75 | --- | unaffected |
firefox76 | --- | unaffected |
firefox77 | --- | verified |
People
(Reporter: RyanVM, Assigned: saschanaz)
References
(Regression)
Details
(Keywords: regression)
Attachments
(2 files)
Local bisection is pointing at bug 1627163 as the culprit.
Traceback (most recent call last):
File "c:/Users/Ryan/repos/mozilla/configure.py", line 181, in <module>
sys.exit(main(sys.argv))
File "c:/Users/Ryan/repos/mozilla/configure.py", line 57, in main
return config_status(config)
File "c:/Users/Ryan/repos/mozilla/configure.py", line 176, in config_status
return config_status(args=[], **normalize(sanitized_config))
File "c:\Users\Ryan\repos\mozilla\python\mozbuild\mozbuild\config_status.py", line 145, in config_status
definitions = list(definitions)
File "c:\Users\Ryan\repos\mozilla\python\mozbuild\mozbuild\frontend\emitter.py", line 193, in emit
objs = list(self._emit_libs_derived(contexts))
File "c:\Users\Ryan\repos\mozilla\python\mozbuild\mozbuild\frontend\emitter.py", line 227, in _emit_libs_derived
collection = cls(contexts[root])
KeyError: 'C:/Users/Ryan/repos/mozilla/objdir-clang/dom/bindings'
Reporter | ||
Comment 1•4 years ago
|
||
I'm using python 3.8.2 if that matters.
Comment 2•4 years ago
|
||
Out of curiosity, if you insert a print(contexts)
right above that line, what are the contents of that dict? There's likely some easy fix we can do with path normalization here, but I don't know exactly what the issue is here and I can't reproduce the problem.
Comment 3•4 years ago
|
||
Does this patch fix the issue for you? For the record I'd prefer to fix the problem at the source and just make sure that whenever we emit any paths, they're always normalized from the jump, but today's a bit too busy of a day for me to do the "correct" structural fix :)
Reporter | ||
Comment 4•4 years ago
|
||
Same error as before, except it's an upper-case C:/ now :(
KeyError: 'C:/Users/Ryan/repos/mozilla/objdir-clang/dom/bindings'
Comment 5•4 years ago
|
||
We've isolated the issue to Python 3.8. Since 3.7 is the version we use in MozillaBuild I don't see this affecting many (or any) people before I'm able to put in a proper fix.
Reporter | ||
Updated•4 years ago
|
Updated•4 years ago
|
Assignee | ||
Comment 6•4 years ago
•
|
||
The keys of contexts
are same, its root
that differs. It starts with c:/
on Py3.7 but C:/
on Py3.8. (And posixpath.normpath
does nothing with it.)
Assignee | ||
Comment 7•4 years ago
•
|
||
The key change is that os.path.realpath(os.path.abspath('.'))
in 3.8 returns C:/...
while 3.7 returned c:/...
.
Updated•4 years ago
|
Comment 8•4 years ago
|
||
Err, mis-click on the assignee field there :)
Thanks for pushing that upstream bug :saschanaz!
Assignee | ||
Comment 9•4 years ago
|
||
os.path.realpath
in Python 3.8 now always uppercases Windows drive letter, while it was just an alias of os.path.abspath
in Windows. This patch uses .realpath()
consistently to get topobjdir
to fix the incompatibility from the behavior change.
Comment 10•4 years ago
|
||
Pushed by krosylight@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/58be14829411 Use os.path.realpath consistently r=rstewart
Comment 11•4 years ago
|
||
bugherder |
Reporter | ||
Updated•4 years ago
|
Updated•4 years ago
|
Updated•4 years ago
|
Description
•