1672940 - Doing osx cross builds fails with biplist.InvalidPlistException: module 'plistlib' has no attribute 'Data'

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Emilio Cobos Álvarez (:emilio)]

This happens during configure...

0:04.20 Traceback (most recent call last):
 0:04.20   File "/home/emilio/src/moz/gecko-3/third_party/python/biplist/biplist/__init__.py", line 126, in readPlist
 0:04.20     result = reader.parse()
 0:04.20   File "/home/emilio/src/moz/gecko-3/third_party/python/biplist/biplist/__init__.py", line 234, in parse
 0:04.20     return self.readRoot()
 0:04.20   File "/home/emilio/src/moz/gecko-3/third_party/python/biplist/biplist/__init__.py", line 248, in readRoot
 0:04.20     raise NotBinaryPlistException()
 0:04.20 biplist.NotBinaryPlistException
 0:04.20 During handling of the above exception, another exception occurred:
 0:04.20 Traceback (most recent call last):
 0:04.20   File "/home/emilio/src/moz/gecko-3/third_party/python/biplist/biplist/__init__.py", line 141, in readPlist
 0:04.20     result = wrapDataObject(result, for_binary=True)
 0:04.20   File "/home/emilio/src/moz/gecko-3/third_party/python/biplist/biplist/__init__.py", line 154, in wrapDataObject
 0:04.21     elif isinstance(o, (bytes, plistlib.Data)) and for_binary:
 0:04.21 AttributeError: module 'plistlib' has no attribute 'Data'
 0:04.21 During handling of the above exception, another exception occurred:
 0:04.21 Traceback (most recent call last):
 0:04.21   File "/home/emilio/src/moz/gecko-3/configure.py", line 216, in <module>
 0:04.21     sys.exit(main(sys.argv))
 0:04.21   File "/home/emilio/src/moz/gecko-3/configure.py", line 57, in main
 0:04.21     sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))
 0:04.21   File "/home/emilio/src/moz/gecko-3/python/mozbuild/mozbuild/configure/__init__.py", line 500, in run
 0:04.21     func(*args)
 0:04.21   File "/home/emilio/src/moz/gecko-3/python/mozbuild/mozbuild/configure/__init__.py", line 544, in _value_for
 0:04.21     return self._value_for_depends(obj)
 0:04.21   File "/home/emilio/src/moz/gecko-3/python/mozbuild/mozbuild/util.py", line 1021, in method_call
 0:04.21     cache[args] = self.func(instance, *args)
 0:04.21   File "/home/emilio/src/moz/gecko-3/python/mozbuild/mozbuild/configure/__init__.py", line 553, in _value_for_depends
 0:04.21     value = obj.result()
 0:04.21   File "/home/emilio/src/moz/gecko-3/python/mozbuild/mozbuild/util.py", line 1021, in method_call
 0:04.21     cache[args] = self.func(instance, *args)
 0:04.21   File "/home/emilio/src/moz/gecko-3/python/mozbuild/mozbuild/configure/__init__.py", line 155, in result
 0:04.21     return self._func(*resolved_args)
 0:04.21   File "/home/emilio/src/moz/gecko-3/python/mozbuild/mozbuild/configure/__init__.py", line 1208, in wrapped
 0:04.21     return new_func(*args, **kwargs)
 0:04.21   File "/home/emilio/src/moz/gecko-3/build/moz.configure/toolchain.configure", line 162, in macos_sdk
 0:04.21     obj = readPlist(os.path.join(sdk, 'SDKSettings.plist'))
 0:04.21   File "/home/emilio/src/moz/gecko-3/third_party/python/biplist/biplist/__init__.py", line 143, in readPlist
 0:04.21     raise InvalidPlistException(e)
 0:04.21 biplist.InvalidPlistException: module 'plistlib' has no attribute 'Data'

I'm not sure where it started, it might have been my system upgrade to python 3.9 or something.

Something like this fixes it:

diff --git a/third_party/python/biplist/biplist/__init__.py b/third_party/python/biplist/biplist/__init__.py
index f9d5836dd246..30a7be6f27a0 100644
--- a/third_party/python/biplist/biplist/__init__.py
+++ b/third_party/python/biplist/biplist/__init__.py
@@ -150,8 +150,8 @@ def wrapDataObject(o, for_binary=False):
     if isinstance(o, Data) and not for_binary:
         v = sys.version_info
         if not (v[0] >= 3 and v[1] >= 4):
-            o = plistlib.Data(o)
-    elif isinstance(o, (bytes, plistlib.Data)) and for_binary:
+            o = Data(o)
+    elif isinstance(o, (bytes, Data)) and for_binary:
         if hasattr(o, 'data'):
             o = Data(o.data)
     elif isinstance(o, tuple):

But it's third-party code so I don't know what the right way to update it is.

Comment 1

[Emilio Cobos Álvarez (:emilio)]

This is a better fix:

diff --git a/third_party/python/biplist/biplist/__init__.py b/third_party/python/biplist/biplist/__init__.py
index f9d5836dd246..8a37d3465c0d 100644
--- a/third_party/python/biplist/biplist/__init__.py
+++ b/third_party/python/biplist/biplist/__init__.py
@@ -147,11 +147,12 @@ def readPlist(pathOrFile):
     return result
 
 def wrapDataObject(o, for_binary=False):
+    v = sys.version_info
+    has_plistlib_data = not (v[0] >= 3 and v[1] >= 4)
     if isinstance(o, Data) and not for_binary:
-        v = sys.version_info
-        if not (v[0] >= 3 and v[1] >= 4):
+        if has_plistlib_data:
             o = plistlib.Data(o)
-    elif isinstance(o, (bytes, plistlib.Data)) and for_binary:
+    elif has_plistlib_data and isinstance(o, (bytes, plistlib.Data)) and for_binary:
         if hasattr(o, 'data'):
             o = Data(o.data)
     elif isinstance(o, tuple):

Comment 2

[Emilio Cobos Álvarez (:emilio)]

Attachment: Bug 1672940 - Remove biplist. r=#build

biplist broke with python 3.9, but we don't need this anymore because
Python3's plistlib allows reading binary plists since python 3.4.

I've tested this with all sdks that I have and we support (10.11-10.15).

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ea01ecd10760
Remove biplist. r=firefox-build-system-reviewers,glandium

Comment 4

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/ea01ecd10760

Comment 6

[Emilio Cobos Álvarez (:emilio)]

Comment on attachment 9183423 [details]
Bug 1672940 - Remove biplist. r=#build

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: Build fix to help devs rebase patches to esr on OSX
• User impact if declined: None
• Fix Landed on Version:
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Build time only
• String or UUID changes made by this patch: none

Comment 7

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9183423 [details]
Bug 1672940 - Remove biplist. r=#build

Approved for 78.9esr.

Comment 8

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr78/rev/a0e4ef87b834