1700349 - OTR library comm-central macOS doesn't work

Status: RESOLVED FIXED

(Chat Core :: Security: OTR, defect)

Description

[Kai Engert (:KaiE:)]

libotr.5.dylib in comm-central is broken.

Use a current comm-central build (downloaded from treeherder, target.dmg)
On macOS, error console reports that the OTR library cannot be loaded.

For testing purposes, I took a libotr.5.dylib file from the 78.x release, copied it into a the TB Daily folder, and that library works, it's reported as loaded in error console.

Comment 1

[Rob Lemley [:rjl]]

I ran into this earlier while testing bug 1700240.

I suspect this is a problem with the macOS 10.12 SDK that we switched to a couple of versions ago. It could also be something with the enhanced permissions possibly.

I was testing on macOS 11.0.1 earlier (intel) -- I'll try 10.15.

Comment 2

[Rob Lemley [:rjl]]

Attachment: image.png

This looks like it could be the problem... That's a libgcrypt symbol, libotr was built with libgcrypt 1.9.2 in this case.

Comment 3

[Rob Lemley [:rjl]]

Attachment: gpg-error.patch

Comment 4

[Rob Lemley [:rjl]]

Attachment: libgcrypt.patch

Comment 5

[Rob Lemley [:rjl]]

Attachment: Bug 1700349 - Use --disable-asm for x86_64 macOS libgcrypt. r=kaie

With gpg-error 1.42 and libgcrypt 1.9.2, libgcrypt does not compile with
ASM enabled. This manifested as a failure to initialize libotr previously.

Comment 6

[Rob Lemley [:rjl]]

Attachment: Bug 1700349 - Add a very simplistic test to verify OTRLib.jsm can load libotr. r=kaie

Depends on D109632

Comment 7

[Rob Lemley [:rjl]]

Try build: https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=08c53cf474bad94e13dea3661a1a79c074efcb12

I put the third_party code patches in Bugzilla since Phabricator tends to have problems with them.

Comment 8

[Kai Engert (:KaiE:)]

Comment on attachment 9211286 [details] [diff] [review]
gpg-error.patch

r=kaie this matches what I downloaded

Comment 9

[Kai Engert (:KaiE:)]

Comment on attachment 9211287 [details] [diff] [review]
libgcrypt.patch

r=kaie this matches what I downloaded

Comment 10

[Kai Engert (:KaiE:)]

What failure do you see when building with asm enabled?

Comment 11

[Rob Lemley [:rjl]]

I confirmed that libotr loads on macOS 11.x with these changes.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/783a94bc4803
Update libgpg-error to 1.42. r=kaie
https://hg.mozilla.org/comm-central/rev/aae1f93eb963
Update libgcrypt to 1.9.2. r=kaie
https://hg.mozilla.org/comm-central/rev/898e2c255d17
Use --disable-asm for x86_64 macOS libgcrypt. r=kaie
https://hg.mozilla.org/comm-central/rev/2370fdbabbde
Add a very simplistic test to verify OTRLib.jsm can load libotr. r=kaie

Comment 13

[Geoff Lankow (:darktrojan)]

The toolchain-linux-aarch64-libotr task is failing. I assume it's something to do with this bug as that's when the failure appeared. The task didn't run in your Try job so you wouldn't have seen it.

Comment 14

[Rob Lemley [:rjl]]

Attachment: Bug 1700349 - Follow-up: Fix linux-aarch64 libotr building. r=justdave

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/2fb031f034ae
Follow-up: Fix linux-aarch64 libotr building. rs=bustage-fix

Comment 16

[Rob Lemley [:rjl]]

I verified the linux-aarch64 build is able to load libotr from this build. (Manjaro ARM, Raspi 400).