Closed Bug 1841348 Opened 1 year ago Closed 1 year ago

LDAP Addressbook broken (component does not have a method named: "promptPassword"')

Categories

(Thunderbird :: Address Book, defect, P1)

Thunderbird 115
Unspecified
All

Tracking

(thunderbird_esr102 unaffected, thunderbird_esr115+ fixed, thunderbird116 fixed)

VERIFIED FIXED
117 Branch
Tracking Status
thunderbird_esr102 --- unaffected
thunderbird_esr115 + fixed
thunderbird116 --- fixed

People

(Reporter: Fallen, Assigned: KaiE)

References

(Regression)

Details

(Keywords: regression)

Attachments

(2 files, 6 obsolete files)

I have an LDAP addressbook set up to the company directory. Trying to use it in any way I get this:

NS_ERROR_XPC_JSOBJECT_HAS_NO_FUNCTION_NAMED: JavaScript component does not have a method named: "promptPassword"'JavaScript component does not have a method named: "promptPassword"' when calling method: [nsIAuthPrompt::promptPassword] LDAPListenerBase.jsm:26
onLDAPInit resource:///modules/LDAPListenerBase.jsm:26
onOpen resource:///modules/LDAPConnection.jsm:38
_onOpen resource:///modules/LDAPClient.jsm:164

As a result, LDAP addressbook lookups don't work. I've tried this on a new profile for 115.0b6. I'd consider this a release blocker if confirmed by someone else.

Duplicate of this bug: 1841025

Fails for me also. As far back as 115.0b1.

2023-05-22 good 115.0a1
2023-05-23 bad 115.0a1

Severity: -- → S1
Flags: needinfo?(geoff)
Flags: needinfo?(benc)
OS: Unspecified → All
Priority: -- → P1
Version: unspecified → Thunderbird 115

correction

5-25 bad
5-24 good

Just to confirm, I see the same error, using the latest code build and using a test LDAP server as per: https://www.forumsys.com/2022/05/10/online-ldap-test-server/.

OK, looks to me that the nsIAuthPrompt being returned is a LoginManagerAuthPrompter object (defined in toolkit/components/passwordmgr/LoginManagerAuthPrompter.sys.mjs).

It doesn't seem to define a promptPassword() method, which would be the problem.
Confirmed. If I hack in a stub promptPassword(), it does get called.
No idea how it should be implemented though!

Worth noting that a comment at the top of LoginManagerAuthPrompter claims that nsIAuthPrompt is not used by firefox, so the breakage would have gone unnoticed there.

Flags: needinfo?(benc)

commit 05a4a11191f0 in m-c is where the change happened.

Basically it converts promptPassword() into asyncPromptPassword() (the implementation is almost identical, except for storing the newly-added password, which is now async, meaning everything above it now needs to be async too...).

I think we[1] should:

  1. replace all our uses of promptPassword() with the async asyncPromptPassword() (in c-c).
  2. remove promptPassword() from the nsIAuthPrompt interface entirely (in m-c).

[1] and when I say "we", I mean someone who knows what they're doing. Not me ; -)

Not disagreeing, but we'll need to be careful that async doesn't burn us somewhere.

Regressed by: 1832490

How about of going through the now defective nsIAuthPrompt you copy this code:
https://hg.mozilla.org/mozilla-central/file/e93b163fbaf1dafe0553a3501e4edaa545e522a2/toolkit/components/passwordmgr/LoginManagerAuthPrompter.jsm#l511
You'd also need to copy _getRealmInfo and _getFormattedOrigin.

nsIAuthPrompt was NOT changed.
nsIAuthPrompt.promptPassword still exists.

promptPassword was removed/renamed in toolkit/components/passwordmgr/LoginManagerAuthPrompter.sys.mjs

Contexts that call it and currently aren't async functions will need some kind of refactoring to allow async calling.

(In reply to Kai Engert (:KaiE:) from comment #9)

nsIAuthPrompt was NOT changed.
nsIAuthPrompt.promptPassword still exists.

The module providing those methods is LoginManagerAuthPrompter.sys.mjs. So despite (now incorrectly) being advertised in nsIAuthPrompt.idl, the function implementation no longer exists. A quick fix would be to remove XPCOM from the picture and copy promptPassword from the other version of LoginManagerAuthPrompter.sys.mjs. Or just go to async as BenC suggested.

I think this isn't covered by automated tests.

If I give you a partial patch, for the LDAP places, are you able to test? Or do you need a try run to test?

Philipp, are you able to test the attached patch?

I've tested it with an LDAP server, but I don't have available that requires password authentication.
I tried to simulate, and it might work, the new code waits for the prompt to complete.

Flags: needinfo?(philipp)

The attached patch D182670 doesn't apply to the 115 branch, because the code changed recently in bug 1833651.
https://hg.mozilla.org/comm-central/rev/a5c479e86dc7

Because that was simply cleanup, not a functional change, I suggest we backport it, to avoid the manual merging effort.

Depends on: 1833651

Here's a try build, it's a "115 branch" try build, so you can use it with your 115-branch profile:

https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=f787a55b7248d54dd18e9f8943edd024f141c54b

Wayne, Philipp, are you able to test?

Flags: needinfo?(vseerror)

Withdrawing my request. The patch doesn't work yet (breaks LDAP searches in other ways).

Flags: needinfo?(vseerror)
Flags: needinfo?(philipp)

FWIW, ldaps worked for me

(In reply to Wayne Mery (:wsmwk) from comment #18)

FWIW, ldaps worked for me

Are you referring to the earlier try build? Or are you referring to a different test scenario?

I have an updated patch, and this try build looks better:
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=76b453f00795005b64f18b0be4e5ff8246875aa7

Philipp, can you please test it?

Flags: needinfo?(philipp)

(In reply to betterbird.project+5 from comment #10)

The module providing those methods is LoginManagerAuthPrompter.sys.mjs. So despite (now incorrectly) being advertised in nsIAuthPrompt.idl, the function implementation no longer exists.

Maybe that statement is correct for the JS implementation, only?

We still have C++ code that calls nsIAuthPrompt::PromptPassword(), see
https://searchfox.org/comm-central/rev/5954897ba1b283c79bf2aaf4b295e765fb474ade/mailnews/base/src/nsMsgIncomingServer.cpp#753

If the implementation were indeed gone, wouldn't we fail at link time?

The impact may be less than I had assumed.

For example, SmtpServer.jsm calls promptPassword(), but I tested daily, and the SMTP code still works. So in some places our mail code uses a different prompt implementation.

Based on that, can we wait for more broken places to be discovered, and focus on the LDAP code for now?

(The attached patch also has a fix for some RNP code, but that was an easy change.)

(In reply to Kai Engert (:KaiE:) from comment #22)

The impact may be less than I had assumed.

I've removed the change to RNP. It's unnecessary.

Attachment #9342100 - Attachment description: WIP: Bug 1841348 - Use asyncPromptPassword for RNP and LDAP prompts. → WIP: Bug 1841348 - Use asyncPromptPassword for LDAP prompts.

Based on that, can we wait for more broken places to be discovered, and focus on the LDAP code for now?

All uses of promptPassword() and promptUsernameAndPassword():
https://searchfox.org/comm-central/search?q=promptPassword%28&path=&case=false&regexp=false
https://searchfox.org/comm-central/search?q=promptusernameandPassword%28&path=&case=false&regexp=false

This is totally confusing, three methods with the same name, nsIPrompt/Services.prompt (working), nsIAuthPrompt (broken) and TB's own copy.

calAuthUtils.jsm uses MsgAsyncPrompter.jsm, so it's not broken. imAccounts.sys.mjs uses nsIPrompt.promptPassword. You've already identified RNP, OpenPGP and S/MIME. However, RNP, OpenPGP use Services.prompt and shouldn't be broken. S/MIME's certFetchingStatus.js uses the same as LDAP and is therefore broken.

The mailnews code (incoming server, NNTP, IMAP, SMTP) appear to use MsgAsyncPrompter.jsm, which in fact has a copy of the M-C code made sync:
https://searchfox.org/comm-central/rev/5954897ba1b283c79bf2aaf4b295e765fb474ade/mailnews/base/src/MsgAsyncPrompter.jsm#272
https://searchfox.org/comm-central/rev/5954897ba1b283c79bf2aaf4b295e765fb474ade/mailnews/base/src/MsgAsyncPrompter.jsm#348

MAPI
https://searchfox.org/comm-central/search?q=prompt.*Password&path=msgMapiHook.cpp&case=false&regexp=true
uses nsIPrompt.promptPassword and shouldn't be broken.

We still have C++ code that calls nsIAuthPrompt::PromptPassword(),

No, it uses mailnew's own:
https://searchfox.org/comm-central/rev/5954897ba1b283c79bf2aaf4b295e765fb474ade/mailnews/base/src/nsMsgIncomingServer.cpp#721

In summary (and only by visual code inspection): Only LDAP and S/SMIME (certFetchingStatus.js) use nsIAuthPrompt and are hence broken.

See also:
https://searchfox.org/comm-central/search?q=Services.ww.getNewAuthPrompter&path=&case=false&regexp=false
This disconcertingly shows that MsgIncomingServer.jsm and SmtpServer.jsm use it as a now broken fallback.

You should see what happens when publishing a calendar to cover calendar/base/content/publish.js.

Instead of going async you might investigate switching to MsgAuthPrompt() (MsgAsyncPrompter.jsm) for now.

I've removed the change to RNP. It's unnecessary.

See above, the ones in OpenPGP are also unnecessary.

Here's another broken one:
https://searchfox.org/comm-central/rev/5954897ba1b283c79bf2aaf4b295e765fb474ade/mailnews/news/src/nsNewsFolder.cpp#1068,1112
We suggest switching to MsgAuthPrompt() since changing C++ code to async could be challenging (if possible, didn't look). To test, you need a news account with a password, available at news.eternal-september.org for example.

See above, the ones in OpenPGP are also unnecessary.

Sorry, this was in an earlier patch and has since been removed.

Here's another broken one:

Sorry, the NNTP code was another now defective fallback. Here's our suggestion, totally untested, but not worse than the current state:
https://github.com/Betterbird/thunderbird-patches/blob/main/115/bugs/1841348-auth-prompt.patch
Just switches to MsgAuthPrompt().

If you use your patch going all async, you still need the cleanup included here.

Here's our suggestion, totally untested, ...

We tested it now, working fine.

There has been no reply from Philipp yet, but I need to know if my patch works to make progress.

I concluded that I need to find a way to test myself.
I found a public LDAP server that supports authentication:
https://www.forumsys.com/2022/05/10/online-ldap-test-server/

Using that server works fine with TB 102 version.

Using Daily, with my current patch, it works once, but I run into more problems. On the second lookup attempt, we run into an exception in core loginmanager code, it complains about adding an existing login again, throws an exception, and our ldap lookup aborts. There's a problem even on the first attempt. When we attempt to fetch the S/MIME cert from LDAP, we run into another exception, extraction of the the cert from the response throws.

I'd like to keep this bug focused on the LDAP lookup failure.
Let's move discussions about other areas that need fixing (related to the prompt changes) to separate bugs.

(In reply to Kai Engert (:KaiE:) from comment #29)

Using Daily, with my current patch, it works once, but I run into more problems. On the second lookup attempt, we run into an exception in core loginmanager code, it complains about adding an existing login again, throws an exception, and our ldap lookup aborts.

Luckily I found the cause, it was caused by my patch, we need to send in an empty password result variable each time, or the login manager code will behave differently and fail.

There's a problem even on the first attempt. When we attempt to fetch the S/MIME cert from LDAP, we run into another exception, extraction of the the cert from the response throws.

I'll add a try/catch for that, if we cannot process the result, we can ignore it.

Assignee: nobody → kaie
Attachment #9342100 - Attachment description: WIP: Bug 1841348 - Use asyncPromptPassword for LDAP prompts. → Bug 1841348 - Use asyncPromptPassword for LDAP prompts. r=leftmostcat
Status: NEW → ASSIGNED

Test instructions:

Settings, Composition, Addressing, enable Directory Server, edit...
Add
Name: forumsys
host: ldap.forumsys.com
base dn: dc=example,dc=com
bind dn: cn=read-only-admin,dc=example,dc=com

When later prompted for a password, enter: password

Compose a message. Enter: Ein (and it should find Einstein)

For advanced additional testing, configure yourself an S/MIME certificate, and it should lookup for certs without exceptions (but won't find a cert).

If you want to test successful cert fetching, use
https://tu-dresden.de/zih/dienste/service-katalog/arbeitsumgebung/e_mail/pki/e_mail_ldap_dfn_pki

Had to start a slow full build, because artifact builds don't work yet:
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=f7579996728ac5961fa8eeef3daf7c284243d0d9

I don't know if using MsgAuthPrompt could introduce new side effects.

(In reply to Kai Engert (:KaiE:) from comment #35)

Had to start a slow full build, because artifact builds don't work yet:
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=f7579996728ac5961fa8eeef3daf7c284243d0d9

ldaps with password worked fine

Attachment #9342254 - Flags: review?(leftmostcat)
Attachment #9342254 - Flags: review?(leftmostcat)

Comment on attachment 9342254 [details] [diff] [review]
1841348-auth-prompt.patch

I'm marking this as obsolete for this bug, but portions of the patch might be reused in a follow-up bug (for the non-LDAP scenarios).

Attachment #9342254 - Attachment is obsolete: true
Attachment #9342254 - Flags: review?(leftmostcat)

Comment on attachment 9342279 [details] [diff] [review]
1841348-switch-ldap-smime-to-async.patch

using await is the right thing to do here.

I have not checked all callers of getPassword() etc. - they also must be called with await, because the function returns the password, and would not return a promise, which would fail silents, because it might be converted to a string and we'd try to use "[Object]" as password.

Attachment #9342279 - Flags: review+

Thanks for checking. getPassword() is local to the file and has only one caller:
https://searchfox.org/comm-central/rev/2aea6787d5e76036e12549516a5d8478cb3db298/mailnews/extensions/smime/certFetchingStatus.js#98

Compose a message. Enter: Ein (and it should find Einstein)

In order for that to work one needs to set the LDAP server as directory server in the compose settings. As noted above, "View Certificates of Recipients" finds no certificate for Einstein.

Comment on attachment 9342279 [details] [diff] [review]
1841348-switch-ldap-smime-to-async.patch

I'm sorry Ben, but we don't approve patches that don't pass through phabricator anymore.
They're a nightmare to track, land, and uplift properly and a moz-phab submit is very easy to do.

Kai's patch works and we can wait for a final review from Sean.

Attachment #9342279 - Flags: review?(leftmostcat)
Attachment #9342279 - Flags: review+

In addition to what Alex said, I have the following concerns with the attached patch:

  • it applies only to comm-esr115, but we need a patch that applies to comm-central.
    I have suggested to uplift the second patch from bug 1833651 to esr115,
    so we can have a common patch.
  • function onLDAPInit is defined in IDL as returning "void".
    By declaring it as "async" in JS, you're effectively changing its return type to a promise.
    I didn't use that approach, because I'm worried this unexpected return value could
    cause some of the JS/C++/XPCom code to stumble when processing the stack
    of return values.
    I don't know if my worry is justified. (I'll ask on #developers)
    However, my phabricator patch simply avoids this potential risk.

(In reply to Kai Engert (:KaiE:) from comment #49)

I didn't use that approach, because I'm worried this unexpected return value could
cause some of the JS/C++/XPCom code to stumble when processing the stack
of return values.
I don't know if my worry is justified. (I'll ask on #developers)

Based on initial reactions, the IDL would have to be changed to declare onLDAPInit() as returning a Promise (not void).

Thanks for the hint, return value changed to Promise.

Attachment #9342308 - Attachment is obsolete: true

Well, if you change the return type to Promise, you'd have to ensure that all implementations, everywhere, are changed to async. Which probably makes your code less simple than you had assumed it would be.

Also, you still don't include the bottom-most chunk from the phabricator patch, which is necessary to avoid an exception, and aborted processing, of missing certificate data.

I suggest that you stop adding attachments to this bug, unless the reviewer Sean/leftmostcat has started the review, and explicitly gives a clear signal that they would prefer your approach.

(In reply to betterbird.project+6 from comment #48)

This code is unmaintainable.

Please avoid that kind of general dismissive statements.

(In reply to Kai Engert (:KaiE:) from comment #49)

  • function onLDAPInit is defined in IDL as returning "void".
    By declaring it as "async" in JS, you're effectively changing its return type to a promise.
    I didn't use that approach, because I'm worried this unexpected return value could
    cause some of the JS/C++/XPCom code to stumble when processing the stack
    of return values.
    I don't know if my worry is justified. (I'll ask on #developers)

I got a comment from Mossop, suggesting that it should work to return a promise from an IDL function that declares return type void, because a JS function always returns something, even if it's just "undefined".

See Also: → 1841690

Comment on attachment 9342317 [details] [diff] [review]
1841348-switch-ldap-smime-to-async-c-c.patch

We shouldn't take any patches from betterbird, because of their threat of legal actions in bug 1841690 comment 4.

I will completely stop interacting with betterbird people because of that.

Attachment #9342317 - Attachment is obsolete: true
Attachment #9342317 - Flags: review?(leftmostcat)

Hi Kai, I don't know the whole context here, and the Thunderbird project is generally free to interact or not with somebody or another project, but from what I can see from the comment you cited, Betterbird was simply asking to keep the authorship information in git correctly, to not claim patches as somebody else having written it, when in fact it was a patch from Betterbird. Bad wording aside, that is a very reasonable request, fully in line with Open-Source customs and copyright law.

Personalities and Persons aside, the patch looks good to me. Using async/await indeed seems like the right approach to me for this problem.

I fully agree that it's a reasonable request to ask for the correct attribution.

I don't like that I was threatened with legal consequences.
It would have been better to make me aware in a friendly way that the authorshop information was missing in the patch.

I didn't pretend to have been the author of that patch, because in the initial comment in bug 1841690 I had referred to the origin of the patch, which includes the attribution.

Flags: needinfo?(geoff)
See Also: 1841690

I have moved my work to avoid the S/MIME certificate download exception to a separate patch.

See Also: → 1842033

Pushed by kaie@kuix.de:
https://hg.mozilla.org/comm-central/rev/6b6fc374f9f2
Use asyncPromptPassword for LDAP prompts. r=leftmostcat
https://hg.mozilla.org/comm-central/rev/b3df2118c727
Avoid an exception when S/MIME certificates cannot be downloaded from LDAP. r=leftmostcat

Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → 117 Branch

Fails for me on Mac - I don't get a password prompt. Works on Windows. Nothing in the error console.

Mac users - Philipp, others, does daily 117.0a1 2023-07-07 work?
Windows users - works for you?
https://archive.mozilla.org/pub/thunderbird/nightly/2023/07/2023-07-07-09-46-19-comm-central/

We still should have this for 115.0, straight to esr or delay 115. I'm OK with taking it to esr and beta at the same time if it tests OK on Windows.

Flags: needinfo?(soeren.hentzschel)
Flags: needinfo?(karl.rossing)

Success. My test on Mac was faulty - the ssl checkbox was unchecked in properties.

Still, further confirmations of success will be welcome.

Mac users - Philipp, others, does daily 117.0a1 2023-07-07 work?

Tested with the testing instructions of comment 32 on Thunderbird Daily 17.0a1 (2023-07-07) on macOS 13.4. Works for me.

Flags: needinfo?(soeren.hentzschel)

Comment on attachment 9342100 [details]
Bug 1841348 - Use asyncPromptPassword for LDAP prompts. r=leftmostcat

[Triage Comment]
Approved for beta
Approved for esr115

Attachment #9342100 - Flags: approval-comm-esr115+
Attachment #9342100 - Flags: approval-comm-beta+

Comment on attachment 9342530 [details]
Bug 1841348 - Avoid an exception when S/MIME certificates cannot be downloaded from LDAP. r=leftmostcat

[Triage Comment]
Approved for beta
Approved for esr115

Attachment #9342530 - Flags: approval-comm-esr115+
Attachment #9342530 - Flags: approval-comm-beta+

Apologies folks I missed the comments here, my bugzilla notifications are a mess and I had also been on PTO. I've tested in 116 beta and it works great for me, thanks!

Status: RESOLVED → VERIFIED
Flags: needinfo?(philipp)
Flags: needinfo?(karl.rossing)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: