Open Bug 1599029 (Heidelberg-autoconfig) Opened 5 years ago Updated 2 years ago

ISP defined configuration to use IMAP+ SMTP is not honoured for exchange servers

Categories

(Thunderbird :: General, defect)

Product:
Thunderbird
Component:
General
Type:
defect

Tracking

(thunderbird_esr68 affected)

Tracking Flags:
Tracking Status
thunderbird_esr68 --- affected

People

(Reporter: mkmelin, Assigned: BenB)

References

(Regression)

Details

(Keywords: regression)

Attachments

(5 files)

Heidelberg.png
16.80 KB, image/png
Details
Heidelberg-2.png
12.58 KB, image/png
Details
Heidelberg-3.png
18.26 KB, image/png
Details
Heidelberg, wrong password, with bugfix 1593122, shows prompt as expected
39.58 KB, image/png
Details
Show username field only when the other methods fail, v4, with status message
4.52 KB, text/plain
BenB
: feedback-
Details

From bug 1500105.

STR: try to set up an account for abc@medma.uni-heidelberg.de. Skip the password, or fill it in incorrectly.

If you set mailnews.auto_config.fetchFromExchange.enabled false, it will find the ISP defined configuration for IMAP and SMTP.

Unfortunately, with mailnews.auto_config.fetchFromExchange.enabled true (the default) you'll find nothing. This has a ISP defined configuration set up to use IMAP+SMTP so let's please skip the discussion on whether they wanted users to use it or not.

Looks like the failure is due to the exchange autoconfig results fetch causing an auth failure, which causes the failure mode, overriding the ISP defined configuration that was already found.

Expected results: the IMAP and SMTP configuration will show up and let the user use those. They have configured the username to instruct the user to use their student id - so yes a manual step is required, but apparently the uni (ISP) didn't want to map email addresses to ids so opted for this solution.

Assignee: nobody → ben.bucksch

[Exchange email address]

Skip the password, or fill it in incorrectly.

That's not a valid test case. For Exchange servers, you MUST enter the password, to find a valid config. Microsoft invented it like that. Please complain to them.

with mailnews.auto_config.fetchFromExchange.enabled true (the default) you'll find nothing.

That's not true. You will get a prompt asking you for the username. If you enter the correct username and password, you will get the right config.

This bug is like saying "I entered the wrong password, and I couldn't see my mail". Yeah, well, of course. That's by design. Likewise, by Microsoft's design, you need a valid login even to configure an Exchange account.

-> INVALID

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → INVALID

Clearly not invalid. The ISP (the university) said please use IMAP/SMTP and set up a configuration file to make its users do so. There's no reason Thunderbird couldn't find configuration and let the user select it.

Status: RESOLVED → REOPENED
Resolution: INVALID → ---

Somehow there seems a "quality" issue with this bug :-(

The reporter didn't tell us which ISP configuration he was referring to, perhaps this one:
http://autoconfig.medma.uni-heidelberg.de/mail/config-v1.1.xml

The he didn't tell us which version he was using. I'm using TB 68.3 out of the box with mailnews.auto_config.fetchFromExchange.enabled true and trying to set up abc@medma.uni-heidelberg.de with no password gives me exchange.uni-heidelberg.de and mail.urz.uni-heidelberg.de, like is written in the XML. So no problem.

The same happens for trunk. So where is the problem? What am I missing? Oh, I see it now, it behaves differently with an invalid password, at least on trunk. There I get the Exchange/Domain login box (Your login).

So gentlemen, how about we take deep breath (breathe in, breathe out, repeat until cooled down) and make sure that:
a) we report bugs with complete details to make them reproducible
b) look at the issue carefully before dismissing it.

BTW, I think this is a bug: The config says IMAP and SMTP, so that's what we should do despite the Exchange server under the cover.

Status: REOPENED → NEW

Clearly not invalid

Steps for reproduction are invalid. Password is required to configure this account.

Here's what you're missing: To get the correct config for an Exchange server, you MUST enter the correct username and password. It won't work otherwise.

Any other configs that we might find may not be valid, and may not work.

http://autoconfig.medma.uni-heidelberg.de/mail/config-v1.1.xml

I've actually personally worked with the admin on that domain. That's the old config before our username field fix. He specifically did not want this config to be used. What Magnus demands here is in direct contradiction to the expressed wish of the admins of this domain. In this case, I happen to be certain, because I personally spoke to them.

We added the username field, because this is not an edge case, but a very common case for on-premise Exchange server to require Windows-Domain-login style usernames, and to require a login before we can find the correct config. In fact, that is - as far as I know - the standard configuration for on-premise Exchange.

That's why I am 100% certain that this bug is not only INVALID as stated, but that Magnus' requests here and in various related bugs (e.g. bug 1593122) will actually break currently working configs. It will destroy the specific fixes we made for these Exchange servers.

So you're saying uni-heidelberg.de want so pay 10e/user a year to you so that their users could use Thunderbird when they could use it for free? That seems... like something hard to sell to management?

Jörg, the "behaves differently" is just a race condition AFAIKT.

(In reply to Ben Bucksch (:BenB) from comment #5)

Here's what you're missing: To get the correct config for an Exchange server, you MUST enter the correct username and password. It won't work otherwise.

That's besides the point. It will work with wrong/missing password for the thunderbird specific config client they explicitely put on their autoconfig server. And for free!

Ben, sorry, you've lost me.

There is a server that happens to be an Exchange server which exclusively advertises IMAP and SMTP via config-v1.1.xml. So why wouldn't TB behave like TB 52 (without bug 1500105) and show IMAP and SMTP, like it actually does with a blank password. Why does a wrong password suddenly send it off into Exchange land?

Looks like you're saying that this server not only supplies this config-v1.1.xml file but also an AutoDiscover file with a heap of Exchange stuff in it and that's triggered by whether the password is empty or not?

Maybe we should stop and agree on the defined behaviour in all cases. To me it looks like this server offers IMAP and SMTP via the config-v1.1.xml file and maybe Exchange via the AutoDiscover file, so it's a clear variation of bug 1592258. For now, IMAP should be offered, and maybe in the future, Exchange as an option.

So you're saying uni-heidelberg.de want to pay 10e/user a year to you so that their users could use Thunderbird when they could use it for free?

Magnus, sadly may unis, like Aalto and FU Berlin, have a deal with Microsoft. Their user get Office including Outlook for free.

this server not only supplies this config-v1.1.xml file but also an AutoDiscover file with a heap of Exchange stuff in it and that's triggered by whether the password is empty or not?

Like I said, the config-v1.1.xml was a band-aid they put up for old Thunderbird versions, before the username field fix. They were not happy with that setup. They pleaded the case to me, and it's a widespread problem. Most other Exchange admins do not put up such a Thunderbird-specific config. We'll just be broken.

See comment 1 and comment 5. I can't explain it any better than that.

Hmm, which "username field fix"? Bug 1518155?

I still think we need to define the desired behaviour for Exchange servers:

Like in the case of Aalto.fi (bug 1598861) where IMAP/POP3 and SMTP are not advertised, we should probe the server from the AutoDiscover. If that works, nothing is broken. If that doesn't work, Exchange is the last resort and we offer OWL. So nothing is broken either, the user just has to pay.

In case of Heidelberg in this bug here, we should prefer the config-v1.1.xml over any AutoDiscover result. Our software is TB, so why wouldn't that make sense? TB 68 should just behave like TB 52. Once an agreement about OWL's product placement is in place, IMHO, we should offer IMAP and Exchange for this server. Adding exchange to the config-v1.1.xml, like for O365, would make this easier since we wouldn't have to get the options from two different sources.

What am I missing?

we should prefer the config-v1.1.xml over any AutoDiscover result

We do. Just the steps of reproduction are wrong. The bug as filed is simply INVALID.

You MUST enter correct credentials for an Exchange server. Without the correct username, IMAP will fail, too.

Hmm, you didn't answer which "username field fix" you were referring to, see comment #10, first line.

What does "IMAP will fail" mean? The server advertises IMAP in its config-v1.1.xml, we supposedly the prefer that, so then we should say "password wrong" and not get into Exchange processing and pop the login box. I guess you're saying: We preferred config-v1.1.xml, due to the server being an Exchange server, "IMAP failed" without a valid password, so we discarded the IMAP option altogether and moved into the next option which is Exchange. That didn't work with the wrong password either, so the login box got popped.

That seems a little unfortunate and it comes back to what "IMAP failed" means. Can you not get anything out of the IMAP port to determine that IMAP in general works, just not with the wrong/missing password?

Having spent more time now on this code I think I understand where Ben's coming from. Let me try to summarize:

The code that does the detection has or is intended to have two main phases:

  • phase 1:
    • (1a) check for a Thunderbird ISP config.xml from local disk
    • (1b) check for a Thunderbird ISP config.xml from local-ish network
    • (1c) check the Thunderbird ISPDB database on thunderbird.net
    • (1d) check through DNS MX records
    • (1e) check exchange autoconfiguration detection
  • phase 2: do guessing/probing if phase1 failed.

Phases 1a-e requests are all sent in parallel, and we use the first config we found. This is a bug (too), in that in theory the config file even put on your installation can be overridden by later succeeding requests if those request are faster. Since the requests are sent in parallel, we might get more than one response, and the code is not (properly) prepared for that - addOneFinishedObserver doesn't care what it got. In particular, the case where the ISP config.xml settings are found, and phase 1e run into authentication problems, that simply disregards that there were already an ISP config found (as I saw), or that one might come in later.

In theory, IF we require the password to be set before "Continue" is pressed in the account setup, then the problem is more that we do not actually tell the user about a failed authentication attempt. AIUI, probably bug 1593122 would take care.

However, we DO NOT require the user to fill out the password before "Continue". Exchange autodiscovery (1e) will always fail when password is missing, and in fact seems pointless even trying that with empty password.

In summary: a whole lot of buggy stuff interacting, with unclear prioritization scenarios. I think the badly working parallelization of phase 1 could probably be dropped since usually this whole process will be quite fast, I'd estimate under 3 sec in general - i.e. not worth the complication. Also avoiding pointless network requests if we already found what we need.

Still we need to figure out what to prioritize: Arguably, we do need to honour the primary-for-Thunderbird created config file at the ISP. If they want us to rely on exchange autodiscover, they need to remove the config.xml file they had earlier from their server or local desktops. Should be reasonable since autodiscover code is in 60 already and any older Thunderbird's setting up accounts should not really be supported anyway. Those people on 52 or older just badly need to upgrade. When no ISP config file is found, and exchange autodiscover authentication works, in combination with fixing bug 1598861 those results would be good to make use of.

AIUI now, Ben's point is that if it's an exchange server and you do provide the right creds from get go, what the server says should be honoured and everything else disregarded. While that's perhaps not far fetched, it's just not how the system is set up at the moment (e.g. the UI doesn't enforce password, nor properly tell users about what's wrong), nor is it in line with what some administrators will have set up Thunderbird configuration files to explicitly do.

Hello Magnus,

I have to reply on my smartphone, because I'm away.

Thank you for taking the time to look more into the code. I'm glad you could understand the reasoning behind the code.

Having spent more time now on this code I think I understand where
Ben's coming
from.

  • phase 1:
  • (1a) check for a Thunderbird ISP config.xml from local disk
  • (1b) check for a Thunderbird ISP config.xml from local-ish network
  • (1c) check the Thunderbird ISPDB database on thunderbird.net
  • (1d) check through DNS MX records
  • (1e) check exchange autoconfiguration detection
  • phase 2: do guessing/probing if phase1 failed.

Yes, that is correct. You also correctly listed the order of preference. We should be using the results in this order of preference, if we get several results. (More below)

Phases 1a-e requests are all sent in parallel, and we use the first
config we
found. This is a bug (too), in that in theory the config file even put
on your
installation can be overridden by later succeeding requests if those
request
are faster. Since the requests are sent in parallel, we might get more
than
one response, and the code is not (properly) prepared for that -
addOneFinishedObserver doesn't care what it got.

It's possible that there is a bug here that I'm not aware of. if this is the case, i would definitely want to fix that.

Here is show it's supposed to work (if that's not the case, see below):
Abortable allows to cancel network calls. that implements the Cancel button.
ParallelAbortable accepts several different calls at the same time. It allows to follow them, and tracks the results of each. It also allows to cancel them.
PriorityOrderAbortable inherits from ParallelAbortable and implements the order of preference: It can fire all requests at the same time, but still use the most preferred successful result, independent from when it returns. It's specifically written to enforce this order. If we have calls a b c d, and a fails, b and d are still in flight, and c succeeds, it immediately aborts d, and it waits for b to return. if b then succeeds, we use b, otherwise c.

This is perfectly optimized and gives maximum performance. we needed that since a long time, even mozillamessaging people suggested parallelization back then. it does matter, because these calls can timeout (i.e take 10-15s each), due to firewall DROP rules on many domains. the parallelization allows us now to add HTTPS for the config file at the provider, even though nobody uses https yet. given that they are all parallelized, it doesn't add wait time.

you said it doesn't actually behave as described above. if there is a bug in there, I would very much like to know about it and i would swiftly fix it.

case where the ISP config.xml settings are found, and phase 1e run into
authentication
problems, that simply disregards that there were already an ISP config
found
(as I saw), or that one might come in later.

Did you try this with working auth? what i would expect is that we show the username field, and wait for it to be filled out, and then re-run the config, but that eventually the config.xml at the provider is used. it's possible that this is not working as expected, because this is a very special case. this is the only domain i know of that has both exchange with a custo username and the config.xml. and the admin wanted the username here. if you care a lot about this, i could look into it, even if the combination will be rare.

In theory, IF we require the password to be set before "Continue" is
pressed in
the account setup, then the problem is more that we do not actually
tell the
user about a failed authentication attempt. AIUI, probably bug 1593122
would
take care.

Yes. IIRC, the patch from aleca and me there fixes this. it says you need to enter a username. if you did and it still fails, i don't know what we would show.

However, we DO NOT require the user to fill out the password before
"Continue".
Exchange autodiscovery (1e) will always fail when password is missing,
and in
fact seems pointless even trying that with empty password.

yes. i would very much make the password mandatory. it's pointless to try to set up an IMAP account, and not enter a password.

the only reason why i did not suggest that, was that i thought you would fire back immediately, if i suggest that.

Still we need to figure out what to prioritize

Thrist you made above is the intended order of priority. if that doesn'tmatch reality, i'd be happy to fix it.

they need to remove the config.xml file
they had
earlier from their server or local desktops. Should be reasonable since
autodiscover code is in 60 already and any older Thunderbird's setting
up
accounts should not really be supported anyway. Those people on 52 or
older
just badly need to upgrade.

i concur.

i can contact the admin of medma.uni-heidelberg.de and ask him to remove it.

AIUI now, Ben's point is that if it's an exchange server and you do
provide the
right creds from get go, what the server says should be honoured and
everything
else disregarded. While that's perhaps not far fetched, it's just not
how the
system is set up at the moment

That's not exactly what i maintain, but close:
If we get a config from provider using config.xml, or ISPDB, we should use that and prefer that over autodiscover.

if we get no results from these config methods, then we should honor what the server tells us to use, using autodiscover, yes.

my position is that we should not go and try to probe an IMAP server, if the server specifically told use to use exchange protocols. the autodiscover can return imap, and the lack of that could be intentional.

it clearly wasn't intentional for aalto.fi. i understand. but i think that's misconfiguration that should be fixed there. we should not circumvent. i might be biased here, i admit, but i think that's a reasonable position at least.

Greetings,

Ben

(In reply to Ben Bucksch from comment #14)

one response, and the code is not (properly) prepared for that -
addOneFinishedObserver doesn't care what it got.

It's possible that there is a bug here that I'm not aware of. if this is the case, i would definitely want to fix that.

There is code that is supposed to be handling it. I'd have to dig in deeper to see exactly why it doesn't work. An obvious thing is that step 1e (even if the password was correct) can require additional input (the ad username, for what strange reason I don't understand since you already did authenticate using the email as id, but whatever), and this in-flight interruption requires the UI to pop up and do something. So you get into (semi-)failure mode.

case where the ISP config.xml settings are found, and phase 1e run into
authentication
problems, that simply disregards that there were already an ISP config
found
(as I saw), or that one might come in later.

Did you try this with working auth?

No this was for the uni-heidelberg.de case where I don't have a working login, so can't test that.

yes. i would very much make the password mandatory. it's pointless to try to set up an IMAP account, and not enter a password.

We could consider it, sure, but there are also reasons not to even show the field at all from start: if you know you're going to be using OAuth2, and understand the concept, you will realize the password shouldn't be entered where we fist ask it, and (being suspicious) you would not be entering it at all at the given place, but only let the OAuth2 process take care... At the very least, it's not great UI that we first ask the pwd and then seconds later you'll be asked to enter it again in another place :(
We could maybe just hide/disable password once we know the domain is one of the known OAuth2 domains...

That's not exactly what i maintain, but close:
If we get a config from provider using config.xml, or ISPDB, we should use that and prefer that over autodiscover.

This bug is initially about exactly that, and you say this bug is invalid. So in that case you lost me again.

it clearly wasn't intentional for aalto.fi. i understand. but i think that's misconfiguration that should be fixed there. we should not circumvent. i might be biased here, i admit, but i think that's a reasonable position at least.

If the server actually requires that you don't use IMAP, that can be turned off on the server. If IMAP is on, we should use it.

you say this bug is invalid

I say that simply because the reproduction steps start with entering a wrong password. That's not a valid setup. That invalid login is what caused the downhill problems with the configs.

For Exchange servers, you must enter the password, and possibly even a different username, otherwise nothing will work. This is needed a) for the config, but also b) for login later. It's actually helpful that we detect this case here and ask for the username. Otherwise we'd later try to log in with the email address as username, and it still won't work, independent where the config came from. So, this is actually important.

Alias: Heidelberg-autoconfig

Here's how it's supposed to work:

  • We detect that there's an Exchange server.
  • We try to get a config from the server itself, with login.
  • The login fails. That means the login as we have it right now does not work. That is because:
  • On-premise Exchange servers very often have a username in the form of the Windows domain login (which you use at the Windows login prompt), and not the email address that we try by default. So, we show a username field to get that information. We do that only in the specific case that AutoDiscover worked and returned "login failed".
  • Once the user entered the username and clicks Continue, we try again. If this works, we finish the config process.
  • If that still doesn't work, then there's likely something else wrong. It could be a wrong password etc., but in many cases the user entered the username in the wrong form. Some sites use usernamer in 3-4 different forms (bucksch@example.com, bbucksch, EXAMPLE\bbucksch), depending on the service.
  • Once the config process completed, we choose the best config. In this case, it would be the provider's config.xml, not the AutoDiscover result. (Even though that's expressly not what the admin wants in this case, but that's how we designed it and I think we all agree on this, including both Magnus and me.)

We ask for the username, because we know that the login information has we have it is wrong and will not work, and it's most likely a wrong username in this specific case, because Exchange typically uses Windows username and not email addresses. It encodes knowledge about how Exchange works. If we were to skip this, as Magnus proposes, then we'd try the wrong username and an IMAP login would likely also fail. So, this is actually useful the way it works.

Hmm, why do we detect that it's an Exchange server first? In Magnus "phase 1" we get an config.xml first, which this particular server is providing from http://autoconfig.medma.uni-heidelberg.de/mail/config-v1.1.xml. People would then fix the username MANUELL_BEARBEITEN: IHRE_MEDMA_ID@medma.ad.uni-heidelberg.de and provide the correct password. Or if they don't provide a password, they should be told: Wrong/missing password. I'm not sure why we go down the Exchange path at all given the config.xml.

Sure, if they added type exchange to that file, we'd have to reconsider the flow. I although I still think we should offer IMAP/SMTP and maybe, when agreed, Exchange as another option.

Let's remember what the bug is about: With current settings, the config.xml is ignored.

I've just tried this again on TB 60.9 and TB 68 and both actually show the config.xml result. So it this only a problem on beta/trunk since bug 1518155? I'm confused. Sigh, I've just tried trunk again and now I get the suggestion from config.xml. Before I'm sure I got the login box, but now I can't reproduce it.

config-v1.1.xml. People would then fix the username MANUELL_BEARBEITEN: IHRE_MEDMA_ID@medma.ad.uni-heidelberg.de

That's precisely what the admin does not want. That's the old hack. It's a band-aid, because older TB could not do better. The admin who created this file contacted me specifically asking how to avoid that. The discussion with him lead to the solution in bug 1518155.

Many on-premise Exchange servers have that same problem, because it seems the standard configuration for Exchange to use the Windows login, not the email address.

why do we detect that it's an Exchange server first?

As described in comment 14, we detect everything at the same time, then pick the best result.
Here, we detect that there is an Exchange server that needs a login and we have the wrong credentials, so we ask the user to enter the correct Windows login username. That also helps IMAP, because it would need the same username.

Hmm, well, what the system should reasonably do based on the provided configuration and what the Heidelberg admins wants may be two different things.

Setting up abc@medma.uni-heidelberg.de with an invalid or no password immediately gets me the result from config.xml with the message "Configuration found at email provider". That's with mailnews.auto_config.fetchFromExchange.enabled set to true.

TB 68 seems to think a little longer, but in the end returns the same result. On trunk I've just gotten the login box, but repeating the exercise, I get the suggestion from config.xml. I'll attach a screen shot just to prove it. Please try it yourself.

So there is certainly a bug in that the result is not deterministic.

While we haven't determined what the system behaviour should be, like in TB 52, the thing that the Heidelberg guy apparently doesn't want, or something else, maybe you can let us know what he really wants so we can consider it. Maybe he wants what happens in a fraction of the cases that the login box is popped up. And what should happen after that? An IMAP connection? An Owl offer? In this context it would be good to have some credentials at Heidelberg med school so we can actually try it for real.

Summarising, it would be helpful to have direct answers to the following question:

  • With the config.xml in place as it is, what is the expected outcome for account setup with and without and with invalid password
    a) As Thunderbird would define it
    b) As the Heidelberg admin or Ben would define it.
  • Should this be an IMAP setup or ultimately an Exchange setup with Owl?
  • Does the system currently match behaviour a) or b).
  • What could be other ways to achieve what the Heidelberg admin wants.
Attached image Heidelberg.png —

I've tried a few times and cannot reliably reproduce the bug. In older versions of TB (60 + 68) and mostly on trunk, I get what the attached picture shows.

Attached image Heidelberg-2.png —

Rarely I get this.

Attached image Heidelberg-3.png —

And then it's back to this. I'd call that buggy.

Regressed by: 1518155

So there is certainly a bug in that the result is not deterministic.
I'd call that buggy.

It is. That's what bug 1593122 fixes.
But Magnus has blocked the patch. :-(

OK, I'll try with the patch over there. Meanwhile please answer my questions for comment #20.

Well, using the patch from bug 1593122, I now get the behaviour from attachment 9112357 [details] all the time, that is, the (felt) ten times I tried.

So is that the desired outcome? You said that the Heidelberg admin didn't want that behaviour.

Totally confused :-(

Magnus, can you still reproduce the issue on trunk with or without the patch from bug 1593122?

P.S.: The summary of bug 1593122 is: [autoconfig] Status area goes blank while running guess config. How would I know that is fixes honouring config.xml? That's not related to guessing at all. I'm sure I'm missing something again ;-)

As filed, I can reproduce (with exchange lookup disabled).

Re not deterministic, well yes I agree, it probably depends on what comes in in what order.
The problem with the exchange case is that you can get into the wrong-password case, and that's not a success/failure answer, but a "don't know yet".

Also to note, it's also buggy when you do get into the asking-usernames state. If you enter it, but get it wrong, you will get stranded: no feedback, just disabled buttons.

As filed, I can reproduce (with exchange lookup disabled).

You all want to confuse me, right? As filed, see comment #0:

If you set mailnews.auto_config.fetchFromExchange.enabled false, it will find the ISP defined configuration for IMAP and SMTP.

With the config.xml in place as it is, what is the expected outcome for account setup with and without and with invalid password

I think I answered that in comment 17.
With invalid login or no login: Open username prompt. Wait until user enters a valid login. We need to ensure that we have the right credentials. Without that, nothing will work. That includes IMAP.
With valid login: If config.xml exists, use that config, otherwise use AutoDiscover result.

It is not desired by admin that the user has to manually edit the config. So, he can remove the config.xml and it will work as expected.

The expected behavior needs a valid login. You don't have one, so you cannot get the expected behavior.

The confusion comes from the basic setup:

  1. Test with valid login. You don't have one, you cannot test this. Neither a wrong password nor an empty password are valid situations. For Exchange servers, you MUST enter a valid login at the config stage, otherwise nothing works. I hate it even more than you do, but that's how Microsoft designed it.
  2. Apply fix for bug 1593122, which should fix the non-deterministic part.

Once we have these 2 things, only then can we test whether there are bugs left. Without these 2 things, any discussion here is going to go in circles.

We're going around in circles here :-(

With invalid login or no login: Open username prompt. Wait until user enters a valid login. We need to ensure that we have the right credentials.

Please note my comment #26:

Well, using the patch from bug 1593122, I now get the behaviour from attachment 9112357 [details] all the time, that is, the (felt) ten times I tried.

So if you expect the login prompt that Magnus is complaining about, it's not there. Please test it yourself. I didn't do the screenshots in Photoshop, they are real.

Please test it yourself.

I did, even with a wrong or no password, and I do get the login prompt consistently. See screenshot.

I am using a fresh profile, no pref changes, trunk (Nov 20 and included fix for bug 1598041) on Linux, plus the bug fixes from:

Attachment #9112423 - Attachment is patch: false
Attachment #9112423 - Attachment mime type: text/plain → image/png
Attachment #9112423 - Attachment description: tb-autoconfig-username-error.png → Heidelberg, wrong password, with bugfix 1593122, shows username prompt as expected
Attachment #9112423 - Attachment description: Heidelberg, wrong password, with bugfix 1593122, shows username prompt as expected → Heidelberg, wrong password, with bugfix 1593122, shows prompt as expected

The mentioned admin replied back to me.

He said he wholeheartedly agrees with comment 19:

That's precisely what the admin does not want. That's the old hack. It's a band-aid, because older TB could not do better.
The admin who created this file contacted me specifically asking how to avoid that. The discussion with him lead to the solution in bug 1518155.
Many on-premise Exchange servers have that same problem, because it seems the standard configuration for Exchange to use the
Windows login, not the email address.

So, he disabled the config.xml now.

He added:
The item "MANUELL_BEARBEITEN: IHRE_MEDMA_ID@medma.ad.uni-heidelberg.de" was a workaround, but not a solution, because many users do not exactly understand what they are supposed to do at this point. First, only one language can be used, and second, many users do not understand YOUR_MEDMA_ID. :-( In the log, I can see that some users need 2 or 3 attempts to get the correct setting. Many users have no technical affinity and are completely overwhelmed and contact IT support. Mail out of the box is something else.

I guess we lost the easy test case then.

Anyway, I suggest for this bug we just focus on ensuring that the ordering from comment 13 is respected. As a lower order check, the autodiscover check should not be able to interrupt the process and discard results from the higher order checks. It should wait until all higher order checks have been completed and failed until it would cause an interruption.

we just focus on ensuring that the ordering from comment 13 is respected.

Agreed.

As a lower order check, the autodiscover check should not be able to interrupt the process and discard results from the higher order checks.

No, that doesn't automatically follow. I've explained:

If AutoDiscover returns HTTP 401, we know for a fact that the credentials we have are wrong. The user must first enter the correct credentials, before we proceed. It makes no sense to continue with known-wrong credentials. They will fail for any kind of config, even IMAP from config.xml.

This is a speciality of the on-premise Exchange servers. We are lucky to be able to detect the specific situation and react correctly. Your suggestion would break that.

If you skip that, and fail at the IMAP password verification, you give the user no hint that the username might be wrong, and you run into exactly the situation explained in comment 33.

You really can't have it both ways. If steps 1a-d gives a result that must be used, in specified order. Possible results from 1e are not allowed to drop those results. Results from those can also easily contain the username already, and I'd expect especially for local disk case (1a) that would usually be set if needed. A 401 for step 1e can mean that you need to enter another username. But the config.xml result from previous steps can already have set you up to use the right, different, username.

the config.xml result from previous steps can already have set you up to use the right, different, username.

This is not a real world case. For this case to be applicable, it would need to be
a) an Exchange server
b) which has a config.xml (that alone is really exceptional, because Exchange uses AutoDiscover), and
c) uses the email address parameter to look up in the user database and specifically insert a concrete custom username (not the email address or its local part, otherwise we wouldn't need the username field), and
d) ideally uses HTTPS (see bug 1594372).
All of the above need to be true for the case you mentioned to take effect. Even if there is such a server, that would be extremely rare. I've never seen one.

Also, if there is a deployment that uses config.xml files on disk, again with concrete custom username and with an Exchange server, I would really like to know about it.

I am trying to fix the standard configuration of on-premise Exchange servers.

Since it's a way to get set up automatically, I'm sure some people would be using it. Obviously there is no data on such a thing.

I am trying to fix the standard configuration of on-premise Exchange servers.

That's not what this bug is about. In a average case, this bug wouldn't be encountered.

That's not what this bug is about. In a average case, this bug wouldn't be encountered.

a) Then let's not regress the average case, with whatever we do here.

b) What is this bug about? Can you please give an actual bug with realistic steps for reproduction?
(The original bug as filed here is no longer existent, because the server setup has meanwhile been fixed.)

If we enforce the order like the code (supposedly) does there is no problem for the average case, since that would not find any config.xml. Since you claim such cases don't exist, you can't claim at the same time we should not enforce such ordering because that would then regress the average case.

You have cited this bug as reason to block bug 1593122, and that other bug might be encountered on an normal case. So, can we unblock that other bug now, please? So that the normal case is fixed first, before we talk about extreme outliers?

I don't know any actual bug here, presuming bug 1593122 is fixed. Please apply the current patch for bug 1593122 before testing here.
From what I saw, the code works as intended.

So, please, for this bug here: Can you please give an actual bug with realistic steps for reproduction?

Agreed we don't have to do this before bug 1593122.
But as such, we should fix it. That we found this case in the first place shows it's a realistic case (even if it turned out they no longer wanted it). That we have something as non-deterministic as this also makes is basically impossible to have automated tests for it.

This is the same as the patch in bug 1593122, still with the username message, but with one important change:

Based on Magnus feedback here, I moved the username field check to after all config methods fail. So, if the provider config.xml or ISPDB comes back with a result, we will not show the username field anymore. This implements exactly what Magnus asked for.

But that's still WRONG, because we know that the config as given does not work. Even if the ISPDB or config.xml with automatic placeholders comes back, the config will still not work. We will show "wrong password etc.", but in reality, the username is in the wrong format. The user will try other passwords, because we do not even show a username field! This is not only a UX disaster, but also a security disaster, because the user will give multiple different passwords.

I don't know a single server in the world that puts in the concrete username here, is an Exchange server, uses HTTPS etc.. If you know any, please show me. For all cases I know, this will do the wrong thing.

Attachment #9113010 - Flags: feedback-
Version: unspecified → 66
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: