1590636 - The manual config for setting up a new account has moved !!

Status: RESOLVED FIXED

(Thunderbird :: Account Manager, defect)

Description

[musiquegraeme]

+++ This bug was initially created as a clone of Bug #1590397 +++

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0

Steps to reproduce:

I opened the dialogue for a new account
I entered the name, email and deselected the keep password
I clicked Continue

Actual results:

I looked for the Manual config button but it wasn't there...!

Expected results:

There should have been a manual config button to allow me to skip sending my details to several different places but go straight to manual set up.

The manual config button comes up if you enter a known ISP after the data you have entered has been sent to the ISP.
What I want is a manual config button for the case where my ISP is unknown so that my data isn't sent to all the known ISPs, a button in the place it used to be.
I cloned the bug I started because it was closed and I wasn't sure if further comments to the old bug could cause it to be reopened.

Comment 1

[alta88]

This is absolutely bad privacy violating behavior. There must be a clear user choice UI to opt out of autoconfig, which does indeed spray email and name to several places. And to those which aren't even secure http (there is an ignored bug on that). It takes setting 4 prefs false and clearing 2 pref urls manually to be totally free of this current unwanted forced autoconfig behavior.

I understand there is a privacy policy being formulated. It needs to remove decisions on privacy and opt in/out from the whims of coders and commercial third party contributors. Where to place the opt in/out may be UX; whether to have it or not is for sure not a UX decision. So far, the mail module owner has failed to demonstrate privacy considerations both passively (Bug 971347) and actively (the backed out Bug 1532388).

This needs to be fixed, following a Firefox-like privacy first policy.

Comment 2

[Alessandro Castellani [:aleca]]

Thanks for bringing this up, I wasn't aware of the important privacy implications and removing that option entirely is not a great solution.

Nonetheless, the previous method, currently in 60-68, is not a great solution either. Prompting that button only after the user has clicked "Continue", forcing the user to quickly move the mouse to hit the Manual Config button before any info is sent to any ISP.

We should definitely improve this, and implement an option to allow manual configuration right away might be needed.
I'm removing Kai's needinfo as I spoke to him directly regarding this and he agrees.

I'm pulling Manus into this to get his opinion before starting to work on a patch.

Comment 3

[Magnus Melin [:mkmelin]]

The account setup doesn't send your details out (only the domain of the address), so the question comes down to if you think "a user on the network has a example.org address" is a privacy concern. Foil hat on perhaps you can argue that, but realistically speaking that just isn't going to matter the slightest to anyone.

That said, I wouldn't mind if there is way to skip into the manual setup. Maybe there should be a drop down on top with automagically and manually as options.

Comment 4

[alta88]

The account setup doesn't send your details out (only the domain of the address),

That is incorrect.

In fetchConfigFromISP() the default is to send the email; further, the default is to not use |requireSecureAuth|, which means email is also sent to an http address for the domain, since the default is to not require secure https.

In fetchConfigFromExchange() email, username, password are sent; only the email is sent if the url is insecure http.

Foil hat on

Perhaps this dismissive and pejorative language comes from not understanding what the code is doing.

just isn't going to matter the slightest to anyone.

Wrong. But anything more belongs in another venue.

Comment 5

[Magnus Melin [:mkmelin]]

(In reply to alta88 from comment #4)

In fetchConfigFromISP() the default is to send the email; further, the default is to not use |requireSecureAuth|, which means email is also sent to an http address for the domain, since the default is to not require secure https.

You're right, we probably shouldn't send email address when it's not over https.

just isn't going to matter the slightest to anyone.

Wrong. But anything more belongs in another venue.

I am genuinely interested. Why would it matter? It's not like the network operator can't see that you're connecting to (say) gmail later on, so why would setting the account up be a special case?

Comment 6

[Alessandro Castellani [:aleca]]

Attachment: 1590636-manual-config.patch

Here's a patch that might make people happy...hopefully.

With this update, the "Manual Config" button will appear alongside the "Continue" once a valid email address has been written.
Also, for both spacing and workflow reasons, the "Get a new email address.." link gets hidden in this specific scenario.

Now, this brings up a question.
Might this be confusing for regular users? Like, I'm setting up my email and suddenly another button appears. I'm worried that with this change, clicking "Continue" might not be the obvious choice for non-technical users.

Comment 7

[alta88]

Have you confirmed that using the Manual Config button no longer invokes any code in autoconfig? It was intentionally coded to force network requests even with the old manual config button visible (iirc). Also, manual config could surely use better defaults; ie the most secure defaults should be chosen first, which any user doing manual config can and will know how to override. But that may be for another bug.

If you feel "Continue" isn't clear enough, it can easily be "Continue with Automatic Config" or such to indicate a choice different from manual config. But given that the button used to be visible even for 'regular users', I doubt confusion here is a valid concern.

This opt out of autoconfig is even more important given the gross ethical violation in Bug 1592258, which I alluded to in comment 1, and bizarre disinterest in strict network privacy/security as demonstrated in the patches I already referenced. I also don't understand why the new resident security expert isn't being involved and setting stronger direction here. Frankly, I don't trust this project to do the right thing.

Comment 8

[Wayne Mery (:wsmwk)]

I don't have power nor review authority here. But it seems the relevant people are engaged who can properly address privacy.

Comment 9

[Magnus Melin [:mkmelin]]

I recently had to set up an account on a Windows machine. They have the "Get a new email address" link right aligned under the actual email address. I kind of like that, as it's contextual and you don't end up with too many buttons/links at the end.

How about moving the link there, and keeping the manual config button always available? Similar to this: https://www.windowscentral.com/sites/wpcentral.com/files/styles/w830/public/field/image/2017/09/create-new-ms-account-settings.jpg?itok=6LxdFWMt

Re network traffic: at the moment, some OPTIONS requests are sent, and resent many times (pointless, filed bug 1594366). No data is sent before you press continue.

Comment 10

[Magnus Melin [:mkmelin]]

(In reply to alta88 from comment #7)
I filed bug 1594372. If there's more, please file bugs. I still don't see a case where exposing the email domain would be much of a problem, so you'd have to spell out what the perceived problems would be with that.

Comment 11

[alta88]

(In reply to Wayne Mery (:wsmwk) from comment #8)

I don't have power nor review authority here. But it seems the relevant people are engaged who can properly address privacy.

I don't actually believe that, meaning your use of 'properly'.

According to a minutes post published on tb-planning, you are developing a privacy policy, correct? That policy will override and guide any decisions regarding actual code and design. So let's not pass the buck and rather publish the policy, as in comment 1. User opt in will be a part of that, and that is what this bug is all about.

Comment 12

[Alessandro Castellani [:aleca]]

(In reply to Magnus Melin [:mkmelin] from comment #9)

How about moving the link there, and keeping the manual config button always available? Similar to this: https://www.windowscentral.com/sites/wpcentral.com/files/styles/w830/public/field/image/2017/09/create-new-ms-account-settings.jpg?itok=6LxdFWMt

Sounds good, I'll update the patch and test it out to see if we risk introducing any visual quirks while cycling through the various panels.

Comment 13

[Alessandro Castellani [:aleca]]

Attachment: 1590636-manual-config.patch

I moved the link to get a new email address underneath the email field, and the manual config button is now aligned to the left like it was in the original wizard.
The button remains disabled until a valid email address as been written.

Comment 14

[alta88]

(In reply to Magnus Melin [:mkmelin] from comment #10)

(In reply to alta88 from comment #7)
I filed bug 1594372. If there's more, please file bugs. I still don't see a case where exposing the email domain would be much of a problem, so you'd have to spell out what the perceived problems would be with that.

If a user wants to use autoconfig, obviously the domain must be sent to fulfill the lookup function. However, http must be deprecated everywhere for lookup, and you've made a followup. If the user wants to verify the account, obviously the email/pwd must be sent to the imap/pop/smp domains.

What should happen in this bug is:

1. Ensure an easy path to mail account creation guaranteeing zero network traffic, ie same behavior in online as in offline mode. (Offline is/was sometimes not respected re network traffic).
2. Make sure that if manual config is chosen, the best (most secure, not most likely) defaults are pre-selected in the following screen - for example, default IMAP, port 143 or 993, auth Encrypted Password. That screen should also be ergonomic for manual config, I'm sure aleca knows how to do that.

Regarding the 2 bugs filed, I'll followup there.

Comment 15

[Magnus Melin [:mkmelin]]

Comment on attachment 9106988 [details] [diff] [review]
1590636-manual-config.patch

Review of attachment 9106988 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good, thx! I don't think you need to keep the manual config button disabled until there is a valid email though

(Doesn't change anything wrt network traffic, but I think that's a topic for other bugs.)

Comment 16

[Alessandro Castellani [:aleca]]

We can definitely change that in a follow up bug, but for now it's better to leave it as is as currently there are conditions in place that switch panel based on what's written in the email field.
A bit of rework will be necessary to leave the manual config enabled by default.

Doing a try-run to see if I introduced any bug: https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=d1dc6be452e8df15aac6a742c491d1696be51192

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/9ba029277a3d
Add option to directly skip to manual config on account setup. r=mkmelin

Comment 18

[Jorg K (CEST = GMT+2)]

TB 71 beta 3:
https://hg.mozilla.org/releases/comm-beta/rev/d66670635cca535201a53d1e7037dd9bd57fa331

Comment 19

[Matt]

(In reply to alta88 from comment #14)

2. Make sure that if manual config is chosen, the best (most secure, not most likely) defaults are pre-selected in the following screen - for example, default IMAP, port 143 or 993, auth Encrypted Password. That screen should also be ergonomic for manual config, I'm sure aleca knows how to do that.

[edits per Matt] My observation from spending time in support is most users don't really care about security. They want something that works, first time, no thinking required. If asked they will tell you they value security, but they do not. They will allow any certificate they can to get their mail flowing. That is if cancelling the dialog does not work. They might, and only might, make a support request to check if they did the right thing after the event. Yet others complain about the annoying pop up and the fact that Thunderbird refuses to get mail after they cancel it. They tell me the product has a bug.

For years we had an auto config process and they ended with non working configurations because server responded they offered SSL, except the certificates were for corporate accounts/domains not user domains so it did not "just work". These configurations even made it into the ISPDB. Then we have large national even ISPs (eir.com for example) that really are still in about 1995 with their email.

What you think is most secure is not what the user wants, a product that works is what the user wants. Without a requirement for them to tackle certificates encryption etc. There are enough users that decide for themselves that encrypted passwords is the setting they need and then make a support request because they made the settings more secure in their opinion and now it does not work. We do not need decisions from the developer community that will see the user create a non working account because we default to most secure settings for anything.

Providers publish account settings and we should use them where possible. That they are not the "most secure possible" notwithstanding. I will resist to the best of my ability any change where we try and configure the users account in a "more secure" manner that the provider recommends.

I would also suggest that clicking the manual config button should present a fully blank dialog, no defaults whatsoever. The user is making the decision that they will use settings they already have so we should not be doing anything to interfere with that.

I also think that despite the discussion here, those running a mail server in their lounge room are finding the no use the inability to use self signed certificates a pain, making them use SSL and the refusing to allow them to use self self signed certificate is really just one step to far. Those configuring a server know why they are using it and hopefully what security they desire on it. Thunderbird forcing them to use SSL/TLS is the height of arrogance really. We can beg, ask nicely and suggest. But that is where it should end really. Some users and sites simply do not use SSL/TLS.

Comment 20

[Magnus Melin [:mkmelin]]

There is a weigh-off: We need to make it as secure as possible while as usable as possible.

Comment 21

[Ben Bucksch (:BenB)]

What Magnus said. Exactly that has been my policy as well all along.

Comment 22

[Matt]

(In reply to Magnus Melin [:mkmelin] from comment #20)

There is a weigh-off: We need to make it as secure as possible while as usable as possible.

Your choice. But we look like a bunch of morons when we use settings that do not work, and our settings do not agree with those suggested by the provider. The simple truth is the provider publishes setting that they say will work. They manage the servers so have the inside running on determining what will work.

Someone who has no account with them and therefore no way to actually test any other settings is not in a position to determine what is functional and more secure.

Comment 23

[Magnus Melin [:mkmelin]]

Comment on attachment 9106988 [details] [diff] [review]
1590636-manual-config.patch

We should consider uplifting this so people who want manual config have an easier time.

Comment 24

[Jorg K (CEST = GMT+2)]

Comment on attachment 9106988 [details] [diff] [review]
1590636-manual-config.patch

If you want it, you should rebase it to ESR 68.

Comment 25

[Gunter Ohrner]

We should consider uplifting this so people who want manual config have an easier time.

Backporting this fix would be important as long as the Thunderbird 68 autodiscover assistant has bugs which in some unlucky constellations makes configuring mail accounts simply impossible without any tricks. (See bug 1600493.)

It could probably always happen that the autodiscover assistant is buggy, not only in the specific case described in bug 1600493, and it's really unfortunate to get completely stuck in this case and to have no way to enter manual configuration without workarounds like disabling all network connections before invoking the assistant or similar...

Comment 26

[Jorg K (CEST = GMT+2)]

Patch needs rebase for ESR 68.

Comment 27

[Magnus Melin [:mkmelin]]

Attachment: bug_1590636_manual_button_68.patch

Reworked to 68.

Comment 28

[Magnus Melin [:mkmelin]]

Attachment: acccountsetup68.png

Far from as pretty as trunk

Comment 29

[Ben Bucksch (:BenB)]

Yeah, that provisioner link was for the trunk design (and it's suboptimal even there), and doesn't work well with the TB 68 design. I've leave that change out.

Comment 30

[Alessandro Castellani [:aleca]]

Comment on attachment 9114520 [details] [diff] [review]
bug_1590636_manual_button_68.patch

Review of attachment 9114520 [details] [diff] [review]:
-----------------------------------------------------------------

Overall this looks OK, with the few changes I suggested the improve the alignment.
We could also move the button in between the 2 `<label>` elements, so it would be aligned with the text fields.
Since it gets the focus while navigating with the TAB key, and the new row creates an empty gap, it makes sense to have the link underneath the email field. (screenshot coming).

With these few changes, r+

P.S.: can you update the commit message and put r=aleca so  it matches my bugzilla nickname.

::: mail/components/accountcreation/content/emailWizard.xul
@@ +157,5 @@
>                     class="warningicon"/>
>              <description id="emailerror" class="errordescription" hidden="true"/>
>            </hbox>
>          </row>
> +        <row pack="end">

This should be `pack="start"` otherwise the link will shift to the right if the user resizes the dialog.

@@ +160,5 @@
>          </row>
> +        <row pack="end">
> +          <label value=""/>
> +          <label value=""/>
> +          <button id="provisioner_button"

`align="left"` to the button so the text is properly aligned with the helper text above.

Comment 31

[Alessandro Castellani [:aleca]]

Attachment: Screenshot from 2019-12-09 10-46-48.png

Comment 32

[Ben Bucksch (:BenB)]

Personally, I think this is way too prominent for the provisioner link. We actually wanted to remove it entirely. We do not want the focus on it.

Actual data shows that 99.9% of our users of the dialog set up an existing account and only 0.1% (one in a thousand) create a new account.

And that number was even when the provisioner dialog was a lot more prominent, it was the first dialog after TB install, and the user had to go through extra hoops to set up an existing account. Monetarily, it's also insignificant. This is getting way too many eye balls. Pretty much nobody actually wanted it, even when we forced it on users.

Comment 33

[Magnus Melin [:mkmelin]]

Attachment: bug_1590636_manual_button_68.patch

Updated to review comments. I also adjusted the id up to the row so that the whole row would hide better.

Comment 34

[Geoff Lankow (:darktrojan)]

68.4:
https://hg.mozilla.org/releases/comm-esr68/rev/b6efcf577283768cee435542da4fbe56b1e15aef

This patch has the wrong version bug number. I've just noticed. :-/

Comment 35

[Geoff Lankow (:darktrojan)]

Backed out and landed with the right bug number:
https://hg.mozilla.org/releases/comm-esr68/rev/8496ed2b85a85849237bbef5f90556088352e437

Comment 36

[Jorg K (CEST = GMT+2)]

(In reply to Geoff Lankow (:darktrojan) from comment #34)

This patch has the wrong version number. I've just noticed. :-/

You mean "bug number". The C-C and C-B patches were alright, sadly not the rebased 68 version. They also reinvented the commit message instead of just using the C-C/C-B commit message.