Closed Bug 1530123 Opened 6 years ago Closed 9 months ago

Thunderbird times out after connecting to VPN / switching network

Categories

(MailNews Core :: Networking: IMAP, defect)

Product:
MailNews Core
Component:
Networking: IMAP
Platform:
Unspecified
macOS
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: helmut, Unassigned)

References

(Blocks 2 open bugs)

Details

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:65.0) Gecko/20100101 Firefox/65.0

Steps to reproduce:

  1. Start TB
  2. read your emails (IMAP account)
  3. connect to a VPN
  4. try to read your emails (also click on several different folders)
  5. disconnect from the VPN
  6. try to read your emails (also click on several different folders)

Actual results:

Sometimes the connection times out and I can't read my email. The spinning thingy appears and nothing happens.

Expected results:

I should be able to read my emails.

I think this has something to do with the backend IMAP connections. It seems they are not refreshed when the network config changes and thus it comes to a stale connection that can result in a hang.

It is also very strange that this bug tracker does not have a component 'network', 'IMAP', 'connections', ....

Component: Untriaged → Networking: IMAP
Product: Thunderbird → MailNews Core

Thanks for changing the component.

It seems that component is only available in the list, after the bug's been created. Maybe someone should update the guided bug report submission UI...

(In reply to Helmut K. C. Tessarek from comment #0)

...
3. connect to a VPN
4. try to read your emails (also click on several different folders)
5. disconnect from the VPN
6. try to read your emails (also click on several different folders)
...
Sometimes the connection times out and I can't read my email. The spinning thingy appears and nothing happens.

So it doesn't always happen, correct?

And is your mail server IP address within the VPN address space, OR the VPN is NOT doing split tunnel?

Perhaps this will be helped by Bug 1535969 - Implement TCP keepalive for IMAP protocol, to detect if server dropped connection or machine suspended longer than server (application layer) IMAP timeout - curently in nightly builds

Flags: needinfo?(helmut)

So it doesn't always happen, correct?

Yes, it always happens when you switch networks (VPN on/off).
Start TB while VPN is off. Then start VPN. Hang. Kill TB.
Start TB while VPN io on. Then stop VPN. Hang. Kill TB.

And is your mail server IP address within the VPN address space,

No, it is not.

OR the VPN is NOT doing split tunnel?

No, not DNS split tunneling is done. Reason being: ALL traffic goes through the VPN servers (except local traffic of course).

Flags: needinfo?(helmut)

Reporter, when you say it "hangs", how long did you actually wait for the spinner to stop? I'm not sure what happens when you turn off and on a vpn, but if tb doesn't get a FIN or RST it might think the connection is still there. In this case it will take about 100s for tb to know the connection is dead and close it itself and open a new one.
Of course if you wait much more than 100s and it still keeps spinning, I guess that's not the problem.

Also, you mention clicking on other folders. When you click on a folder that you haven't visited since starting tb, a new connection is established so it should not be confused as to the possibly changed vpn or non-vpn state. This is assuming you have more than 1 "cached" connection set in you advanced server page (default is 5 conns). But if all 5 of your connections are established, clicking on a new folder won't help.

A possible workaround is to go offline and then back online. This should close and restart all connection using the changed routes without restarting tb.

Reporter, when you say it "hangs", how long did you actually wait for the spinner to stop?

I think the longest was about 5 minutes. But I will test it again, if something happens after 100s.

When you click on a folder that you haven't visited

I will test all this and a few other scenarios tomorrow.

A possible workaround is to go offline and then back online.

Thanks for the info. I will test this also tomorrow.

Ok, I ran a few tests. 5 cached connections are set in the advanced config editor page.

When switching to a different folder [1] the spinning stops at around 100s. If I click on a folder that I haven't clicked before and not all 5 connections are cached, the folder switch is instantly (without any spinning or waiting).

However, when I switch back to Inbox from any folder (after a successful folder switch or after the 100s timeout), the spinning never stops [2]. It seems that the Inbox is a special case.

The "go offline" workaround only works during the 100s [1] for folders, but it does not work when it's spinning [2] on Inbox. In that case nothing helps but to restart TB.

So, being able to change the 100s timeout to 2s would help a lot. Seriously, who needs a 100s timeout? The times where bits were transferred by donkeys (acoustic couplers) are long over.
Changing the timeout to 2s won't help with the Inbox issue though.

Helmut, you can adjust the 100s timeout with the config editor item mailnews.tcptimeout which default to 100s.

Not sure why offline transition doesn't free up inbox or why it is "special" and hangs forever.

I don't currently use a vpn so it is kind of hard for me to test this. Do you know of a reasonably safe and free vpn I can try this with? A few years ago I experimented with using a free vpn but just for a short time and don't remember how to set it up on linux, so I will have to re-learn...

I did find some old bugs that mention a problem similar to what you observer by searching on google:site:bugzilla.mozilla.org thunderbird vpn. But no fix mentioned other than offline/online transition.

you can adjust the 100s timeout with the config editor item mailnews.tcptimeout which default to 100s.

Thanks a lot for that info.

Do you know of a reasonably safe and free vpn I can try this with?

No, I don't know any free VPN services. However, afaik nordvpn and ipredator offer free trials. I am mentioning these 2 services because they are the only ones I trust. (We can discuss this in more detail offline. I don't want to sounds like a sales person or get too far off the topic here. You can always drop me an email.)

It's also not very complicated to setup an OpenVPN server yourself.

I did find some old bugs

Right, I think it's about time this gets fixed. While I understand that there were more important thing to tackle in the past, the further development is pretty much dead (for the last 3 years - and I'm talking about new features, because the Gecko engine is kept up-to-date and so on). I'm not saying that's a problem, since TB usually works great. But the product could get some love for internals like this issue.

Hmm, if I understand it correctly, the non-split-tunnel VPN kicks in and TB loses it's connections that weren't going through the VPN. Has anyone checked whether bug 1535969 included in the current release 68.1.2 has made a difference?

I think this problem my be related: My employer's IMAP server uses one IP address for the external world (call this VPN off) and a different IP address (but same server name) for the internal network (behind the firewall) (call this VPN on). I usually start Thunderbird with the VPN off, and (after logging in to the IMAP server) can read and reply to email on that server. If I then start the VPN, I can continue to read email, but if I click Reply, compose a reply, and click Send, it may or may not succeed. If it fails, clicking Send a second time usually works.

However, if I click Reply when VPN is off, but turn VPN on before clicking Send, it might send, but it won't save a copy in the Sent folder. Sometimes Retry on the saving to Sent works, but (especially with 68.7.0) often it doesn't. Even the trick of toggling offline/online and then clicking Get Messages doesn't solve the problem, although that seemed to work with 68.6.0 and earlier.

The workaround seems to be, don't switch VPN from on to off or vice versa if a message is unsent, and do toggle offline/online and click Get Messages after switching VPN from on to off or vice versa.

If you think this is totally unrelated, I'll be happy to put this into a separate bug report.

(The reason I don't leave the VPN on all the time is that my employer's internal network blocks the Gmail IMAP server, so to access my personal email I have to turn the VPN off.)

Helmut, does this still produce for you?

See Also: → 1638595
Flags: needinfo?(helmut)

Yes, this problem still exists.

However, as a workaround one can set mailnews.tcptimeout to 5 and thus the client is not blocked for several minutes. But this workaround is not a solution.

The mail client should monitor the network connection and in case there's a change reset the current connections. But tbh I gave up on a proper solution already. TB development is rather stagnant and the only thing they care about is messing with addons. But why wasn't the master password hash fixed in FF abd TB? That's way more important for security, yet the hash used is easily cracked with rainbow tables.

Flags: needinfo?(helmut)
Severity: normal → S3

I can reproduce this bug when using Mozilla VPN and Thunderbird 102.5.1 on MacOS. Whenever I connect to a VPN server Thunderbird doesn't seem to be able to communicate with the remote mail servers anymore. I have to restart the application or wait that specific amount of time to get it temporarily fixed until I disconnect VPN.

The Failure that I can see is that Thunderbird cannot connect to the server because the remote connection is refused. There is no specific error visible in the DevTools console.

Status: UNCONFIRMED → NEW
Ever confirmed: true

This is a kinda annoying behavior for VPN users. I always miss email notifications because Thunderbird is no longer able to automatically fetch new emails. I'm happy to provide more details if needed. Geoff, is that a problem which may need to be prioritized?

Flags: needinfo?(geoff)

This isn't actually happening to all VPN users, I can't reproduce it at all on Windows using Mozilla VPN. When I turn the VPN on and off Thunderbird does have to create a new connection when clicking on a folder, which is a bit slower, but it doesn't time out or fail.

So there's more to this than just "VPNs". Might be MacOS specific, but I'm just guessing. I'm not sure we'll be working on this explicitly any time soon, but it may end up "fixed by accident" due to the IMAP JS work or some other IMAP protocol work.

Flags: needinfo?(geoff)

Additionally: I tested this on my MacBook and it does seem like Thunderbird struggles right after switching VPNs, but connecting to folders still works after a few tries. It wouldn't surprise me if the autosync gets stuck or fails though, as we have a number of autosync issues.

OS: Unspecified → macOS

@andrei switching to another folder only works if that folder is not associated with an open (and thus stale) connection.

Currently the only workaround is to set a low timeout value. What I am trying to explain for the past 4 years is that the client should invalidate (and re-establish) the connections as soon as the network status changes.

So either this can be done, or this bug will be open until the end of time or the end of Thunderbird. Whatever comes first.

But I have given up on any real bugs being fixed by Mozilla. I believe the fact that the "master password" uses an insecure unsalted md5 hash has been known for over 10 years, yet a simple change of the hash algorithm hasn't been done. The only thing the so-called SW engineers care about is making add-on deveolpment more and more bloated and ineffective similar to Apple's eco-system where you have to rewrite your application code almost every year due to deprecated APIs that never get a proper replacement.

(In reply to Helmut K. C. Tessarek from comment #17)

Currently the only workaround is to set a low timeout value. What I am trying to explain for the past 4 years is that the client should invalidate (and re-establish) the connections as soon as the network status changes.

So either this can be done, or this bug will be open until the end of time or the end of Thunderbird. Whatever comes first.

This is the first time I've seen this bug. Thunderbird development was practically nonexistent 4 years ago, we have more resources now. But it's not productive to just randomly work on every bug from 10+ years as we come across them. There are many thousands, prioritization is necessary.

It is possible Mozilla VPN client will add split tunneling on MacOS(it is supported on Linux and Windows already...) at some point which will make it easier to work around this.

But I have given up on any real bugs being fixed by Mozilla. I believe the fact that the "master password" uses an insecure unsalted md5 hash has been known for over 10 years, yet a simple change of the hash algorithm hasn't been done. The only thing the so-called SW engineers care about is making add-on deveolpment more and more bloated and ineffective similar to Apple's eco-system where you have to rewrite your application code almost every year due to deprecated APIs that never get a proper replacement.

This sort of unrelated rant isn't going to help any bug get fixed faster FYI. And no, MP definitely does not use MD5. IIRC it uses 10K iterations of SHA1. But that is way off-topic for this bug, so please, lets not.

(In reply to Andrei Hajdukewycz [:sancus] from comment #16)

Additionally: I tested this on my MacBook and it does seem like Thunderbird struggles right after switching VPNs, but connecting to folders still works after a few tries. It wouldn't surprise me if the autosync gets stuck or fails though, as we have a number of autosync issues.

I'm considering the switching of folders - where you have to actually also pick the right ones - as a workaround. Usually I have my inbox folder open and don't have to switch at all. Using the VPN under such conditions makes the Thunderbird autosync broken and you wont notice that because no failure is shown by autosync as well. As such I frequently forget to switch folders and end-up with not seeing emails.

It would be great if this bug could be considered at least for the triage process. Thanks.

Possible related Thunderbird bugs https://mzl.la/3Nkibix; Gecko bugs https://mzl.la/41a7pRH

Blocks: 1864458, 1638595
See Also: 1638595 → 1230823

(In reply to Andrei Hajdukewycz [:sancus] from comment #15)

This isn't actually happening to all VPN users, I can't reproduce it at all on Windows using Mozilla VPN. When I turn the VPN on and off Thunderbird does have to create a new connection when clicking on a folder, which is a bit slower, but it doesn't time out or fail.

So there's more to this than just "VPNs". Might be MacOS specific, but I'm just guessing.

I regularly use Mozilla VPN (viscosity) and another VPN on Mac and don't notice a problem. I agree, something more specific is involved.

(In reply to Wayne Mery (:wsmwk) from comment #20)

Possible related Thunderbird bugs https://mzl.la/3Nkibix

modified query https://mzl.la/3RtmAks

Summary: Thunderbird times out after connecting to VPN → Thunderbird times out after connecting to VPN / switching network

(In reply to Henrik Skupin [:whimboo][⌚️UTC+1] from comment #19)

I frequently forget to switch folders and end-up with not seeing emails.

It would be great if this bug could be considered at least for the triage process. Thanks.

Henrik, Is this better or worse with version 115? (Some have reported it is worse)

Flags: needinfo?(hskupin)

So far I haven't seen this problem since I've upgraded to 115.

Flags: needinfo?(hskupin)

That's great news. Thanks Henrik.

Status: NEW → RESOLVED
Closed: 9 months ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.