1892521 - NordVPN's Web Protection feature is breaking Firefox auto-updates on a massive scale

Status: RESOLVED WORKSFORME

(Toolkit :: Application Update, defect, P3)

Description

[Noah (NoahSUMO/SolidSnake) [:Noah]]

Attachment: Firefox Update Failure popup.png

NordVPN's Web Protection feature is causing auto-updates to fail for Windows users. Maybe even for Mac & Linux users too.
It's also causing users to constantly see a "Failed to check for updates" error popup or get stuck in a update loop that produces lots of “Restart Required” tabs..

So far I'm only seeing Windows reports & a few Mac reports (it doesn't help that bug 1890764 recently broke Mac auto-updates at the exact same time as this is surfacing).

NordVPN breaking Fx auto-updates:
https://support.mozilla.org/questions/1445093
https://support.mozilla.org/questions/1445093#answer-1648171
^ 2 error messages resolved by disabling NordVPN Web Protection:
"Failed to check for updates." & "Firefox is installing components needed". [WideVine plugin, Needed for popular streaming sites]
https://support.mozilla.org/questions/1445030
https://support.mozilla.org/questions/1445254
https://support.mozilla.org/questions/1445068
https://support.mozilla.org/questions/1445187
https://support.mozilla.org/questions/1443087 [Mac + 1 Windows user]
https://support.mozilla.org/questions/1445127
https://support.mozilla.org/questions/1444859
https://support.mozilla.org/questions/1445275

Waiting on confirmation:
https://support.mozilla.org/questions/1445013
https://support.mozilla.org/questions/1445017
https://support.mozilla.org/questions/1444938
“I have manually upgraded to 125.0. Unfortunately this issue still occurs where the “Restart Required” page pops up. It repeatedly pops up when I leave my computer and it idles and I come back to multiple tabs opened with the same message. It seems to be getting worse the longer I stay inactive on the computer.”

Mac only case, waiting to see if NordVPN is breaking it here:
https://support.mozilla.org/questions/1445059

R/firefox:
https://www.reddit.com/r/firefox/comments/1bly9xz/firefox_suddenly_unable_to_auto_update/

---

Updater breakage first surfaced back in February:
I saw in Bug 1882059 (reported back on Feb 26, 2024) that we don't have any NordVPN contacts to reach out to. What is the process for establishing a contact? Does a Mozilla engineer have to physically drive down to the NordVPN offices to secure one? If that's what it takes, I suggest it be done.
(I hope that made someone laugh)

Maybe it's easier just to @ them on Twitter?
"Why is your VPN breaking millions of Firefox users ability to auto-update? @NordVPN @tom_okman @laura_tyrell Please reply quickly, thanks"

Reach out to NordVPN's CEO or CTO. Can any of this be done? Let's not accept a "wait and see" approach.
Twitter:
https://twitter.com/tom_okman (Co-founder Nord Security. Lithuanian. @nordvpn)
https://twitter.com/laura_tyrell (Laura Tyrell - Head of Public Relations @ #NordSecurity)
Marijus Briedis - NordVPN CTO - IG: https://www.instagram.com/marijusbriedis/
Threads: https://www.threads.net/@marijusbriedis - LinkedIn: https://www.linkedin.com/in/marijusbriedis/
Karolis Pabijanskas - Staff engineer, NordVPN - LinkedIn: https://www.linkedin.com/in/kpabijanskas/

"Meet the experts" section has staff options:
https://nordvpn.com/cybersecurity/ask-an-expert/

I would like to see as much pressure as possible applied on Nord to get a few NordVPN contacts to investigate this issue with us. A ton of Firefox users use their VPN.

Anyway I help out on support.mozilla.org & would like to see this escalated before it turns into a huge issue. I can only do so much. If it turns out to be a bug/regression on our side that caused this, so be it. But can we please treat it seriously in the meantime? Thanks

Comment 1

[Robin Steuber (she/her) [:bytesized]]

Can you reproduce this problem? Or you are just reporting a problem that other people are experiencing?

Comment 2

[Noah (NoahSUMO/SolidSnake) [:Noah]]

Reporting a problem many others are experiencing.
I don't have access to NordVPN so I can't test this myself but I've been tracking this emerging issue for a few days now on SUMO.

I guess you could say I'm triaging these reports so that the severity & urgency can been seen quickly by the people who make the decisions to raise its priority. That's why I CCed RyanVM. I trust him to get the word out to the right people.

I'm hoping our QA team can reproduce this or even any of you who have the time & means to do so.

Comment 3

[Noah (NoahSUMO/SolidSnake) [:Noah]]

Attachment: Firefox Restart Required page 2024-04-16.png

Screenshot of a update loop a Win 10 user was experiencing the longer he left his Firefox idle
https://support.mozilla.org/en-US/questions/1444938#answer-1647761

Comment 4

[Noah (NoahSUMO/SolidSnake) [:Noah]]

Clearing the NIs. I forgot how to properly CC people. D'oh! Please forgive me, I haven't filed a bug in years. :)

Comment 5

[Nick Alexander :nalexander [he/him]]

In this case, a bunch of NIs is not a terrible approach!

I just paid for and installed NordVPN on my Windows 11 box. I can confirm the failures exactly as written here and in various SUMO tickets. I conjecture that Nord Web Thread Protection has a very broad "block *.mozilla.org" policy which explains why users are failing to install:

• updates;
• addon updates;
• CDMs.

I will file a support ticket now and try to get this escalated.

Comment 6

[Nick Alexander :nalexander [he/him]]

I can also confirm that disabling "Web protection" in NordVPN allows these connections to succeed.

Comment 7

[Nick Alexander :nalexander [he/him]]

Support ticket with NordVPN opened: ID #16004598. Will update as and when.

Comment 8

[Noah (NoahSUMO/SolidSnake) [:Noah]]

Thanks Nick! Guess I still know what I'm doing haha

And thanks a million for purchasing NordVPN, confirming the breakage & escalating a ticket with them!
I'm really happy it was that easy to reproduce.
I wonder how long this might have been breaking Firefox updates. Hopefully Nord can let us know when they added *.mozilla.org urls to their blocklist & can just as easily reverse it.

Thanks to everyone here. I'll just sit back & watch you do your stuff. ;)

Comment 9

[Lucas Siebert]

SUMO KB article: https://support.mozilla.org/en-US/kb/troubleshoot-firefox-auto-update-issues

Comment 12

[Nick Rishel [:nrishel]]

Dropping severity as we have circumstantial reports this has been fixed (but have not heard back from NordVPN directly).

Comment 13

[Nick Alexander :nalexander [he/him]]

I updated my Windows 11 device to NordVPN 7.23.1.0 and, with "Web protection" in NordVPN enabled, application updates succeeded.

I haven't had any messaging from Nord about the status of my ticket, so it's technically possible that I'm in one branch of an A/B test. I'll ask Nord for confirmation shortly.

Comment 14

[Nick Alexander :nalexander [he/him]]

I'm thrilled to report that Nord has gotten back to me, saying (in part):

The issue has been addressed for everyone with the release of the NordVPN Windows application version 7.23.1.0.

This agrees with my testing so I'm going to resolve this. Hurray for Nord, and thanks to everybody who reported the ticket.