1363651 - http://detectportal.firefox.com/ is hammering our firewalls

Status: RESOLVED FIXED

(Core :: Networking, defect)

Description

[oldfirefoxuser69]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0
Build ID: 20170419042421

Steps to reproduce:

The clients have firefox installed/updated on their desktops/laptops.


Actual results:

Now that http://detectportal.firefox.com/success.txt is enabled by default, we have hundreds and hundreds of requests constantly hammering our firewalls.


Expected results:

Instead of having the firefox installs all try to reach multiple ip addresses on the public web, a solution such as anycast dns could be used. Thus one ip address is used globally to resolve detectportal.firefox.com. An example is google.com dns at 8.8.8.8. It is probably hosted in a town next to you. Firewall admins in large enterprises could allow that one ip address through instead of watching the firewalls performance/user experience being affected.

Comment 1

[oldfirefoxuser69]

Hi all,
Please find the constantly growing lists of ip addresses that we recorded the clients firefox browsers trying to connect so as to reach http://detectportal.firefox.com/success.txt
This is as of today this morning and will very likely have more added...
54.192.117.50
54.192.117.45
54.192.117.169
54.192.117.249
54.192.117.236
54.192.117.19
52.84.50.51
54.192.117.130
54.192.117.190
52.84.50.197
52.84.50.13
52.85.107.207
52.85.82.197
52.84.50.209
52.84.50.194
52.84.50.227
52.84.246.210
52.222.232.9
52.84.246.178
52.222.174.149
52.84.246.103
52.222.232.110
52.85.107.173
52.85.82.109
52.85.82.32
N52.85.82.197
52.85.82.56
52.85.82.42
52.85.82.4
52.85.82.53
52.85.82.89
52.84.50.210 
52.85.77.102 
52.85.77.27
13.32.16.157 
13.32.16.76 
52.84.50.150 
52.85.77.79 
54.230.197.223 
54.230.197.151 
54.230.197.109 
52.85.77.177 
52.85.77.149 
52.84.50.254 
52.84.50.196 
52.84.50.159 
52.84.126.106 
52.222.232.72 
52.222.232.133 
52.222.174.27 
52.222.174.210 
52.222.174.15 
52.222.149.130 
13.32.16.77 
13.32.16.226 
13.32.16.125
52.222.232.202 
52.222.232.252 
54.230.197.207 
52.222.232.190 
54.192.130.75 
52.222.232.82 
52.222.232.65 
52.222.232.136 
52.84.63.235 
52.84.63.69
52.84.50.77 
54.230.5.4 
54.230.197.63 
54.230.141.60 
54.230.141.236 
54.230.141.199 
54.230.141.180 
54.230.141.166 
54.230.141.123 
54.192.22.70 
54.192.22.32 
54.192.22.27 
54.192.22.176 
52.85.82.250 
52.85.82.242 
52.85.82.183 
52.85.82.180 
52.85.82.165 
52.85.82.13 
52.85.77.32 
52.85.77.213 
52.85.77.116 
52.85.107.74 
52.85.107.71 
52.85.107.54 
52.85.107.52 
52.85.107.134 
52.84.7.21 
52.84.7.116 
52.84.50.64 
52.84.50.22 
52.84.50.216 
52.84.126.67 
52.84.126.235 
52.84.126.21 
52.84.126.208 
52.84.126.164 
52.84.126.124 
52.222.232.54 
52.222.232.5 
52.222.232.245 
52.222.232.23 
52.222.232.137 
13.32.188.48 
13.32.16.20 
13.32.16.141 
13.32.16.14
52.84.50.16
52.85.82.109 
52.84.246.223 
52.85.77.19 
52.85.77.63 
52.85.82.128 
52.85.82.32 
52.85.82.53 
54.230.141.116 
54.230.141.149 
54.230.141.186 
54.230.141.22 
54.230.141.220 
54.230.141.254 
54.230.141.53 
54.230.141.98
13.32.16.68 
13.32.16.152 
52.222.232.33 
52.85.107.82 
52.84.246.132 
52.222.174.9 
52.222.232.28 
52.84.246.39 
52.84.50.75 
52.85.82.56 
52.84.246.39 
52.84.50.75 
52.222.232.4 
52.85.82.42 
52.222.232.199 
52.222.232.28 
52.222.232.48 
52.222.232.182 
52.85.82.4 
52.84.246.49 
52.85.107.82 
52.84.246.246 
52.85.82.89

Comment 2

[Valentin Gosu [:valentin] (he/him)]

Thanks Yves. This is probably we should look at.
I am curious why detectportal.firefox.com is a problem and our update/addons/telemetry endpoints are not.
I suspect it's because of the way the hosting and DNS is configured.
Benson, in bug 1112330 we were making progress towards setting up a new endpoint for this. Are we able to change the existing endpoint to have less of an impact on operators, similar to our other services?

Comment 3

[Benson Wong [:mostlygeek]]

The reason detectportal.firefox.com has so many IP addresses is because it served from the CDN (cloudfront). I would like to know more from the Reporter about what their firewall is doing that this is a problem.

Comment 4

[oldfirefoxuser69]

(In reply to Benson Wong [:mostlygeek] from comment #3)
> The reason detectportal.firefox.com has so many IP addresses is because it
> served from the CDN (cloudfront). I would like to know more from the
> Reporter about what their firewall is doing that this is a problem.

(In reply to Valentin Gosu [:valentin] from comment #2)
> Thanks Yves. This is probably we should look at.
> I am curious why detectportal.firefox.com is a problem and our
> update/addons/telemetry endpoints are not.
> I suspect it's because of the way the hosting and DNS is configured.
> Benson, in bug 1112330 we were making progress towards setting up a new
> endpoint for this. Are we able to change the existing endpoint to have less
> of an impact on operators, similar to our other services?

Thanks Valentin and Benson for sparing your valuable time to assist with this.
In our environment, every user has to authenticate to the firewall prior to having access.

Firefox is perhaps one of the most preferred/used web app and our app dev gurus packaged Firefox52 and deployed it to our managed devices, that is already quite a substantial number.
And the bulk of visiting clients who are also bringing in their own devices also have ther own installations of firefox.
If we were to run some metrics, I think you would be pleasantly surprised towards the popularity of the firefox web browser in our environment.

We discovered Firefox52 on just one device that had yet to authenticate would try to access http://detectportal.firefox.com/success.txt repeatedly every 3 to 5 seconds.
Multiply that to a few hundred devices.

There is also a report, Firefox52 continues to poll the url even after the user has authenticated to the firewall. I'll have to confirm that.

Below is an excerpt of the firewall logs towards one client device. I've removed the ip as this post is I think readily accessible to most.

May 12 08:08:07 *removed*: x.x.203.28 Accessed URL 52.84.50.59:http://detectportal.firefox.com/success.txt

May 12 08:08:10 *removed*: x.x.203.28 Accessed URL 52.84.50.59:http://detectportal.firefox.com/success.txt

May 12 08:08:13 *removed*: x.x.203.28 Accessed URL 52.84.50.59:http://detectportal.firefox.com/success.txt

May 12 08:08:17 *removed*: x.x.203.28 Accessed URL 52.84.50.59:http://detectportal.firefox.com/success.txt

May 12 08:08:20 *removed*: x.x.203.28 Accessed URL 52.84.50.59:http://detectportal.firefox.com/success.txt

May 12 08:08:23 *removed*: x.x.203.28 Accessed URL 52.84.50.59:http://detectportal.firefox.com/success.txt

May 12 08:08:26 *removed*: x.x.203.28 Accessed URL 52.84.50.59:http://detectportal.firefox.com/success.txt

May 12 08:08:30 *removed*: x.x.203.28 Accessed URL 52.84.50.59:http://detectportal.firefox.com/success.txt

May 12 08:08:33 *removed*: x.x.203.28 Accessed URL 52.84.50.59:http://detectportal.firefox.com/success.txt

May 12 08:08:36 *removed*: x.x.203.28 Accessed URL 52.84.50.59:http://detectportal.firefox.com/success.txt

May 12 08:08:39 *removed*: x.x.203.28 Accessed URL 52.84.50.59:http://detectportal.firefox.com/success.txt

May 12 08:08:43 *removed*: x.x.203.28 Accessed URL 52.84.50.59:http://detectportal.firefox.com/success.txt

Comment 5

[Valentin Gosu [:valentin] (he/him)]

I think bug 1359697 might also help in this situation. It should make Firefox not poll for CP status when it doesn't detect a captive portal. Do you think you could download a Nightly browser and see if its behaviour is improved? (It should).

Comment 6

[oldfirefoxuser69]

(In reply to Valentin Gosu [:valentin] from comment #5)
> I think bug 1359697 might also help in this situation. It should make
> Firefox not poll for CP status when it doesn't detect a captive portal. Do
> you think you could download a Nightly browser and see if its behaviour is
> improved? (It should).

Hello Valentin,

I'll download for both Linux and Windows and test.

We still have hundreds of visiting clients (and staff) with the own versions of Firefox installed polling for the url.
That is also an ongoing hurdle.
Perhaps your network gurus could be approached?

Thanks in advance.

Yves

Comment 7

[Valentin Gosu [:valentin] (he/him)]

Thanks for helping. We will uplift that patch to beta after it's confirmed to fix the issue.
However, it will take about a month until beta graduates to release.

(In reply to Benson Wong [:mostlygeek] from comment #3)
> The reason detectportal.firefox.com has so many IP addresses is because it
> served from the CDN (cloudfront). I would like to know more from the
> Reporter about what their firewall is doing that this is a problem.

ni? reporter for more details.
Yves, could you give Benson more info about your network setup? And why this is an issue?
Thanks!

Comment 8

[oldfirefoxuser69]

(In reply to Valentin Gosu [:valentin] from comment #7)
> Thanks for helping. We will uplift that patch to beta after it's confirmed
> to fix the issue.
> However, it will take about a month until beta graduates to release.
> 
> (In reply to Benson Wong [:mostlygeek] from comment #3)
> > The reason detectportal.firefox.com has so many IP addresses is because it
> > served from the CDN (cloudfront). I would like to know more from the
> > Reporter about what their firewall is doing that this is a problem.
> 
> ni? reporter for more details.
> Yves, could you give Benson more info about your network setup? And why this
> is an issue?
> Thanks!

Hi Benson,

It's quite a straightforward setup.
'clients devices' --> 'infrastructure' --> 'gateway-firewall with AD authentication'

Note: I also have a separate ticket with the firewall vendor to workout why the firewall crawled during the now dubbed as 'firefox ddos'... maybe I should rename the event as the FFF (Firefox Friendly Fire) event :-)

Comment 9

[oldfirefoxuser69]

Hi Benson and Valentin,

I have uninstalled v53 (it auto upgraded from v52 without asking...)  installed nightly firefox-55.0a1.en-US.win64.installer.exe and tested on Windows7.

Captured in wireshark the continuous poll to detectportal.firefox.com.
Interestingly, once I authenticated to the firewall, the polling of detectportal.firefox.com slowed down to once a min or so.

I’ve sent you two an email with the screen caps as it appears I cannot paste/attach pics to this bug request.

Please inform if you require further testing performed.

Thanks,
Yves

Comment 10

[Daniel Veditz [:dveditz]]

(In reply to oldfirefoxuser69 from comment #9)
> [...] it auto upgraded from v52 without asking...

That's the default behavior for all major browsers. Given the number, ease of access, and persistence of attackers on the internet and the number of users who don't bother with manual/optional update any other setting is malpractice. You can adjust it to ask first in settings.

Comment 11

[Benson Wong [:mostlygeek]]

Attachment: before firewall auth (nightly 55)

Comment 12

[Benson Wong [:mostlygeek]]

Attachment: Screencap-with-Nightly-auth.jpg

Comment 13

[Benson Wong [:mostlygeek]]

Yves: Is it possible for you to also screencap the response before and after authenticating for the firewall?
I believe you don't need to use wireshark. You can use the Browser Console's [1] network tab to view all the traffic Firefox is doing. Only enable the "Net" message and filter for detectportal.mozilla.org.

[1] https://developer.mozilla.org/en/docs/Tools/Browser_Console

Comment 14

[oldfirefoxuser69]

(In reply to Daniel Veditz [:dveditz] from comment #10)
> (In reply to oldfirefoxuser69 from comment #9)
> > [...] it auto upgraded from v52 without asking...
> 
> That's the default behavior for all major browsers. Given the number, ease
> of access, and persistence of attackers on the internet and the number of
> users who don't bother with manual/optional update any other setting is
> malpractice. You can adjust it to ask first in settings.

That's a discussion best kept for much later and a different thread where you would have replies from experienced experts in the field... perhaps best discussed on /.?

Comment 15

[oldfirefoxuser69]

(In reply to Benson Wong [:mostlygeek] from comment #13)
> Yves: Is it possible for you to also screencap the response before and after
> authenticating for the firewall?
> I believe you don't need to use wireshark. You can use the Browser Console's
> [1] network tab to view all the traffic Firefox is doing. Only enable the
> "Net" message and filter for detectportal.mozilla.org.
> 
> [1] https://developer.mozilla.org/en/docs/Tools/Browser_Console

Hi Benson,
No worries.
Which version of Firefox?

Comment 16

[Valentin Gosu [:valentin] (he/him)]

So, is the reduced number of requests to the captive portal enough to consider this bug fixed?

Comment 17

[Benson Wong [:mostlygeek]]

(In reply to oldfirefoxuser69 from comment #15)

> Which version of Firefox?

With 55nightly, like your previous screenshots I uploaded. 

Also: 

We are rolling out FF53.0.3 today which includes Bug 1359697, which should fix the polling. However, that patch should already be in Nightly (FF55). Which is why I'd like to see if it is still polling in your network. It shouldn't be.

Comment 18

[Valentin Gosu [:valentin] (he/him)]

A fair amount of polling is still expected, especially if the reporter is running a captive portal in their network. However, with these changes the amount of requests should be lower, so their firewalls shouldn't fail under the load.

Comment 19

[Benson Wong [:mostlygeek]]

FF 53.0.3 went out last weekend to all users. This patch includes :valentin's patch. When clients update their firefox versions your firewall should see significantly less traffic.

Comment 20

[Valentin Gosu [:valentin] (he/him)]

Thanks Benson for your help in fixing these issues.
I'm closing this as FIXED, and we'll reopen it if the reporter still has issues.

Comment 21

[oldfirefoxuser69]

Hello Benson and Valentin,

Apologies for the delay.

Have tested v53.0.3 this morning
I can confirm this bug as fixed in v53.0.3.
A picture has been emailed to your attention.

Really appreciate the assistance with our hurdles.
If you do need us to do the testing with future releases, I volunteer to test in my spare time.

Thanks again and hope you have a great year.


Yves

Comment 22

[SomaticCorpse]

I can confirm that this still occurs in Nightly [56.0a1 (2017-07-29) (64-bit)]. Since you use the same ETag and X-Amz-Cf-Id values, I can't demonstrate the multiple requests per second in this form but I can provide a Fiddler proving reproduction of the issue.

HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 8
Last-Modified: Mon, 15 May 2017 18:04:40 GMT
ETag: "ae780585f49b94ce1444eb7d28906123"
Accept-Ranges: bytes
Server: AmazonS3
X-Amz-Cf-Id: XzFHsVNetMNLaMez4UH5lr3M6gYjPtoqfOt5jKMXSsV4pQw41BHvrw==
Cache-Control: no-cache, no-store, must-revalidate
Date: Sun, 30 Jul 2017 15:53:20 GMT
Connection: keep-alive

success

HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 8
Last-Modified: Mon, 15 May 2017 18:04:40 GMT
ETag: "ae780585f49b94ce1444eb7d28906123"
Accept-Ranges: bytes
Server: AmazonS3
X-Amz-Cf-Id: XzFHsVNetMNLaMez4UH5lr3M6gYjPtoqfOt5jKMXSsV4pQw41BHvrw==
Cache-Control: no-cache, no-store, must-revalidate
Date: Sun, 30 Jul 2017 15:53:24 GMT
Connection: keep-alive

success

HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 8
Last-Modified: Mon, 15 May 2017 18:04:40 GMT
ETag: "ae780585f49b94ce1444eb7d28906123"
Accept-Ranges: bytes
Server: AmazonS3
X-Amz-Cf-Id: XzFHsVNetMNLaMez4UH5lr3M6gYjPtoqfOt5jKMXSsV4pQw41BHvrw==
Cache-Control: no-cache, no-store, must-revalidate
Date: Sun, 30 Jul 2017 15:53:27 GMT
Connection: keep-alive

success

HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 8
Last-Modified: Mon, 15 May 2017 18:04:40 GMT
ETag: "ae780585f49b94ce1444eb7d28906123"
Accept-Ranges: bytes
Server: AmazonS3
X-Amz-Cf-Id: XzFHsVNetMNLaMez4UH5lr3M6gYjPtoqfOt5jKMXSsV4pQw41BHvrw==
Cache-Control: no-cache, no-store, must-revalidate
Date: Sun, 30 Jul 2017 15:53:28 GMT
Connection: keep-alive

success

HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 8
Last-Modified: Mon, 15 May 2017 18:04:40 GMT
ETag: "ae780585f49b94ce1444eb7d28906123"
Accept-Ranges: bytes
Server: AmazonS3
X-Amz-Cf-Id: XzFHsVNetMNLaMez4UH5lr3M6gYjPtoqfOt5jKMXSsV4pQw41BHvrw==
Cache-Control: no-cache, no-store, must-revalidate
Date: Sun, 30 Jul 2017 15:53:29 GMT
Connection: keep-alive

success

(It seems I can't reactivate the bug, or I would do that.)

Comment 23

[SomaticCorpse]

Sorry, full session:

GET http://detectportal.firefox.com/success.txt HTTP/1.1
Host: detectportal.firefox.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0
Accept: */*
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Cache-Control: no-cache
Pragma: no-cache
Connection: keep-alive


HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 8
Last-Modified: Mon, 15 May 2017 18:04:40 GMT
ETag: "ae780585f49b94ce1444eb7d28906123"
Accept-Ranges: bytes
Server: AmazonS3
X-Amz-Cf-Id: XzFHsVNetMNLaMez4UH5lr3M6gYjPtoqfOt5jKMXSsV4pQw41BHvrw==
Cache-Control: no-cache, no-store, must-revalidate
Date: Sun, 30 Jul 2017 15:53:20 GMT
Connection: keep-alive

success

GET http://detectportal.firefox.com/success.txt HTTP/1.1
Host: detectportal.firefox.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0
Accept: */*
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Cache-Control: no-cache
Pragma: no-cache
Connection: keep-alive


HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 8
Last-Modified: Mon, 15 May 2017 18:04:40 GMT
ETag: "ae780585f49b94ce1444eb7d28906123"
Accept-Ranges: bytes
Server: AmazonS3
X-Amz-Cf-Id: XzFHsVNetMNLaMez4UH5lr3M6gYjPtoqfOt5jKMXSsV4pQw41BHvrw==
Cache-Control: no-cache, no-store, must-revalidate
Date: Sun, 30 Jul 2017 15:53:24 GMT
Connection: keep-alive

success

GET http://detectportal.firefox.com/success.txt HTTP/1.1
Host: detectportal.firefox.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0
Accept: */*
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Cache-Control: no-cache
Pragma: no-cache
Connection: keep-alive


HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 8
Last-Modified: Mon, 15 May 2017 18:04:40 GMT
ETag: "ae780585f49b94ce1444eb7d28906123"
Accept-Ranges: bytes
Server: AmazonS3
X-Amz-Cf-Id: XzFHsVNetMNLaMez4UH5lr3M6gYjPtoqfOt5jKMXSsV4pQw41BHvrw==
Cache-Control: no-cache, no-store, must-revalidate
Date: Sun, 30 Jul 2017 15:53:27 GMT
Connection: keep-alive

success

GET http://detectportal.firefox.com/success.txt HTTP/1.1
Host: detectportal.firefox.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0
Accept: */*
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Cache-Control: no-cache
Pragma: no-cache
Connection: keep-alive


HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 8
Last-Modified: Mon, 15 May 2017 18:04:40 GMT
ETag: "ae780585f49b94ce1444eb7d28906123"
Accept-Ranges: bytes
Server: AmazonS3
X-Amz-Cf-Id: XzFHsVNetMNLaMez4UH5lr3M6gYjPtoqfOt5jKMXSsV4pQw41BHvrw==
Cache-Control: no-cache, no-store, must-revalidate
Date: Sun, 30 Jul 2017 15:53:28 GMT
Connection: keep-alive

success

GET http://detectportal.firefox.com/success.txt HTTP/1.1
Host: detectportal.firefox.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0
Accept: */*
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Cache-Control: no-cache
Pragma: no-cache
Connection: keep-alive


HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 8
Last-Modified: Mon, 15 May 2017 18:04:40 GMT
ETag: "ae780585f49b94ce1444eb7d28906123"
Accept-Ranges: bytes
Server: AmazonS3
X-Amz-Cf-Id: XzFHsVNetMNLaMez4UH5lr3M6gYjPtoqfOt5jKMXSsV4pQw41BHvrw==
Cache-Control: no-cache, no-store, must-revalidate
Date: Sun, 30 Jul 2017 15:53:29 GMT
Connection: keep-alive

success

Comment 24

[Valentin Gosu [:valentin] (he/him)]

I thought the fix worked, but maybe it regressed since we landed it.
Could you open a new bug and include your steps to reproduce?
Also, it would be super helpful if you included some logs.
Use the steps mentioned here: https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging#Using_aboutnetworking
In step 5, please paste in the following modules: timestamp,sync,CaptivePortalService:5

Thanks!

Comment 25

[Valentin Gosu [:valentin] (he/him)]

Still waiting on reply from reporter (SomaticCorpse) .

Comment 26

[Marvil]

I can confirm the same issue in 55.0.2 (64-bit) over Ubuntu 16.04. The clients flood the proxy trying to connect  detectportal.firefox.com 

Marvil

Comment 27

[Benson Wong [:mostlygeek]]

The CDN metrics have been showing traffic increasing fairly steadily for the past few weeks: 

https://screenshots.firefox.com/erBTkNBDhFv6hiJn/app.datadoghq.com

Comment 28

[Benson Wong [:mostlygeek]]

Re: Comment 27 

The blue bars are the daily traffic average and the purple line is the weekly average.

Comment 29

[Benson Wong [:mostlygeek]]

:valentin do you think we should reopen/look at this bug in case it affects 57?

Comment 30

[Valentin Gosu [:valentin] (he/him)]

We haven't made any changes to the captive portal logic recently, so that either means we are getting more users or that large ISPs are deploying captive portals (I hope it's the first).

(In reply to Benson Wong [:mostlygeek] from comment #29)
> :valentin do you think we should reopen/look at this bug in case it affects
> 57?

Regarding the "flooding the proxy" issue, there may be two sides of this
1. The networks are running a captive portal, in which case it is normal to expect a higher number of requests to our endpoint as the browser attempts to find out what the status of the CP is.
2. The resolved IP address of the endpoint changes too often, causing too many rules to be installed in firewalls.

Benson, do you think there is something we can do about no 2.?  Is there any way to configure the CDN to only offer one IP address, with a longer TTL, and that changes less often?

Comment 31

[Benson Wong [:mostlygeek]]

The IP ranges in Cloudfront are published in: https://ip-ranges.amazonaws.com/ip-ranges.json.  It looks like there are 42 IP ranges for all the edge nodes. 

My main concern was with comment 20 and comment 21.  Since there doesn't seem to be a follow up I'm not sure we should pursue it.

Comment 32

[Benson Wong [:mostlygeek]]

I mean comment 22 and comment 23.

Comment 33

[Valentin Gosu [:valentin] (he/him)]

(In reply to Benson Wong [:mostlygeek] from comment #31)
> My main concern was with comment 22 and comment 23.  Since there doesn't
> seem to be a follow up I'm not sure we should pursue it.

Indeed. If we get any more reports of this we'll open a different bug. Thanks!

Comment 34

[Marvil]

(In reply to Valentin Gosu [:valentin] from comment #30)
...
> Regarding the "flooding the proxy" issue, there may be two sides of this
> 1. The networks are running a captive portal, in which case it is normal to
> expect a higher number of requests to our endpoint as the browser attempts
> to find out what the status of the CP is.
> 2. The resolved IP address of the endpoint changes too often, causing too
> many rules to be installed in firewalls.
...

There's another side: very restrictive CSO and policies, and no captive portal. They're not going to whitelist "detectportal.firefox.com". The whole (and huge) list of clients are banging their GETs against the proxy, only to get five dozens of 403 errors, every single client... per minute!. It doesn't have any sense.

Comment 35

[Marvil]

Correct: five dozens of 403 errors, every single client... per seconds!

Comment 36

[Benjamin Bach]

Attachment: syslog output

I'm running an offline environment and trying to disable online behaviors.

I can see from my DNS server (syslog output below) that a single Firefox instance will create this hoard of DNS lookups, with a seemingly increasing interval. I have disabled NetworkManager from this behavior, but Firefox does this on its own, effectively duplicating Network Manager's responsibility.

You can see the log to get an impression of how bad this is.

Reading this: https://support.mozilla.org/en-US/kb/how-stop-firefox-making-automatic-connections

...makes me REALLY miss a system-wide killswitch for all such outgoing requests.

```

```

Comment 37

[Benjamin Bach]

(above comment posted without my consent, apparently pasting + attaching posts the comment with no edit options)

Above syslog output is the behavior of Firefox 69.0

The reason why I'm posting on a closed bug report, is that I wanted to hear back from the audience of the currently closed issue: I would propose 2 things...

1. A killswitch setting for all outgoing connections
2. Disabling Hotspot detection by default since this is normally done by the operating system

Do people here find it relevant/sufficient to continue along these two tracks?

Comment 38

[oldfirefoxuser69]

This behaviour was not observed in just Firefox 69... It has been ongoing issue I would suspect even with the latest releases.
This bug was closed but the issue never ceased... I can still pull logs from our firewalls showing the behaviour to this day.

My 0.02c below:

1. A killswitch would have to be controlled by the sysadmins in an enterprise environment. Usually Windows clients within GPOs/SCCM/etc...
   For the advanced home user type, this could be a great option.
   Towards BYOD clients, the device owner would have to be aware of the option and care about using it...

2. The hostspot detection is very useful to us as it redirects non dot1x clients to a web portal where they authenticate prior to accessing the web.
   True, most OSes now do this but if some mechanism like dot1x is used, the web browsers redirects to the authentication portal where an authorised session is created with the clients credentials.

3. What I think would also really assist is something like Using Cloudflares Anycast DNS services, see https://www.cloudflare.com/learning/dns/what-is-anycast-dns/
   Thus, the web browser would always query something like Anycast DNS IPs such as 1.1.1.1, 8.8.8.8, 9.9.9.9, etc for a Firefox Global Anycast IP address of whatever your network/sys admins decides on. You can also use Anycast with IPv6.

Thoughts everyone?