1502025 - Use captive portal service to perform connectivity checks

Status: RESOLVED FIXED

(Core :: Networking, enhancement, P3)

Description

[Valentin Gosu [:valentin] (he/him)]

Right now the checks are done the socketTransport layer, but that means that regular browsing on websites that don't work properly may lead to the connectivity service returning the wrong results.

It would be better if we could use things the captive portal check, or the addon update check do so.

Considering the things we want to check, it should be something:
- accessible over IPv4 and IPv6
- that we connect to often
- it doesn't matter if we connect over IPv4/v6 or both at the same time

Comment 1

[Valentin Gosu [:valentin] (he/him)]

Hi Jeremy,

From what I understand the endpoint we use for captive portal checks detectportal.firefox.com is hosted on an S3 bucket.
Would it be possible to enable IPv6 for it? We want to use it to check if the user's network supports IPv6.

Thanks!

Comment 2

[Jeremy Orem [:oremj]]

This is working now:

✗ curl -v http://detectportal.firefox.com/success.txt
*   Trying 23.59.190.137...
* TCP_NODELAY set
* Connected to detectportal.firefox.com (23.59.190.137) port 80 (#0)
> GET /success.txt HTTP/1.1
> Host: detectportal.firefox.com
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 8
< Last-Modified: Mon, 15 May 2017 18:04:40 GMT
< ETag: "ae780585f49b94ce1444eb7d28906123"
< Accept-Ranges: bytes
< Server: AmazonS3
< X-Amz-Cf-Id: kUgYdtbDIrNX_jkcyy6MvN4hq0Cy_EscpxcYwco2FM-wif_8vyNkzA==
< Cache-Control: no-cache, no-store, must-revalidate
< Date: Thu, 01 Nov 2018 21:27:46 GMT
< Connection: keep-alive
<
success
* Connection #0 to host detectportal.firefox.com left intact
✗ curl -6 -v http://detectportal.firefox.com/success.txt
*   Trying ::ffff:23.59.190.114...
* TCP_NODELAY set
* Connected to detectportal.firefox.com (::ffff:23.59.190.114) port 80 (#0)
> GET /success.txt HTTP/1.1
> Host: detectportal.firefox.com
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 8
< Last-Modified: Mon, 15 May 2017 18:04:40 GMT
< ETag: "ae780585f49b94ce1444eb7d28906123"
< Accept-Ranges: bytes
< Server: AmazonS3
< X-Amz-Cf-Id: kUgYdtbDIrNX_jkcyy6MvN4hq0Cy_EscpxcYwco2FM-wif_8vyNkzA==
< Cache-Control: no-cache, no-store, must-revalidate
< Date: Thu, 01 Nov 2018 21:31:31 GMT
< Connection: keep-alive
<
success
* Connection #0 to host detectportal.firefox.com left intact

Comment 3

[Valentin Gosu [:valentin] (he/him)]

(In reply to Jeremy Orem [:oremj] from comment #2)
> This is working now:

That is awesome, thanks a lot!

Comment 4

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1502025 - Add NS_HTTP_DISABLE_IPV4 and NS_HTTP_DISABLE_IPV6 flags r=dragana!

Comment 5

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1502025 - Remove failure counters from nsSocketTransport2 r=dragana!

Depends on D13332

Comment 6

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1502025 - Use captive portal endpoints for connectivity checks r=dragana!

Depends on D13333

Comment 7

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1502025 - Add IPv6 support to httpd.js r=dragana!

Depends on D13334

Comment 8

[Valentin Gosu [:valentin] (he/him)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=89ec1ea0db8d3dea4b019691314e4449748fbfad

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/99eed75874f1
Add NS_HTTP_DISABLE_IPV4 and NS_HTTP_DISABLE_IPV6 flags r=dragana
https://hg.mozilla.org/integration/autoland/rev/9bfdb64bf81e
Remove failure counters from nsSocketTransport2 r=dragana
https://hg.mozilla.org/integration/autoland/rev/731cbca9930d
Use captive portal endpoints for connectivity checks r=dragana
https://hg.mozilla.org/integration/autoland/rev/bf4254adcdfe
Add IPv6 support to httpd.js r=dragana

Comment 10

[Stefan Hindli [:stefan_hindli]]

Backed out 4 changesets (Bug 1502025) for X failures in netwerk/test/unit/test_network_connectivity_service.js CLOSED TREE

https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&fromchange=add6ce065c17774c038c6e5762cd2996c28060fe&selectedJob=214967452

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=214967452&repo=autoland&lineNumber=13617

22:51:10     INFO -  TEST-PASS | netwerk/test/unit/test_network_connectivity_service.js | testDNS - [testDNS : 77] Check DNSv4 support (expect OK) - 1 == 1
22:51:10     INFO -  TEST-PASS | netwerk/test/unit/test_network_connectivity_service.js | testDNS - [testDNS : 78] Check DNSv6 support (expect OK) - 1 == 1
22:51:10     INFO -  TEST-PASS | netwerk/test/unit/test_network_connectivity_service.js | testDNS - [testDNS : 90] Check IPv4 support (expect UNKNOWN) - 0 == 0
22:51:10     INFO -  TEST-PASS | netwerk/test/unit/test_network_connectivity_service.js | testDNS - [testDNS : 91] Check IPv6 support (expect UNKNOWN) - 0 == 0
22:51:10     INFO -  PID 6500 | [6500, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80004002: file z:/build/build/src/toolkit/components/resistfingerprinting/nsRFPService.cpp, line 667
22:51:10     INFO -  PID 6500 | [6500, Main Thread] WARNING: NSS will be initialized without a profile directory. Some things may not work as expected.: file z:/build/build/src/security/manager/ssl/nsNSSComponent.cpp, line 1489
22:51:10     INFO -  PID 6500 | [6500, Main Thread] WARNING: This method is lossy. Use GetCanonicalPath !: file z:/build/build/src/xpcom/io/nsLocalFileWin.cpp, line 3263
22:51:10     INFO -  PID 6500 | [6500, Main Thread] WARNING: This method is lossy. Use GetCanonicalPath !: file z:/build/build/src/xpcom/io/nsLocalFileWin.cpp, line 3263
22:51:10     INFO -  PID 6500 | [6500, Main Thread] WARNING: This method is lossy. Use GetCanonicalPath !: file z:/build/build/src/xpcom/io/nsLocalFileWin.cpp, line 3263
22:51:10     INFO -  PID 6500 | [6500, Main Thread] WARNING: site security information will not be persisted: file z:/build/build/src/security/manager/ssl/nsSiteSecurityService.cpp, line 506
22:51:10     INFO -  TEST-PASS | netwerk/test/unit/test_network_connectivity_service.js | testDNS - [testDNS : 99] Check IPv4 support (expect OK) - 1 == 1
22:51:10     INFO -  TEST-PASS | netwerk/test/unit/test_network_connectivity_service.js | testDNS - [testDNS : 100] Check IPv6 support (expect OK) - 1 == 1
22:51:10  WARNING -  TEST-UNEXPECTED-FAIL | netwerk/test/unit/test_network_connectivity_service.js | testDNS - [testDNS : 108] Check IPv4 support (expect NOT_AVAILABLE) - 0 == 2
22:51:10     INFO -  Z:/task_1543616066/build/tests/xpcshell/tests/netwerk/test/unit/test_network_connectivity_service.js:testDNS:108
22:51:10     INFO -  exiting test
22:51:10     INFO -  Unexpected exception NS_ERROR_ABORT:
22:51:10     INFO -  _abort_failed_test@Z:\task_1543616066\build\tests\xpcshell\head.js:759:9
22:51:10     INFO -  do_report_result@Z:\task_1543616066\build\tests\xpcshell\head.js:866:5
22:51:10     INFO -  Assert<@Z:\task_1543616066\build\tests\xpcshell\head.js:56:5
22:51:10     INFO -  proto.report@resource://testing-common/Assert.jsm:214:5
22:51:10     INFO -  equal@resource://testing-common/Assert.jsm:250:3
22:51:10     INFO -  testDNS@Z:/task_1543616066/build/tests/xpcshell/tests/netwerk/test/unit/test_network_connectivity_service.js:108:3
22:51:10     INFO -  async*run_next_test/_run_next_test/<@Z:\task_1543616066\build\tests\xpcshell\head.js:1454:22
22:51:10     INFO -  async*_run_next_test@Z:\task_1543616066\build\tests\xpcshell\head.js:1454:10
22:51:10     INFO -  run@Z:\task_1543616066\build\tests\xpcshell\head.js:705:9
22:51:10     INFO -  _do_main@Z:\task_1543616066\build\tests\xpcshell\head.js:226:3
22:51:10     INFO -  _execute_test@Z:\task_1543616066\build\tests\xpcshell\head.js:546:5
22:51:10     INFO -  @-e:1:1
22:51:10     INFO -  exiting test
22:51:10     INFO -  PID 6500 | [6500, Socket Thread] WARNING: cannot post event if not initialized: file z:/build/build/src/netwerk/protocol/http/nsHttpConnectionMgr.cpp, line 278
22:51:10     INFO -  PID 6500 | [6500, Socket Thread] WARNING: cannot post event if not initialized: file z:/build/build/src/netwerk/protocol/http/nsHttpConnectionMgr.cpp, line 278
22:51:10     INFO -  PID 6500 | [6500, Main Thread] WARNING: OOPDeinit() without successful OOPInit(): file z:/build/build/src/toolkit/crashreporter/nsExceptionHandler.cpp, line 3156
22:51:10     INFO -  PID 6500 | nsStringStats
22:51:10     INFO -  PID 6500 |  => mAllocCount:           6273
22:51:10     INFO -  PID 6500 |  => mReallocCount:            0
22:51:10     INFO -  PID 6500 |  => mFreeCount:            6273
22:51:10     INFO -  PID 6500 |  => mShareCount:           9623
22:51:10     INFO -  PID 6500 |  => mAdoptCount:            226
22:51:10     INFO -  PID 6500 |  => mAdoptFreeCount:        226
22:51:10     INFO -  PID 6500 |  => Process ID: 6500, Thread ID: 10044
22:51:10     INFO -  <<<<<<<

Comment 11

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d20e7d5440cb
Backed out 4 changesets for X failures in netwerk/test/unit/test_network_connectivity_service.js CLOSED TREE

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e18883f3328e
Add NS_HTTP_DISABLE_IPV4 and NS_HTTP_DISABLE_IPV6 flags r=dragana
https://hg.mozilla.org/integration/autoland/rev/552ef6767e3b
Remove failure counters from nsSocketTransport2 r=dragana
https://hg.mozilla.org/integration/autoland/rev/389ecc0ddf76
Use captive portal endpoints for connectivity checks r=dragana
https://hg.mozilla.org/integration/autoland/rev/29385deef9ea
Add IPv6 support to httpd.js r=dragana

Comment 13

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/e18883f3328e
https://hg.mozilla.org/mozilla-central/rev/552ef6767e3b
https://hg.mozilla.org/mozilla-central/rev/389ecc0ddf76
https://hg.mozilla.org/mozilla-central/rev/29385deef9ea