1915729 - Add a test for client transitioning networks, HTTP/3 requests

Status: RESOLVED WONTFIX

(Core :: Networking, task, P3)

Description

[Andrew Creskey [:acreskey]]

Test of Firefox allowing the HTTP/3 network request to complete even if the client changes network interfaces midway.

For example, when a mobile user migrates from Wifi to Cellular.

i.e. as described here:
https://www.debugbear.com/blog/http3-quic-protocol-guide

By attaching an unencrypted connection identifier (CID) to each QUIC packet header, QUIC doesn’t have to reset the connection like TCP if the device switches to a new network (for example, from a 4G network to Wi-Fi, or vice versa) or the IP addresses or port numbers change for any other reason.

With the help of connection migration, QUIC doesn’t have to redo the handshake under the new conditions and HTTP/3 doesn’t have to

Comment 1

[Andrew Creskey [:acreskey]]

Attachment: Bug 1915729 - Add a test for client transitioning networks, HTTP/3 requests r=#necko-reviewers,sparky

WIP

Comment 2

[Max Inden]

Idea for an automated test:

We somehow need to trick the client into believing that its local interface changed. I assume that we don't have any test tooling, allowing us to change client interfaces. If I am wrong, i.e. we do, let's use that, not the hack I am proposing below.

If I understand correctly we don't leverage IPV6_V6ONLY socket option anywhere. In other words, all our IPv6 sockets are dualstack sockets, i.e. accept both incoming v4 and v6 packets.

1. Have Firefox connect to an IPv6 HTTP3 server, e.g. [::1]:8080. Given the above assumption of Firefox not doing IPV6_ONLY, this will result in Firefox binding to a dualstack (v4 & v6) socket. See socket instantiation in HttpConnectionUDP.cpp. Common behavior across OSs is to bind to the same port, e.g. Firefox will have the local address [::1]:12345 and 127.0.0.1:12345.
2. Wait till the connection is fully established.
3. Fake the client going from a v4&v6 network to a v4 network. This can be done on the server side, no changes to Firefox needed. On the server side, when reading a datagram, simply replace the source address (i.e. the Firefox address) from [::1]:12345 to 127.0.0.1:12345. The server will automatically respond to the v4 client address, i.e. 127.0.0.1:12345. See sample unit test, using ports instead of addresses.
4. Wait till client and server finish the migration, by validating that the client can still send and receive data from the server and does not throw TransportError::InvalidMigration.

Comment 3

[Andrew Creskey [:acreskey]]

(In reply to Max Inden from comment #2)

Idea for an automated test:

We somehow need to trick the client into believing that its local interface changed. I assume that we don't have any test tooling, allowing us to change client interfaces. If I am wrong, i.e. we do, let's use that, not the hack I am proposing below.

If I understand correctly we don't leverage IPV6_V6ONLY socket option anywhere. In other words, all our IPv6 sockets are dualstack sockets, i.e. accept both incoming v4 and v6 packets.

1. Have Firefox connect to an IPv6 HTTP3 server, e.g. [::1]:8080. Given the above assumption of Firefox not doing IPV6_ONLY, this will result in Firefox binding to a dualstack (v4 & v6) socket. See socket instantiation in HttpConnectionUDP.cpp. Common behavior across OSs is to bind to the same port, e.g. Firefox will have the local address [::1]:12345 and 127.0.0.1:12345.
2. Wait till the connection is fully established.
3. Fake the client going from a v4&v6 network to a v4 network. This can be done on the server side, no changes to Firefox needed. On the server side, when reading a datagram, simply replace the source address (i.e. the Firefox address) from [::1]:12345 to 127.0.0.1:12345. The server will automatically respond to the v4 client address, i.e. 127.0.0.1:12345. See sample unit test, using ports instead of addresses.
4. Wait till client and server finish the migration, by validating that the client can still send and receive data from the server and does not throw TransportError::InvalidMigration.

Thanks Max.
While exploring this, I decided that I wanted a scenario that mimics the issue found in bug 1910991 (client changes networks).
Can you have a look at the attached patch? It's not exactly what I wanted, but it's effective for running this locally.

Comment 4

[Lars Eggert [:lars]]

The QUIC network simulator has rebind scenarios which simulate this: https://github.com/quic-interop/quic-network-simulator/tree/master/sim/scenarios/rebind

We should enable them in CI, if possible.

Comment 5

[Max Inden]

(In reply to Andrew Creskey [:acreskey] from comment #3)

Can you have a look at the attached patch? It's not exactly what I wanted, but it's effective for running this locally.

Patch makes sense to me. That said, given that it can only run locally, i.e. needs manual user input, does it make sense to merge it into mozilla-central? I assume anything not continuously run by CI is doomed to become outdated. Feel free to ignore, as I am not familiar with the conventions at Mozilla.

Comment 6

[Andrew Creskey [:acreskey]]

(In reply to Max Inden from comment #5)

(In reply to Andrew Creskey [:acreskey] from comment #3)

Can you have a look at the attached patch? It's not exactly what I wanted, but it's effective for running this locally.

Patch makes sense to me. That said, given that it can only run locally, i.e. needs manual user input, does it make sense to merge it into mozilla-central? I assume anything not continuously run by CI is doomed to become outdated. Feel free to ignore, as I am not familiar with the conventions at Mozilla.

I think the current state of the test is far from ideal.
It's handy (at least for me) to have it in-tree, but I agree it loses most of its value and runs the risk of bit-rotting.

Comment 7

[Andrew Creskey [:acreskey]]

(In reply to Lars Eggert [:lars] from comment #4)

The QUIC network simulator has rebind scenarios which simulate this: https://github.com/quic-interop/quic-network-simulator/tree/master/sim/scenarios/rebind

We should enable them in CI, if possible.

This is interesting -- are there existing tests that run against the quic-network-simulator?

Comment 8

[Andrew Creskey [:acreskey]]

Some discussion in the patch, https://phabricator.services.mozilla.com/D224261.
Ideally I can find a way to test this through the full Firefox stack, as the bug we're seeing with cloudflare would then be covered.

Comment 9

[Andrew Creskey [:acreskey]]

Might be getting close to a test that's CI friendly and exercises the code we want.
Using the idea of moving the client, Firefox, from ipv6 to ipv4.
https://phabricator.services.mozilla.com/D224261

Comment 10

[Max Inden]

(In reply to Andrew Creskey [:acreskey] from comment #7)

(In reply to Lars Eggert [:lars] from comment #4)

The QUIC network simulator has rebind scenarios which simulate this: https://github.com/quic-interop/quic-network-simulator/tree/master/sim/scenarios/rebind

We should enable them in CI, if possible.

This is interesting -- are there existing tests that run against the quic-network-simulator?

Yes. Thanks to Lars, we now run them on each pull request, against various QUIC implementations (e.g. Cloudflare and Google Quiche).

https://github.com/mozilla/neqo/actions/workflows/qns.yml

See for example the test results on the pull request below:

https://github.com/mozilla/neqo/pull/2186#issuecomment-2419643838

Once https://github.com/mozilla/neqo/pull/2180/ merges, we will run the various quic-interop-runner migration tests continuously on Neqo's CI.

Comment 11

[Andrew Creskey [:acreskey]]

Unfortunately, it's not currently feasible to test this at the Firefox level in CI.