Open Bug 1982955 Opened 1 year ago Updated 19 days ago

12 xpcshell/test/unit/* tests fail on new android 14 emulator

Categories

(Core :: Networking, defect, P2)

defect
Points:
5

Tracking

()

People

(Reporter: jmaher, Unassigned)

References

Details

(Whiteboard: [necko-triaged] [necko-priority-next])

I found through a lot of try pushes a list of 12 tests that fail on the new android 14 emulator. These timeout and the task times out, so skipping one makes another one show up.

  • test_http3.js (try log)
  • test_http3_0rtt.js (try log)
  • test_http3_421.js
  • test_http3_early_hint_listener.js
  • test_http3_large_post.js
  • test_http3_prio_disabled.js
  • test_http3_trans_close.js
  • test_http3_version1.js
  • test_progress_no_proxy_and_proxy.js
  • test_tls13_disabled.js
  • test_websocket_with_h3_active.js
  • test_http3_timings.js

all of these seem to share a skip-if with windows.

I plan to skip these as part of bug 1982952.

and in the unit_ipc folder there is:

  • test_http3_prio_disabled_wrap.js (log file)
Severity: -- → S3
Priority: -- → P2
Whiteboard: [necko-triaged] [necko-priority-new]
Points: --- → 5
Rank: 3
Whiteboard: [necko-triaged] [necko-priority-new] → [necko-triaged] [necko-priority-next]

I don't have an Android 14 emulator to confirm this, so it needs a try push — but I believe this is UDP GSO, i.e. the same failure mode as bug 1979279.

Why the emulator upgrade would newly trigger it:

  • quinn-udp enables UDP GSO only when the kernel is >= 4.18, plus a live UDP_SEGMENT setsockopt probe (third_party/rust/quinn-udp/src/unix.rs:1005-1037). The old system-images;android-24;default;x86_64 image runs a 3.18/4.4 goldfish kernel, so GSO (and UDP_GRO) were never active there. The new system-images;android-34;google_apis;x86_64 image runs a 6.x kernel, so max_gso_segments becomes 64. Since network.http.http3.use_nspr_for_io defaults to false, all H3 I/O goes through this path.
  • security.tls.enable_kyber and network.http.http3.enable_kyber are both true by default, so the first client flight is two datagrams and is sent as a single GSO sendmsg with 2 segments. That is exactly the case bug 1979279 diagnosed on Windows ("the first is sent, but the second is lost"), which was never root-caused and is still papered over with max_gso_segments = 1 under #ifdef XP_WIN.
  • If the trailing segment is dropped on the emulator's virtio-wifi path, the ClientHello never completes, so every H3 connection fails on the first flight. That matches the immediate "WebTransport connection rejected" in bug 1982953. Here it presents as a hang instead because setup_altsvc() (head_http3.js) and waitForHttp3Route() (http3_common.js) retry with no bound.

Two hypotheses I think can be ruled out:

  • 10.0.2.2 unreachable / UDP not forwarded. Most test_trr_*.js run unskipped on android 14 and reach the host at 10.0.2.2 over TCP (head_trr.js:37). In the emulator's slirp, UDP gets the same 10.0.2.2 -> 127.0.0.1 alias translation as TCP (sotranslate_out, external/qemu slirp/socket.c), and the API 34 virtio-wifi netdev keeps the 10.0.2.0/24 defaults, only moving the guest to 10.0.2.16.
  • network.http.http3.disable_when_third_party_roots_found. xpcshell sets XPCSHELL_TEST_PROFILE_DIR, so Http3Session::Authenticated takes the automation override, which defaults to false.

Suggested try push — pref-only, on top of un-skipping two or three of these (the manifest already supports per-test prefs = [...]):

  1. network.http.http3.max_gso_segments=1 — if green, it's GSO.
  2. network.http.http3.enable_kyber=false — shrinks the first flight to one datagram; green also implicates multi-datagram sends.
  3. network.http.http3.use_nspr_for_io=true — bypasses quinn-udp entirely (no GSO/GRO/ECN); green means the fault is in the Rust datapath.

Separately, and regardless of the outcome above: the unbounded retry loops in setup_altsvc() / waitForHttp3Route() are why a single broken connection times out the whole task and makes the failing-test list look arbitrary ("skipping one makes another one show up"). Bounding them would make this class of breakage self-diagnosing.


If you'd like to provide feedback on this comment, please use the 👍 or 👎 reaction.

Flags: needinfo?(mail)

I assume this is fixed with Bug 2050874. We now retry a failed GSO by sending each datagram individually.

Flags: needinfo?(mail)
You need to log in before you can comment on or make changes to this bug.