Closed Bug 2089206 Opened 12 days ago Closed 7 days ago

Thunderbird tests failure in netwerk/test/unit/test_http3_with_proxy.js | test_multiple_requests

Categories

(Core :: Networking: HTTP, defect, P3)

Unspecified
Windows
defect

Tracking

()

RESOLVED FIXED
162 Branch
Tracking Status
firefox-esr153 --- unaffected
firefox159 --- unaffected
firefox160 --- unaffected
firefox161 --- fixed
firefox162 --- fixed

People

(Reporter: KaiE, Assigned: kershaw)

References

(Regression)

Details

(Keywords: intermittent-failure, regression, Whiteboard: [necko-triaged][necko-priority-next])

Attachments

(2 external files, 1 obsolete file)

Thunderbird CI reports a new test failure
netwerk/test/unit/test_http3_with_proxy.js | test_multiple_requests - [test_multiple_requests : 264] "" == "foo.example.com:55102"

Windows, only.

AI guesses it's potentially related to bug 2053088:

  • It's the only networking change in the push that touches the MASQUE proxy path this test uses. It routes the outer Firefox↔proxy connection through Happy Eyeballs, controlled by a
    new pref, network.http.happy_eyeballs_proxy_enabled. The commit message says the pref is "default off", but in StaticPrefList.yaml its value is @IS_NIGHTLY_BUILD@, so it's on in
    Daily.
  • In the failing log, test_https_alt_svc gets HTTP/3 through the proxy fine. Then most of the parallel requests in test_multiple_requests go out without H3: Alt-Used is empty, and
    some abort with 0x80004004.

AI's guess about the cause:

  • The most likely Thunderbird-specific factor is pref("network.http.speculative-parallel-limit", 0) in all-thunderbird.js, where Firefox's default is 20. The new code relies on a
    speculative transaction to set up the outer proxy connection, and the commit had to add a separate test_http3_proxy_no_speculative.js variant for exactly that case.
  • There's precedent from the same day: mkmelin's Bug 2088786 (d6c255992d96) fixed test_trr_foreground_preconnect.js on Thunderbird by setting this pref inside the test. Bug 2052138
    did the same earlier for test_http3_alt_svc.js and test_http3.js.
  • I can't explain why only Windows is affected; it's probably timing on Windows' UDP/QUIC path.

AI suggested two tests to confirm the theory.

  1. network.http.speculative-parallel-limit set to 20. If that fixes it, it's the test-side workaround with precedent, though it would also hide a real bug in Thunderbird's
    configuration.
  2. network.http.happy_eyeballs_proxy_enabled set to false. This confirms the culprit commit.

I've submitted Thunderbird try builds with that

(1)
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=eb4b1963c5afc13b12a4f28dd47a1021813e0957
(based on https://treeherder.mozilla.org/jobs?repo=try&revision=a1fc99acfdf0d2c9722285860b8c4d59077806d7 )

(2)
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=e407063ea3f5cb1196b809a4c45e3f7febc58c35
(based on https://treeherder.mozilla.org/jobs?repo=try&revision=0ad9bc192949cc7eb1ff3eb223ccf1662e305fcf )

Set release status flags based on info from the regressing bug 2053088

:kershaw, since you are the author of the regressor, bug 2053088, could you take a look? Also, could you set the severity field?

For more information, please visit BugBot documentation.

Duplicate of this bug: 2089392

@kershaw:

Both approaches fix the test failure in Thunderbird.

Which one should we use?

Let’s try option 1. I’ll prepare a patch and continue investigating the root cause.

Flags: needinfo?(kershaw)
Assignee: nobody → kershaw
Status: NEW → ASSIGNED

Set release status flags based on info from the regressing bug 2053088

Attachment #9674825 - Attachment is obsolete: true
Pushed by kjang@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/6b0fd381f545 https://hg.mozilla.org/integration/autoland/rev/3ed20c28f475 Keep alt-svc mappings when a racing connection attempt fails, r=necko-reviewers,valentin
Pushed by agoloman@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/a275bed1b2b7 https://hg.mozilla.org/integration/autoland/rev/9e3cfa75bfbf Revert "Bug 2089206 - Keep alt-svc mappings when a racing connection attempt fails, r=necko-reviewers,valentin" for causing xpc failures @test_http3_proxy_reset_keeps_alt_svc.js.

Backed out for causing xpc failures @test_http3_proxy_reset_keeps_alt_svc.js.

Flags: needinfo?(kershaw)
Flags: needinfo?(kershaw)
Pushed by kjang@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/86915956c2ca https://hg.mozilla.org/integration/autoland/rev/e7966face79b Keep alt-svc mappings when a racing connection attempt fails, r=necko-reviewers,valentin
Severity: -- → S3
Priority: -- → P3
Whiteboard: [necko-triaged][necko-priority-next]
Status: ASSIGNED → RESOLVED
Closed: 7 days ago
Resolution: --- → FIXED
Target Milestone: --- → 162 Branch

The patch landed in nightly and beta is affected.

:kershaw, please make an uplift decision for beta:

  • For beta, nominate the patch for beta uplift approval if the fix should be included in this release, or set status-firefox161 to wontfix if the fix can wait for the next release.

Uplift requests can be submitted from Lando: D340348.

See Requesting an Uplift for documentation on how to request an uplift.

For more information, please visit BugBot documentation.

Flags: needinfo?(kershaw)

firefox-beta Uplift Approval Request

  • User impact if declined/Reason for urgency: HTTP/3 Alt-svc setting could be removed during racing.
  • Code covered by automated testing?: yes
  • Fix verified in Nightly?: yes
  • Needs manual QE testing?: no
  • Steps to reproduce for manual QE testing: N/A
  • Risk associated with taking this patch: low
  • Explanation of risk level: The patch is straightforward.
  • String changes made/needed?: N/A
  • Is Android affected?: yes
Attachment #9678512 - Flags: approval-mozilla-beta?

Clearing the uplift needinfo request since an uplift has been requested for all the affected channels.

For more information, please visit BugBot documentation.

Flags: needinfo?(kershaw)
Regressions: 2091152
Attachment #9678512 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: