Closed Bug 1595779 Opened 1 year ago Closed 1 year ago

Crash [@ neqo_http3::connection::Http3Connection<T>::close<T> ] with HTTP3 enabled

Categories

(Core :: Networking: HTTP, defect, P1)

x86_64
Windows 10
defect

Tracking

()

RESOLVED FIXED
mozilla72
Tracking Status
firefox-esr68 --- unaffected
firefox70 --- unaffected
firefox71 --- unaffected
firefox72 + fixed

People

(Reporter: sciguyryan, Assigned: dragana)

References

(Regression)

Details

(4 keywords, Whiteboard: [necko-triaged])

Crash Data

Hi all.

After updating to today's nightly (12/11/2019) I've hit quite a few crashes with the signature [@ neqo_http3::connection::Http3Connection<T>::close<T> ].

This is (obviously) with HTTP3 enabled. Upon disabling it the crash is no longer reproducible.

11c6529d-4342-4f0f-a44e-663970191112 and db7f5454-dccf-4e59-bfba-fa9f00191112 are two examples that I have found but there are more listed here:
https://crash-stats.mozilla.org/signature/?product=Firefox&signature=neqo_http3%3A%3Aconnection%3A%3AHttp3Connection<T>%3A%3Aclose<T>&date=>%3D2019-11-05T15%3A04%3A00.000Z&date=<2019-11-12T15%3A04%3A00.000Z&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_columns=install_time&_columns=startup_crash&_sort=-date&page=1

I am hitting this crash, but with HTTP3 disabled. I have toggled the network.http.http3.enabled several times over the past few days, but I just confirmed that it was set to FALSE for the last two crashes.

I've just encountered this bug twice in a row while simply having Twitch and Twitter open!

(In reply to Mike Conca [:mconca] from comment #1)

I am hitting this crash, but with HTTP3 disabled. I have toggled the network.http.http3.enabled several times over the past few days, but I just confirmed that it was set to FALSE for the last two crashes.

That's interesting. I've yet to hit this without the preference enabled but I'll have to keep an eye out.

I disabled the preference and the constant crashes stopped on my end.

Adding the macOS specific signature.

Crash Signature: [@ neqo_http3::connection::Http3Connection<T>::close<T> ] → [@ neqo_http3::connection::Http3Connection<T>::close<T> ] [@ neqo_http3::connection::Http3Connection<T>::close ]

I disabled my network.http.http3.enabled pref, but I'm still hitting this crash signature (like a dozen times in a row) when loading this Bugzilla bug dashboard:

https://cpeterso.github.io/burndown/?since=2019-06-01&f1=cf_fission_milestone&o1=anyexact&v1=M4%2CM4.1%2CM5%2CM6%2CMVP

"assertion failed: self.state_active()"

AFAICT this assertion was added in bug 1576051.

Flags: needinfo?(shravanrn)
Flags: needinfo?(nfroyd)
Keywords: regression
Regressed by: 1576051

The bug was fixed in neqo. I need to pull new version of neqo into mozilla-central.
I will make a quick fix for bug 1595337 (the bug needsto be fixed in neqo) and I will pull a new version of neqo.

Assignee: nobody → dd.mozilla
Status: NEW → ASSIGNED

(In reply to Chris Peterson [:cpeterson] from comment #6)

I disabled my network.http.http3.enabled pref, but I'm still hitting this crash signature (like a dozen times in a row) when loading this Bugzilla bug dashboard:

https://cpeterso.github.io/burndown/?since=2019-06-01&f1=cf_fission_milestone&o1=anyexact&v1=M4%2CM4.1%2CM5%2CM6%2CMVP

I need to understand this one:
if your pref is disable you should not be hitting exactly this signature. If you do we have a different bug and I would like a http log to figure out what is happening. You may hit this bug if you had the pref on and you disable it, after disabling it there may be still some http3 connections running and when they are closed we hit this issue.

Flags: needinfo?(cpeterson)

AFAICT this assertion was added in bug 1576051.

So 1576051 was basically a mach vendor rust on the Firefox repo when working on an unrelated feature. Unfortunately, as i understand, mach vendor rust is a bit of a blunt tool, and may choose to upgrade unrelated packages which can be upgraded. From a cursory examination of 1576051, I don't believe any modules added there depends on neqo (will follow-up if I discover anything on further examination). It looks like this break is due to something in upstream neqo?

Flags: needinfo?(shravanrn)

(In reply to Shravan Narayan from comment #10)

AFAICT this assertion was added in bug 1576051.

So 1576051 was basically a mach vendor rust on the Firefox repo when working on an unrelated feature. Unfortunately, as i understand, mach vendor rust is a bit of a blunt tool, and may choose to upgrade unrelated packages which can be upgraded. From a cursory examination of 1576051, I don't believe any modules added there depends on neqo (will follow-up if I discover anything on further examination). It looks like this break is due to something in upstream neqo?

When I was doing mach vendor rust for neqo, I was told to do it based on autoland not mozilla-central. Doing it on autoland does not update any other crates except one needed.

(In reply to Dragana Damjanovic [:dragana] from comment #11)

(In reply to Shravan Narayan from comment #10)

AFAICT this assertion was added in bug 1576051.

So 1576051 was basically a mach vendor rust on the Firefox repo when working on an unrelated feature. Unfortunately, as i understand, mach vendor rust is a bit of a blunt tool, and may choose to upgrade unrelated packages which can be upgraded. From a cursory examination of 1576051, I don't believe any modules added there depends on neqo (will follow-up if I discover anything on further examination). It looks like this break is due to something in upstream neqo?

When I was doing mach vendor rust for neqo, I was told to do it based on autoland not mozilla-central. Doing it on autoland does not update any other crates except one needed.

I just realize there is another fix for that neqo crates problem. I have filed a bug.

(In reply to Dragana Damjanovic [:dragana] from comment #9)

(In reply to Chris Peterson [:cpeterson] from comment #6)

I disabled my network.http.http3.enabled pref, but I'm still hitting this crash signature (like a dozen times in a row) when loading this Bugzilla bug dashboard:

https://cpeterso.github.io/burndown/?since=2019-06-01&f1=cf_fission_milestone&o1=anyexact&v1=M4%2CM4.1%2CM5%2CM6%2CMVP

I need to understand this one:
if your pref is disable you should not be hitting exactly this signature. If you do we have a different bug and I would like a http log to figure out what is happening. You may hit this bug if you had the pref on and you disable it, after disabling it there may be still some http3 connections running and when they are closed we hit this issue.

I had disabled the pref, but then crashing before cleanly quitting Firefox. Perhaps restarting Firefox from the crash reporter dialog was not launching Firefox in a way that would pick up the new pref value? I'm not able to reproduce the crash (with the pref disabled) today.

Flags: needinfo?(cpeterson)

This by far the worst crash in nightly ATM. Dragana can you get this neqo update in today?

Flags: needinfo?(nfroyd) → needinfo?(dd.mozilla)
Keywords: topcrash

(In reply to Julien Cristau [:jcristau] from comment #14)

This by far the worst crash in nightly ATM. Dragana can you get this neqo update in today?

Just to be clear http3 is disabled by default. I just did a new neqo release and I will make m.-c. patch.

Flags: needinfo?(dd.mozilla)
Priority: -- → P1
Whiteboard: [necko-triaged]

(In reply to Dragana Damjanovic [:dragana] from comment #15)

(In reply to Julien Cristau [:jcristau] from comment #14)

This by far the worst crash in nightly ATM. Dragana can you get this neqo update in today?

Just to be clear http3 is disabled by default. I just did a new neqo release and I will make m.-c. patch.

I found another bug in neqo that needs to be fix before I can pull the new version. I hope to get a review for the neqo bug today.

Fixed by bug 1596069.

Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Depends on: 1596069
Target Milestone: --- → mozilla72
Depends on: 1601997
See Also: → 1601997
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.