1556911 - Startup crash @ nsObserverService::AddObserver involving TRR

Status: VERIFIED FIXED

(Core :: Networking: DNS, defect, P1)

Description

[Rolcol]

Crash reports:
https://crash-stats.mozilla.com/report/index/d36d8175-cd06-48c3-92f7-1a53e0190605
https://crash-stats.mozilla.com/report/index/7ca7ea5f-8fc0-4491-b859-17bc00190605
https://crash-stats.mozilla.com/report/index/733999b9-6fc4-4fc6-b1a6-b88f00190605

Firefox Nightly updated automatically in the background to 69.0a1 (2019-06-04). I restarted the browser using the hamburger menu's notification, and on relaunch, Nightly crashed under my regular profile. Nightly now crashes at launch every time.

Nightly does not crash with a clean profile.

Comment 1

[Ekanan Ketunuti]

You want to reset network.trr.mode to eliminate symptom.

Comment 2

[Chris Peterson [:cpeterson]]

[Tracking Requested - why for this release]:

Startup crash regression when using TRR (DoH).

(In reply to Ekanan Ketunuti from comment #1)

You want to reset network.trr.mode to eliminate symptom.

Does safe mode still use network.trr.mode? I was crashing on launch even when launching in safe mode. I had to manually edit my profile's prefs.js file and add user_pref("network.trr.mode", 0); to undo the pref.

Comment 4

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1556911 - Don't instantiate NetworkConnectivityService off-main-thread r=dragana,mt

This patch calls NetworkConnectivityService::GetSingleton() on the main thread
and keeps a ref to the service until shutdown.
Even though calling ncs->GetIPv6() off-main-thread is technically a data-race
in practice that's OK because only the simple decision whether to send
AAAA requests is made based on that value, which in itself is an optimization.
I filed bug 1556967 for making the connectivity service thread safe.

Comment 5

[Valentin Gosu [:valentin] (he/him)]

(In reply to Chris Peterson [:cpeterson] from comment #2)

Does safe mode still use network.trr.mode? I was crashing on launch even when launching in safe mode. I had to manually edit my profile's prefs.js file and add user_pref("network.trr.mode", 0); to undo the pref.

I feel like it should. In which component should I file the bug?

Comment 6

[Ekanan Ketunuti]

IIUC, safe mode has no effect on mod-prefs.

Comment 7

[Martin Thomson [:mt:]]

FWIW, I had to do the same as Chris.

As for the proposed change: https://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong

Comment 9

[Val Packett]

I'm seeing a NetworkConnectivityService::GetSingleton() crash, but so far only when visiting https://blog.digitalocean.com/an-update-on-last-weeks-customer-shutdown-incident/ specifically :D

Comment 10

[alex_mayorga]

¡Hola!

Yup! Happened to me also on Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0 ID:20190605095225:

bp-e32cb003-eac4-4c1c-a9cc-d1d920190605 6/5/2019, 8:27 AM

The 2nd start after the startup crash had no issue though.

The only modified about:config is:

network.trr.mode 2

¡Gracias!
Alex

Comment 11

[Valentin Gosu [:valentin] (he/him)]

(In reply to Martin Thomson [:mt:] from comment #7)

As for the proposed change: https://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong

Agreed. It's just that making the connectivity service thread safe would have taken longer than it took me to write up this patch.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0d39f4b9fe60
Don't instantiate NetworkConnectivityService off-main-thread r=dragana

Comment 13

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/0d39f4b9fe60

Comment 14

[alex_mayorga]

¡Hola!

The crash no longer occurs on Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0 ID:20190609090758

¡Gracias!
Alex

Comment 16

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

This bug has been identified as part of a pilot on determining root causes of blocking and dot release drivers.

It needs a root-cause set for it. Please see the list at https://docs.google.com/document/d/1FFEGsmoU8T0N8R9kk-MXWptOPtXXXRRIe4vQo3_HgMw/.

Add the root cause as a whiteboard tag in the form [rca - <cause> ] and remove the rca-needed keyword.

If you have questions, please contact :tmaity.