1610136 - Avoid blocking shutdown to persist DataStorage values

Status: NEW

(Core :: Security: PSM, enhancement, P2)

Description

[Alex Thayer [:alexical] (she/her)]

Currently, it looks like we wait for five minutes since the last write to DataStorage to persist the data to disk. I suspect this means that we're very frequently doing the write during shutdown. It's already occurring on a background thread - what would it take to get this timeout lower / could we get it low enough so that we could just skip it on shutdown? What are the implications of not persisting this when, say, the browser crashes or the power goes out?

Comment 1

[Dana Keeler (she/her) (use needinfo) [:keeler]]

HSTS is a trust-on-first-use mechanism, so the implications of not writing out pending data at shutdown would be potentially missing that "first use" (or, in the case of expiry, "first use in a while") and defeating the entire point. I imagine we could lower the timeout. We could also probably be a bit smarter about determining when the data is dirty (i.e. whether or not we have changes to write out). If a site sends the strict-transport-security header with every response, the implementation may repeatedly consider that to be new data that has to be written out, when we could instead do some sort of coalescing and only consider the first header to be new and any subsequent headers that occur in some period of time to be equivalent to it.

All that said, I would gather some data on how often this blocks shutdown before investing more effort into avoiding it (otherwise, how would we know our efforts were effective?)

Comment 2

[Alex Thayer [:alexical] (she/her)]

We fail to gracefully shutdown something like 6.5% (according to the SHUTDOWN_OK probe). It looks like we don't atomically write this file, meaning if we get halfway through writing it we have a corrupt file and have invalidated everything in what used to be there, yes? This seems like it's likely a problem.

(Not diminishing your suggestions - I am working on a solution for collecting information on these sorts of blockers - I just noticed this and it seemed worth calling out.)

Comment 3

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Yes, we could lose data if interrupted while writing. The file is in a newline-delimited format and malformed lines are discarded when reading, so if we only write it halfway, we lose every entry from the unfinished line to the end.

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.