Open Bug 1443925 Opened 3 years ago Updated 11 days ago

Make nsIPrincipal objects threadsafe

Categories

(Core :: DOM: Security, enhancement, P3)

enhancement

Tracking

()

People

(Reporter: nika, Unassigned)

References

(Depends on 2 open bugs)

Details

(Whiteboard: [domsecurity-backlog1])

Attachments

(5 files)

Once we've fixed bug 922464, which Valentin is working on, we should be able to look into the possibility of allowing sending nsIPrincipal objects across threads. There are some problems with this, but this is largely intended as a place where we can hash out ideas which would help us work around them.

Currently nsIPrincipal objects are already largely immutable. There are only a small set of things which are mutable:

1. SetDomain
At any point in time, a document on the web can update its document.domain property, which can change how the principal is treated for the purposes of _some_ origin comparisons.

When this value is set, a call into the JS engine is made to recompute all of the cross-compartment wrappers for the given principal, as they may have changed. In addition, the actual NodePrincipal() of the document has been changed internally.

I'd like to make this immutable if possible, as we currently serialize it, and it seems silly not to. I think we'd do this by:
a) Replacing the document's mNodeInfoManager's mPrincipal with a new one with the domain set, and
b) Calling into the JS engine, both recomputing the CCWs, and rewriting the compartment principals.

I'm not sure how hard the latter would be to implement.

This would leave any code which stashes the NodePrincipal with the old principal. However, every single place where we read this data right now that I can find can be traced back to a read of either the NodePrincipal() of the document, or a compartment's principal, so I think that will be fine.
 (Of course, I'd want to do an audit of our code looking for situations where this might not be true).

2. SetCSP, EnsureCSP, anything CSP

I have no idea if/how we could make this immutable. IIRC this is mutated during loading & parsing of a document, as well as while reading the header from the network, so I don't know what we'd do to make it immutable.

One option would be to move this state off of the nsIPrincipal and onto the nsIDocument or Worker which the CSP is attached to. We'd then need to have some way to get at this information from the compartment other than going through the nsIPrincipal object.

We could also go the same direction as SetDomain, but this seems more common and I would be nervous about adding this much overhead to parsing CSP <meta> tags.

A final option would be to make this property effectively main-thread-only. Attempts to read it OMT would MOZ_CRASH, and it could remain mutable. This does have the unfortunate effect of meaning it would be impossible to serialize a nsIPrincipal OMT, as we currently serialize the CSP data when sending it over IPC. An option would be making OMT nsIPrincipal serialization simply save a null CSP, which is probably correct-enough for all cases where we use it.

This also has the disadvantage that IIRC Workers would love to be able to access CSP data, so making it OMT readable there would also be nice.

Not sure what the best final solution here is.
I forgot one last option: Allow nsIPrincipal objects to be created on any thread etc, but don't allow passing a reference to an nsIPrincipal across threads. If you want to transfer one, you'll have to use IPDL to serialize & deserialize the information on a new thread.

We'd need to add a (debug only?) owning thread flag to the nsIPrincipal object to validate this, but other than that this would be by far the easiest option.
ni? from some people who I have a feeling will have opinions on this / ideas here.
Flags: needinfo?(ehsan)
Flags: needinfo?(bugs)
Flags: needinfo?(bkelly)
Do we want principal to be threadsafe, or easy way to clone a principal to be used in another thread or something like that.
I'm just thinking about the cases when a principal actually is modified.
How does all the broadcast channel communication and such is supposed to work in case document.domain is modified in web page, but worker still uses the initial principal.
(would need to read spec here)

But yes, being able to use principals easily in non-main-thread would be great.
Flags: needinfo?(bugs)
FWIW, we are about to get the CSP off the principal and into the loadinfo (see Bug 965637).
Priority: -- → P3
Whiteboard: [domsecurity-backlog1]
(In reply to Christoph Kerschbaumer [:ckerschb] from comment #4)
> FWIW, we are about to get the CSP off the principal and into the loadinfo
> (see Bug 965637).

Awesome! I'll mark that bug as a blocker.

(In reply to Olli Pettay [:smaug] from comment #3)
> Do we want principal to be threadsafe, or easy way to clone a principal to
> be used in another thread or something like that.
> I'm just thinking about the cases when a principal actually is modified.
> How does all the broadcast channel communication and such is supposed to
> work in case document.domain is modified in web page, but worker still uses
> the initial principal.
> (would need to read spec here)
> 
> But yes, being able to use principals easily in non-main-thread would be
> great.

nsIPrincipal is actually immutable right now with the exception of document.domain (and CSP which is moving off the object). In addition, document.domain is basically only used by CCWs and some methods on nsGlobalWindow related objects:

---

Here's a regex to find every consumer:
https://searchfox.org/mozilla-central/search?q=%5BSs%5DubsumesConsideringDomain%7CGetDomain%7C%5BEe%5DqualsConsideringDomain&case=true&regexp=true&path=

https://searchfox.org/mozilla-central/rev/99df6dad057b94c2ac4d8b12f7b0b3a7fdf25d04/caps/nsScriptSecurityManager.cpp#143
 - Only called in nsScriptSecurityManager::CanCreateWrapper in order to print an access denied error.

https://searchfox.org/mozilla-central/rev/99df6dad057b94c2ac4d8b12f7b0b3a7fdf25d04/caps/nsScriptSecurityManager.cpp#481
 - Only used by gfxFontSrcPrincipal to get a precomputed hash value for its internal principal to make hashing faster. Could be changed to use a different hashing mechanism, or a hashing mechanism which doesn't take the domain into account.

https://searchfox.org/mozilla-central/rev/99df6dad057b94c2ac4d8b12f7b0b3a7fdf25d04/dom/base/Location.cpp#997
 - Location::CallerSubsumes - Is used to guard calls into Location methods which should not be avaliable cross-origin. This should be able to check the domain somewhere else if we need it to, such as the caller's global or compartment.

https://searchfox.org/mozilla-central/rev/99df6dad057b94c2ac4d8b12f7b0b3a7fdf25d04/dom/base/nsGlobalWindowOuter.cpp#6279
 - nsGlobalWindowOuter::GetFrameElementOuter - used to guard a cross-origin getter and return null in the cross-origin call. Can easily access the document's state.

https://searchfox.org/mozilla-central/rev/99df6dad057b94c2ac4d8b12f7b0b3a7fdf25d04/dom/base/nsObjectLoadingContent.cpp#3543
 - nsObjectLoadingContent::GetContentDocument - used to guard a cross-origin getter and return null. Can easily access the document's state.

https://searchfox.org/mozilla-central/rev/99df6dad057b94c2ac4d8b12f7b0b3a7fdf25d04/dom/bindings/CallbackObject.cpp#283
 - CallbackObject::CallSetup::ShouldRethrowException - used to check if we should throw exceptions across compartments. We can probably check something other than the principal here.

https://searchfox.org/mozilla-central/rev/99df6dad057b94c2ac4d8b12f7b0b3a7fdf25d04/dom/html/nsGenericHTMLFrameElement.cpp#108
 - same as nsObjectLoadingContent::GetContentDocument - checking if cross-origin to return nullptr.

https://searchfox.org/mozilla-central/rev/a70da6775d5341a9cca86bf1572a5e3534909153/dom/html/nsHTMLDocument.cpp#886
 - document.domain getter.

https://searchfox.org/mozilla-central/rev/99df6dad057b94c2ac4d8b12f7b0b3a7fdf25d04/dom/script/ScriptSettings.cpp#198
 - Used to clamp principals. 
   The comment above suggests that the clamping case can never happen on the web, and the web is the only place we use document.domain - so TBH we could probably stop checking that here.
   We could also just check the info from another source. We do have globals around & such.

https://searchfox.org/mozilla-central/rev/99df6dad057b94c2ac4d8b12f7b0b3a7fdf25d04/gfx/thebes/gfxUserFontSet.cpp#1548
 - Debug code.

https://searchfox.org/mozilla-central/rev/99df6dad057b94c2ac4d8b12f7b0b3a7fdf25d04/js/xpconnect/wrappers/AccessCheck.cpp#67
https://searchfox.org/mozilla-central/rev/99df6dad057b94c2ac4d8b12f7b0b3a7fdf25d04/js/xpconnect/wrappers/AccessCheck.cpp#77
 - These are called in xpc::ShouldWaiveXray, xpc::WrapperFactory:Rewrap, and DEBUG_CheckUnwrapSafely.
 - Whether the first or second is called is flipped based on OriginAttributes::IsRestrictOpenerAccessForFPI()
 - These methods have access to the whole compartment, so we can almost certainly get the domain from somewhere else.
 - These are only used for CCW handling.

https://searchfox.org/mozilla-central/rev/99df6dad057b94c2ac4d8b12f7b0b3a7fdf25d04/caps/tests/unit/test_origin.js#13
 - A test file which could easily be changed or removed - just tests that the functions work.

----

So, yes. I'm suggesting that principals become fully immutable and sharable between threads. We might even want to start interning them based on their origin string (which is already used for principal equality checks etc.) to make comparing (and even perhaps constructing them?) even cheaper.

I'm mildly tempted to try to move principals over to MozURL to get them threadsafe now, but it's probably more advisable to wait for the nsIURI threadsafety changes.
Depends on: 965637
(In reply to Nika Layzell [:mystor] from comment #5)
> nsIPrincipal is actually immutable right now with the exception of
> document.domain (and CSP which is moving off the object). In addition,
> document.domain is basically only used by CCWs and some methods on
> nsGlobalWindow related objects:

I think the main reason document.domain lives on the principal and not elsewhere is so that it can be inherited correctly in various contexts. We need to have a solution to that to move document.domain elsewhere.

> So, yes. I'm suggesting that principals become fully immutable and sharable
> between threads.

This would be an awesome thing to get to.

> We might even want to start interning them based on their
> origin string (which is already used for principal equality checks etc.) to
> make comparing (and even perhaps constructing them?) even cheaper.

That could require some extra complexity/locking if we want to make them also constructable on arbitrary threads. Probably not an issue but we should weigh it against the benefits given that principal comparisons are a lot cheaper these days.
Note that right now if you set document.domain in a thing that inherited a principal, that also sets it for the place the principal was inherited from (and vice versa).

Also note that you need something like this to have sane behavior: otherwise if you create an about:blank iframe and then set your own document.domain you can no longer access the iframe.

All of which is to say that document.domain needs to mutate some bit of state that is shared across multiple documents.  We used principal for that, but we could conceivably use another object.

Another option is to break up principal into two things (call them MutablePrincipal and ImmutablePrincipal).  The former is what documents hold and gets inherited, etc.  The latter is what's used for security checks and can be threadsafe...  We can use composition or inheritance there, either way.
Yeah, moving the guts of nsIPrincipal into a new immutable inner object sounds like a great plan.
I would be happy to have this.  We have a lot of code using PrincipalInfo simply because it needs to run off-main-thread.  It would be much nicer to just have access to nsIPrincipal and its related security code off-main-thread.

I do think we definitely want to wait for nsIURI thread-safety to land and stabilize first, though.  A lot of nsIPrincipal code also needs to work with nsIURI, so we may not see much benefit without that fix as well anyway.  Valentin seems close to completing OMT nsIURI.

I guess I just always assumed this was the next thing after OMT nsIURI.
Depends on: OMT-nsIURI
Flags: needinfo?(bkelly)
Depends on: 1528272
Depends on: 1532253
Flags: needinfo?(ehsan)
See Also: → 1619584

Just FYI.
We introduced TRRServiceChannel, which can be used OMT for fetching the DNS data. Since LoadInfo is main thread only (because of this bug), we also introduce another TRRLoadInfo for TRRServiceChannel. TRRLoadInfo only implements a few interface of nsILoadInfo, since most of members in LoadInfo are not thread-safe for now.
When this bug is fixed, we can try to make LoadInfo thread-safe and ditch TRRLoadInfo.

I looked into this a bit again recently, and here are a few of my current thoughts:

Mutable State

There are a few pieces of mutable state (after Init()) on principal objects, which makes them not fully immutable. In order to get this working sooner-rather-than-later, we probably can't eliminate all of these pieces of mutable state, but we can at least make them threadsafe:

  • For nsIContentSecurityPolicy on ExpandedPrincipal it should be possible to instead store a DataMutex<nsMainThreadPtrHandle<nsIContentSecurityPolicy>> which would avoid most of the issues there. It wouldn't be settable from OMT, but things like cloning the principal with different OriginAttributes would be fine. This would let us avoid waiting for CSP to be moved off of the type.
  • The nsCOMPtr<nsIURI> mDomain on ContentPrincipal could be also guarded by a DataMutex<...> without too much difficulty, with bool mHasExplicitDomain changed to a std::atomic<bool>. However, that could potentially have a performance impact. I think the impact could be minimized by checking the mHasExplicitDomain bool before even trying to acquire the mutex to make the checks more efficient. Alternatively, we could make the field unguarded and main-thread accessible only, but that feels less obviously safe and means some methods won't be able to work OMT.
  • For WeakPtr<WebExtensionPolicy> on ContentPrincipal, it is a lazily initialized field which is created whenever AddonPolicy() is called. I think one of the easier ways to handle it would be to have a reference counted type using NS_INLINE_DECL_THREADSAFE_REFCOUNTING_WITH_DELETE_ON_MAIN_THREAD which contains the WeakPtr<WebExtensionPolicy> and store a pointer to that instead (perhaps behind a mutex to be safe, even if the field is only ever touched on the main thread or in the destructor, it can of course be optimized if needed)

In general, I think these pieces of mutable state are workable. Many of them could be optimized further if they end up being performance issues, but I think that wrapping specific fields in DataMutex and marking the other fields explicitly as const is probably our best bet for getting an initial threadsafe principal type.

Reference Counting

The reference counting for principal types is special, as the refcount field is inherited from JSPrincipals. However, this field is already atomic (https://searchfox.org/mozilla-central/rev/e3c34d34cc7dce56df377f05183ad379ac1de123/js/public/Principals.h#25), so making reference counting atomic for principals should only require removing the assertions from nsJSPrincipals.cpp (https://searchfox.org/mozilla-central/rev/e3c34d34cc7dce56df377f05183ad379ac1de123/caps/nsJSPrincipals.cpp#27,36)

Init Methods

Currently, principals are generally created with a default constructor and then initialized by calling an init method of some form, which ends in calling BasePrincipal::FinishInit. The principal isn't usable until this method is called. This is a fairly simple change in the majority of cases, as the distance between the constructor & init calls aren't very wide, and the Init methods are generally not exposed on nsIPrincipal. The default constructors will need to be removed and replaced with explicit constructors which take arguments similar to the init methods (although some may need to be made private so that the more ergonomic init API can be exposed as a static ::Create method or similar).

The main issue here is likely to be the remaining uses of principals through the component system, which can all be dealt with in one way or another:

  1. Instances which create a null principal with "@mozilla.org/nullprincipal;1" can be switched to use Services.scriptSecurityManager in JS or invoking NullPrincipal::CreateWithoutOriginAttributes in C++.
  2. The System Principal can remain a singleton but with the init method removed, and the relevant initialization instead happening in the constructor. It can still be fetched using the service manager
  3. Other service types are not accessed directly as components through the service manager, so should be able to have their contract IDs removed (though not their CIDs because of nsISerializable).

nsISerializable

The last issue with the component manager is that every nsIPrincipal implements nsISerializable. There is no longer support for writing out new serialized forms of these types with nsISerializable::Write() (they're all stubbed out now), but they can still be read with ::Read(). This will probably need to be solved in a similar way to how it was for nsIURI, where the CID is changed to map to a different object (e.g. ContentPrincipal::Deserializer) which is non-threadsafe & implements nsISerialzable, and when QI-ed to nsIPrincipal will return the constructed principal from the read call.

The only type which won't need this change is SystemPrincipal whose Read method is already a no-op, so it can continue to directly implement the interface.

Determining which methods are Threadsafe

Once all of these are changed, I think that the core principal object will be mostly threadsafe at rest, though many methods will still be main-thread only. We'll need to go through and check which ones are using main-thread-only services etc. From a quick scan, many methods will not work off-main-thread, and will need to be documented as such.

  • Many methods access AddonPolicy() (even if just to null-check it) which is main-thread-only due to WebExtensionPolicy being cycle-collected. These methods won't work off-main-thread and some important ones may need to be changed.
  • Many methods use various global components which either are not or may not be fully threadsafe, such as: nsIIOService/nsINetUtil, nsIEffectiveTLDService, mozIThirdPartyUtil, BlobURLProtocolHandler, nsScriptSecurityManager, NSPR, and the various nsIProtocolHandler implementations.
  • Finally some methods will just never work of-main-thread as they intentionally call into other components for specific purposes, such as ContentPrincipal::SetDomain which calls into the JSAPI to update compartment wrappers etc.

These include some pretty important methods, like ContentPrincipal::GenerateOriginNoSuffixFromURI, which might make a threadsafe nsIPrincipal somewhat unusable off-main-thread. I imagine that at a minimum we'd want to support the relevant APIs for creating principals.

If we do end up doing this change, we'll probably want to start by annotating basically every method as being main-thread-only, adding a MOZ_ASSERT(NS_IsMainThread()) to it, and then iteratively remove these assertions as we determine that a specific method is actually safe to call off-main-thread.

Depends on: 1711078

I think this might be one of the more invasive changes which are required for
making nsIPrincipal threadsafe, as it requires somewhat fundamentally changing
how we hold a reference to the extension policy in ContentPrincipal. I chose to
use an explicit tearoff type here rather than something else as some properties
of AddonPolicy may need to be used off-main-thread as well in the future, and
this provides a better starting point for changes like that, as specific fields
can be duplicated onto the tearoff.

The actual weak reference needs to be threadsafe so that when the nsIPrincipal
is destroyed, it can drop the WeakPtr on the correct thread.

nsIContentSecurityPolicy is not threadsafe refcounted, so in order to make it
safe to potentially drop the last reference to this type off-main-thread, we
need to store it within a nsMainThreadPtrHandle.

Depends on D118762

This is a way to make the mutable state in nsIPrincipal explicitly threadafe.
In some cases, it would be possible to use more specialized data structures or
otherwise avoid mutexes, but for simplicity it seems better to avoid that
unless mutex overhead appears in profiles.

Depends on D118763

This marks various methods which appear to not be comptible with being used
off-main-thread. This is mostly because they either use particular global
services which are not threadsafe (such as nsScriptSecurityManager,
mozIThirdPartyUtil and nsIEffectiveTLDService). This excludes some fairly
core methods such as CreateContentPrincipal, Subsumes, and GetSiteOrigin,
however I'm hopeful that this can be improved in the future as these methods
are needed and are migrated to work off-main-thread.

Depends on D118764

The refcount field is already atomic and provided by JSPrincipals, and
this change allows these types to be manipulated from off-main-thread.

Depends on D118765

I've attached my early-draft patches which I wrote a while ago for getting nsIPrincipal to be threadsafe without touching any of the services it uses. I don't currently have any plans for landing these as-is, and I think there are probably some other issues which we'd want to clean up first, but this might work as a framework for someone who comes after this.

You need to log in before you can comment on or make changes to this bug.