1494955 - credential caching of nsIHttpChannel fails for multiple connections to the same host with different users.

Status: RESOLVED WORKSFORME

(Core :: Networking: HTTP, defect, P5)

Description

[John Bieling (:TbSync)]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/17.17134

Steps to reproduce:

Create multiple nsIHttpChannels to the same host and try to authenticate these with different users.

The nsIHttpChannels is not really designed for that, because you cannot preset the desired user during creation of nsIHttpChannels, but have to somehow pick the desired username via the asyncPromptAuth callback: I look at the full spec of the associated nsIURI to pick/get username and password from PasswordManager and return them via AuthInfo.


Actual results:

However if you then look at the authenticate header send on these connections, they are messed up and use the wrong username. It looks like only one username per host can be cached and thus all connections use the same username.

If the credential caching is bypassed by adding the Authenticate header manually, everything works as expected. But of course this means all auth methods have to be re-implemented, which I would like to avoid.

I encountered this bug trying to understand a long standing bug in Thunderbird/Lightning, where users cannot subscribe to multiple calendars on the same host, but belonging to different users.



Expected results:

All Connections use the username they received via asyncPromptAuth / AuthInfo

Comment 1

[John Bieling (:TbSync)]

I continued to investigate this. It looks like opening the second and third channel to the same host will not trigger the asyncAuthPrompt callback at all, because the implementation thinks to have valid credentials and I am thus not able to specify a different username for the second and third call.

Only by forcing a 401 by changing the password on the server will trigger the asyncAuthPrompt and lets me specify a new username and a new password, which is then again used for all connections to that host. So this really looks like a caching problem.

So I would propose to allow to specify a username during setup of the nsIHttpChannel, which is then also used to lookup the cached credentials. That username should then also be passed on to the authcallbacks in the AuthInfo. If that username is not specified, there is no change in behaviour.

If that is an acceptable solution to this bug, and someone is willing to mentor me, I would like to start to work on this.

Comment 2

[Junior [inactive]]

It's by design to cache the auth info in the browser, resulting better experience IMO.
Using browser container is a good solution for separating accounts.

ni? thunderbird peer for his point of view

Comment 3

[Jorg K (CEST = GMT+2)]

Sorry, I have to pass on this one, maybe John can comment, otherwise Philipp or Magnus. Looks like the suggestion is doing something based on this: https://support.mozilla.org/en-US/kb/containers
Is there an API we can use?

Comment 4

[Valentin Gosu [:valentin] (he/him)]

I am not super sure this is the correct answer, but judging by the comments in nsIChannel, it seems what you need is nsIChannel.LOAD_EXPLICIT_CREDENTIALS

https://searchfox.org/mozilla-central/rev/1ce4e8a5601da8e744ca6eda69e782318afab54d/netwerk/base/nsIChannel.idl#285

Comment 5

[John Bieling (:TbSync)]

@Valentin: I will try that option, but it would be interesting how this works with digest auth connections, because they do need persistent information stored between calls and if all connections use the same cache (but constantly overriding) it might fail as well.

@Junior Hsu: I will try. Is the cache inside different browser containers persistent between calls? Is there some example code on how to reuse a connection used in a browser container?

Comment 6

[John Bieling (:TbSync)]

removing need info flag...

Comment 7

[Honza Bambas (:mayhemer)]

(In reply to john.bieling from comment #5)
> @Valentin: I will try that option, but it would be interesting how this
> works with digest auth connections, 

there is nothing like "digest auth connection".  digest auth is request-based and (AFAIK) doesn't have to remember anything.  only NTLM/Negotiate/Kerberos are connection based and it takes 3 rtts to establish the connection, between each attempt we do remember the state.  what you probably mean is that such an ambient authenticated connection could be reused for a request that should authenticate with a different user credentials.  we don't map connections per-provided user name and don't indent to.

containers would help, tho, as those separate connection pools.

I think the explicit credentials *may* work but I'm not sure about connection based schemas and connection reuse.

> because they do need persistent
> information stored between calls and if all connections use the same cache
> (but constantly overriding) it might fail as well.
> 
> @Junior Hsu: I will try. Is the cache inside different browser containers
> persistent between calls? Is there some example code on how to reuse a
> connection used in a browser container?

to explain a bit how this works - a container is just an id that we add to origin attributes.  origin attributes are a set of more or less opaque identifiers we simple use to separate stuff, including credential cache (I think) and connection pools.

yes, containers could work here well actually!  good suggestion, Valentin.

Comment 8

[John Bieling (:TbSync)]

:meyhemer Ah I see, thanks, I have seen that the origin is also part of the cache identifier so this could realy work. I will give it a try. 

What I was trying to say with digest auth, is that its NC value needs to be stored between calls and if the cache is reused by multiple Connections by just overwriting username/password, this might not work. But lets just wait if I can get this working using containers. 

Is there a good source to get to know containers? Otherwise I would start to look at the container addon suggested by jorg.

Comment 9

[Junior [inactive]]

Glad to know we figure that container is a good way to try.

Here's another document to know the container more
https://wiki.mozilla.org/Security/Contextual_Identity_Project/Containers

Hope you don't mind I set you as an assignee.
Feel free to ask questions here.

Comment 10

[John Bieling (:TbSync)]

So if I currently use Services.io.newChannelFromURI2 to create my channel like so

Services.io.newChannelFromURI2(
 aUri,
 null,
 Services.scriptSecurityManager.getSystemPrincipal(),
 null,
 Components.interfaces.nsILoadInfo.SEC_ALLOW_CROSS_ORIGIN_DATA_IS_NULL,
 Components.interfaces.nsIContentPolicy.TYPE_OTHER);

Than I have to add originAttributes via the Principal? By adding a trigger principal? All I found is this:
https://dxr.mozilla.org/mozilla-central/source/browser/components/originattributes/test/browser/head.js#40-43

Comment 11

[John Bieling (:TbSync)]

The following runs without error, but does not seperate the caches by users:

let channel = Services.io.newChannelFromURI2(
 aUri,
 null, 
 Services.scriptSecurityManager.createCodebasePrincipal(aUri, {userContextId: aUser}),
 null,
 Components.interfaces.nsILoadInfo.SEC_ALLOW_CROSS_ORIGIN_DATA_IS_NULL,
 Components.interfaces.nsIContentPolicy.TYPE_OTHER);

I do not find working code for nsIChannels and originAttributes. The container AddOn is using browser and fetch() and I am not able to translate that into something useful for me. Apreciate any input.

Comment 12

[Honza Bambas (:mayhemer)]

(In reply to john.bieling from comment #11)
> does not seperate the caches

which caches exactly?

Comment 13

[John Bieling (:TbSync)]

Taken from 

https://wiki.mozilla.org/Security/Contextual_Identity_Project/Containers#What_is_.28and_isn.27t.29_separated_between_Containers

it is this what I am looking for:

-----
The containers project inserts a user-controlled key into storage via OriginAttributes, which allows users to decide which state to use when interacting with a site. They can choose to send the state from their personal context or work context, and they can choose to create a new, short-lived state to browse in a certain context for a few days until a task is completed. The goal of the project is to create a very customizable experience while including a few suggested uses for users who may not want as much control. 

Users can log into multiple accounts on the same site, even when the site does not natively support concurrent sessions. Several examples include: 

- A user may wants to manage their work and personal Gmail accounts side-by-side in the same window.
- A user has a Facebook or eBay account for their business and one for their personal life.

Current solutions: 
 - Users open multiple browsers (this takes users away from Firefox).
 - A user opens one account in Private Browsing mode (this has a limit of 2 accounts, and forces one to be ephemeral).
-----

So with OriginAttributes I should be able to connect to the same host using different users without additional changes to the Firefox code base. From what I have understood, if I use a a new set of originAttributes to connect to a host, asyncopen() should trigger a call to asyncAuthPrompt and should request credentials. But with the code from comment #11, it does not. It has no effect at all, as if I had used the SystemPrincipal as in comment #10.

What am I doing wrong?

Comment 14

[Honza Bambas (:mayhemer)]

Sorry for not getting back to you, John.  I'm adding Tanvi, the author of containers.  She may help you or add people that could.

Tanvi, the goal (John, fix me if I'm wrong) is to programmatically assign a container on a channel (in C++ or chrome JS)

Thanks!

Comment 15

[John Bieling (:TbSync)]

Thanks you!

Yes, I need to use containers / origin attributes with Services.io.newChannelFromURI2 called by a (Thunderbird) AddOn (lightning to be precise), so chrome JS.

The main reason is to be able to connect to the same server using different usernames, which should be made possible by adding origin attributes.

Comment #10 contains the code currently used, Comment #11 my failed attempt to add the used username as an additional origin attribute, but from what I could see, that did not do anything.

If needed, I can point you to the follow up code, which is getting a nsIHttpChannel from the created nsIChannel and later is calling asyncopen().

Comment 16

[Tanvi Vyas[:tanvi]]

redirecting to baku, who may be able to help here.  Also, can you share your sample code?

Comment 17

[John Bieling (:TbSync)]

I would like to give this another spin, as I think originAttributes really is going to fix this, but i am not able to get it done.

I will try to explain in detail, what I want to achieve, using the following code:

createHttpChannel: function (aRequestData, aUri, aMethod, aUser, aHeaders, aNotificationCallbacks, aListener) {
    let channel = Services.io.newChannelFromURI2(
        aUri,
        null,
        Services.scriptSecurityManager.getSystemPrincipal(),        
        //Services.scriptSecurityManager.createCodebasePrincipal(aUri, {user: aUser}),
        null,
        Components.interfaces.nsILoadInfo.SEC_ALLOW_CROSS_ORIGIN_DATA_IS_NULL,
        Components.interfaces.nsIContentPolicy.TYPE_OTHER);

    let httpchannel = channel.QueryInterface(Components.interfaces.nsIHttpChannel);
    httpchannel.notificationCallbacks = aNotificationCallbacks;

    //do something with aRequestData
        
    httpchannel.requestMethod = aMethod;

    for (let header in aHeaders) {
        if (aHeaders.hasOwnProperty(header)) {
            httpchannel.setRequestHeader(header, aHeaders[header], false);
        }
    }

    httpchannel.asyncOpen(aListener, httpchannel);
}

I did not include the code handling aRequestData, for better readability.

The nature of nsIHttpChannel::asyncOpen() is to call the nsIAuthPrompt2 from the aNotificationCallbacks where it sets the aAuthInfo object with username and password. I already know the user (aUser) and look up the matching entry for host+user from the PasswordManager.

Now if I call this function a second time and the url (aUrl) points to the same host, but with a different user (aUser), I want asyncOpen() to call my nsIAuthPrompt2 again, so I can look up the correct password. However, asyncOpen checks its internal credential cache and sees, that it has valid credentials for that host and just uses the cached values, thus connecting to the server with the wrong user.

I hoped switching from getSystemPrincipal to createCodebasePrincipal and adding the user as a key, will resolve this, buit it did not.

I would be very happy, if someone could point me in the right direction.

Comment 18

[John Bieling (:TbSync)]

Oh boy, I solved it. The key must be named userContextId and it must be an integer.

Comment 19

[Honza Bambas (:mayhemer)]

(In reply to john.bieling from comment #18)

Oh boy, I solved it. The key must be named userContextId and it must be an integer.

yes, that is the property defining user context on OA for isolation. But please keep in mind there can be a set of containers (assigned persistent integer ids) that is preconfigured and configurable by the user in the browser (but not sure about mailnews).

You can freely use unallocated numbers (ids) but you must make sure you are not in conflict with any existing containers, unless that would be an intention.

Also, anything the channel persists (including cookies, cached content and possibly number of other stuff) is isolated (duplicated) - this is only to count on, not unnecessarily bad.

Comment 20

[John Bieling (:TbSync)]

(In reply to Honza Bambas (:mayhemer) from comment #19)

You can freely use unallocated numbers (ids) but you must make sure you are not in conflict with any existing containers, unless that would be an intention.

Also, anything the channel persists (including cookies, cached content and possibly number of other stuff) is isolated (duplicated) - this is only to count on, not unnecessarily bad.

For statement 1, is there a procedure to get a list of the reserved ids? Is there a specific range? Also I do not think Thunderbird is using that at all at the moment.

For statement 2, that is excactly what I want :-)