1108971 - Kerberos authentication does not work with alias

Status: RESOLVED FIXED

(Core :: Networking, defect)

Description

[Bart de Kort]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0
Build ID: 20141126041045

Steps to reproduce:

I tried to log on to websites using kerberos in Firefox beta 35.0.

Probably similar to https://bugzilla.mozilla.org/show_bug.cgi?id=804605


Actual results:

It fails when I use an alias (probably a CNAME, but I am not that good with terminology). It is succesful when I use the fully qualified hostname. Note that Kerberos was set up properly and worked fine where required until this build.

I am back on the release channel. Downgrading Firefox has resolved the issue for me.


Expected results:

I should have logged in on the (company/intranet) website.

Comment 1

[Honza Bambas (:mayhemer)]

Andrew, just if you would be interested..

Comment 2

[Patrick McManus [:mcmanus]]

going to need https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging

Comment 3

[Patrick McManus [:mcmanus]]

andrew, can you let us know if you can take a look at this or not? Thanks!

Comment 4

[Andrew Bartlett]

Yes, I'm very happy to comment.  

As a first and simple diagnostic, can you verify that IE behaves any differently?  You may have to use a network monitor to confirm it isn't falling back to NTLM.

In general, access to a CNAME of a kerberos host that isn't registered as a servicePrincipalName (essentially an alias) in AD, or not registered otherwise (more complex, I would add a second principal with the same keys for simplicity) with the MIT KDC, requires steps (a reverse DNS lookup) that open up security attack vectors, slowness and unpredictability.  

The security weakness applies much more to protocols that leverage the mutual authentication aspects of Kerberos, I need to think harder as to if there is a practial value on HTTP.  Encrypted IMAP or SMTP might be more relevent.  In any case, the weakness is that you no longer have a direct mapping between the name you asked the access to, and the name you got the ticket from the KDC for. 

Sadly in MIT domains it is/was often used, with a setting of 'rdns' being set in krb5.conf files on Linux by default.  However, in AD domains, it is always exactly the wrong thing to do.  (Working reverse DNS is also quite rarely correct in an AD domain, because Windows essentially never uses it). 

Indeed, when doing other work in firefox for NTLM recently, I saw the code for bug #804605 and was saddened.  I think this much belongs in the Kerberos libs, and controlled globally by the krb5.conf, not additionally in each application.

Comment 5

[Bart de Kort]

Attachment: log.txt: (new profile; issue occurs when navigating to "intranet". Yes, the website, not the google search.)

Added log.

Comment 6

[Bart de Kort]

Comment on attachment 8534956 [details]
log.txt: (new profile; issue occurs when navigating to "intranet". Yes, the website, not the google search.)

NM, I hadn't set up my trusts yet. in these logs. Attaching new copy.

Comment 7

[Bart de Kort]

Attachment: log.zip: new copy with trust set up. Failure to log in using Kerberos on "intranet". Succes on "test-tophelp".

New logs after actually trusting the intranet domains.

Comment 8

[James Abbatiello]

Since 35.0 has hit the release channel, this problem is now affecting me (and likely other people).

I've bisected the problem to:
Last good revision: 5bd6e09f074e
First bad revision: afc933adf723
Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=5bd6e09f074e&tochange=afc933adf723

Within that range https://hg.mozilla.org/mozilla-central/rev/4d21e0728967 seems most likely to me.

With NSPR_LOG_MODULES=negotiateauth:5, I get the following results:
====== Good ======
  service = friendly.example.com
  using negotiate-sspi
  nsAuthSSPI::Init
  InitSSPI
Using SPN of [HTTP/canonical.example.com]
AcquireCredentialsHandle() succeeded.
nsHttpNegotiateAuth::GenerateCredentials() [challenge=Negotiate]
entering nsAuthSSPI::GetNextToken()
InitializeSecurityContext: continue.
  Sending a token of length 2041
====== End ======

====== Bad ======
  service = friendly.example.com
  using negotiate-sspi
  nsAuthSSPI::Init
  InitSSPI
Using SPN of [HTTP/friendly.example.com]
AcquireCredentialsHandle() succeeded.
nsHttpNegotiateAuth::GenerateCredentials() [challenge=Negotiate]
entering nsAuthSSPI::GetNextToken()
InitializeSecurityContext: continue.
  Sending a token of length 40
====== End ======

The SPN does not have the canonical name like it does in the working version.

See also Bug #857291 which is where the fix was implemented last time this problem came up.

Comment 9

[James Abbatiello]

Running the nightly with build ID 20140923030204 and setting dns.ttl-experiment.variant to 2 or 3 produces a bit of improvement.  The first auth attempt still fails but reloading the page then succeeds.  Setting dns.ttl-experiment.variant=1 or dns.ttl-experiment.enabled=false yields the same broken behavior as in 35.0.  This seems like strong evidence that https://hg.mozilla.org/mozilla-central/rev/4d21e0728967 for Bug #1067679 is in fact responsible.

The config tweak is not effective on 35.0.  I'm assuming that the relevant code has been compiled out of the release builds.

Since this is stopping me from using intranet sites that I need for work, I'd like to get this fixed quickly if possible.  Last time this broke the fix was part of the next x.0.1 release.  Can anyone advise me on what to do next?  Is backing out the above change for 35.0.1 possible?  At the very least can someone change the status from UNCONFIRMED?

Comment 11

[Dragana Damjanovic [:dragana]]

I found the error:

at http://mxr.mozilla.org/mozilla-central/source/netwerk/dns/nsHostResolver.cpp#1403

GetAddrInfo is called with flags being PR_AI_ADDRCONFIG but inside GetAddrInfo and _GetAddrInfo_Portable they are treaded as nsHostResolver flags:
int prFlags = PR_AI_ADDRCONFIG;
if (!(aFlags & nsHostResolver::RES_CANON_NAME)) {
    prFlags |= PR_AI_NOCANONNAME;
}

therefore the flags are unset.

The patch is coming...

Comment 12

[Dragana Damjanovic [:dragana]]

Attachment: bug_1108971_v1.patch

Comment 13

[:Gijs (out for now; he/him)]

Dragana, is this the same problem as bug 1121843 ?

Comment 14

[Dragana Damjanovic [:dragana]]

According to comment 4 in mug 1121461, that bug is the same as this one.

Probably the 1121843 is also the same.

Comment 16

[Burghardt]

Dragana - please excuse my ignorance. But how do we proceed? How do I get the problem in my stable version 35.0 released?

Comment 17

[Dragana Damjanovic [:dragana]]

There must be a new release to fix this problem in 35.0. But I do not know when.

Comment 18

[Emanuel Hoogeveen [:ehoogeveen]]

Once this patch is reviewed (and has successfully run the automated tests on tryserver), it will be landed on mozilla-inbound, and then mozilla-central (currently 38). It then needs to be nominated for 'uplift' to earlier releases. Depending on the complexity of the fix and the severity of the problem, it may be accepted on aurora (currently 37) or even beta (currently 36).

If an unacceptable amount of people are affected, it may be grounds for a point release (35.0.1), or if a point release is already planned it may be taken for that as well. However, the risk/reward tradeoff goes up for each stage, so they may not take it.

Comment 19

[Florian]

Hello,

we also have this bug. Please fix it as soon as possible because this problem exists on most of our internal sites.

Thanks.

Comment 20

[Dragana Damjanovic [:dragana]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=087449621dfa

Comment 21

[Patrick McManus [:mcmanus]]

thanks Dragana.. looks like this suffered some kind of merge conflict - maybe in the backout of 820391.

I'd like steve to do the review

Comment 22

[Steve Workman [:sworkman] (INACTIVE)]

Comment on attachment 8549467 [details] [diff] [review]
bug_1108971_v1.patch

Review of attachment 8549467 [details] [diff] [review]:
-----------------------------------------------------------------

Good catch. r=me.

Comment 25

[j.zimmermann]

> If an unacceptable amount of people are affected, it may be grounds for a
> point release (35.0.1), or if a point release is already planned it may be
> taken for that as well. However, the risk/reward tradeoff goes up for each
> stage, so they may not take it.

point release would be great. having our developers and marketing folks use firefox 34 for a couple of weeks is not exactly an exciting prospect...(In reply to Emanuel Hoogeveen [:ehoogeveen] from comment #18)

Comment 26

[Mark Straver]

(In reply to j.zimmermann from comment #25)
> > If an unacceptable amount of people are affected, it may be grounds for a
> > point release (35.0.1), or if a point release is already planned it may be
> > taken for that as well. However, the risk/reward tradeoff goes up for each
> > stage, so they may not take it.
> 
> point release would be great. having our developers and marketing folks use
> firefox 34 for a couple of weeks is not exactly an exciting prospect...(In
> reply to Emanuel Hoogeveen [:ehoogeveen] from comment #18)

Looking at the general activity in this and duped bugs, I'd say a lot of people are affected - a point release is probably a very good idea to fix this ASAP for many corporate users.

Comment 27

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/d5092aaecfca

Comment 29

[Loic]

Added tracking for FF35 due to corporate users if a chemspill version is released.

Comment 30

[Carsten Wendland]

Hi,

consider that proxy authentication with kerberos is broken too.
So if you don't have a fallback auth on your proxy this bug shuts down internet with firefox.

+1 point release

Regards Carsten

Comment 31

[Lukas Blakk [:lsblakk] use ?needinfo]

Will track and keep this in our queue in case we do a chemspill or dot release for 35.

Comment 32

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/d5092aaecfca

Comment 35

[Kohei Yoshino]

https://developer.mozilla.org/en-US/Firefox/Releases/35/Site_Compatibility#Networking
https://twitter.com/FxSiteCompat/status/557243789711269890

Comment 36

[Lukas Blakk [:lsblakk] use ?needinfo]

Please nominate for uplift to Aurora/Beta/Release branches so we are prepared in the case of a dot release.

Comment 37

[Dragana Damjanovic [:dragana]]

Comment on attachment 8549467 [details] [diff] [review]
bug_1108971_v1.patch

Approval Request Comment
[Feature/regressing bug #]: Bug 1067679
[User impact if declined]:Everything using network alias does not work, someproblems with authentication that use aliases.
[Describe test coverage new/current, TBPL]: comments 10, 12, 13 and 14 from bug 1121843
[Risks and why]: no risk, it revert to behavior that firefox had before bug 1067679. 
[String/UUID change made/needed]: none

Comment 38

[jasonmc]

Another +1 for a point release on this. This is a major break in functionality for just about anyone in a corporate or university environment.

Comment 39

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/f3ddede8bc33
https://hg.mozilla.org/releases/mozilla-beta/rev/2dbbd7362502

Comment 40

[Anton Lavrentiev]

Please make a patch release ASAP.  Could not work with ver 35 at all (it annoyingly kept asking for
the passwords here and there).  Had to downgrade to 34.0.5 (which works fine!), but now it constantly
wants to upgrade to 35.

Comment 41

[Kohei Yoshino]

Enterprise users may want to use Firefox 31 ESR instead:
https://www.mozilla.org/en-US/firefox/organizations/

Comment 42

[Patrick McManus [:mcmanus]]

(In reply to Anton Lavrentiev from comment #40)
> Please make a patch release ASAP.  Could not work with ver 35 at all (it
> annoyingly kept asking for
> the passwords here and there).  Had to downgrade to 34.0.5 (which works
> fine!), but now it constantly
> wants to upgrade to 35.

can you verify the latest nightly to make sure it resolves your problem?

I'm concerned that with your comment about "here and there" you are running into a different problem (bug 1121895)

Comment 43

[Anton Lavrentiev]

"here and there" was to underscore that there're many resources I needed to access all of which suddenly stopped to let me in without the popup for credentials.

Comment 44

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8549467 [details] [diff] [review]
bug_1108971_v1.patch

Taking it for a potential 35.0.1

Comment 45

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-release/rev/de1629707dbf

Comment 46

[[SV Manager] Florin Mezei, QA (:FlorinMezei)]

Setting as qe-verify- since the fix was verified in bug 1121843 (comments: 10, 13, 14).

Comment 48

[Michael Kesper]

Works again for me with 35.0.1, thanks to all making this possible.
Will there be regression tests for this bug?

Comment 49

[Burghardt]

I can not confirm unfortunately. For me, it still doesn't work (comment 48). Firefox still asks for a user name - despite of SSO.

Comment 50

[:Gijs (out for now; he/him)]

(In reply to Burghardt from comment #49)
> I can not confirm unfortunately. For me, it still doesn't work (comment 48).
> Firefox still asks for a user name - despite of SSO.

Are you actually using 35.0.1 yet? (check on about:support (Help > Troubleshooting information) )

If not, download from: https://www.mozilla.org/firefox/all/ .

The update was only released yesterday, and I don't know at what rate people are being updated right now. Please let us know the results of trying this.

Comment 51

[Carsten Wendland]

Hi,

I can confirm proxy authentication via kerberos / sso is working here.

Regards
Carsten

Comment 52

[j.zimmermann]

it's again working here as well. thanks a lot!

Comment 53

[Burghardt]

(In reply to Gijs Kruitbosch from comment 50)
Yes - I'm acutally using the vesion 35.0.1 :-).

Comment 54

[Anton Lavrentiev]

35.0.1 fixed the issue for me, thanks!  Also, NCBI is switching to ESR to (hopefully) avoid any such troubles in the future.

Comment 55

[Lukas Blakk [:lsblakk] use ?needinfo]

This was fixed in 35.0.1 and will be in the ESR 38 that's shipping in 8 weeks.

Comment 56

[kiran]

I am seeing this bug in beta of V40 as well

Comment 57

[:Gijs (out for now; he/him)]

(In reply to kiran from comment #56)
> I am seeing this bug in beta of V40 as well

Dragana, any idea if this regressed?

Comment 58

[Dragana Damjanovic [:dragana]]

Kiran, can you upload a log or send it to me via e-mail:
https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging

Thanks a lot!!!