Closed Bug 1253063 Opened 8 years ago Closed 8 years ago

Bugzilla Auth Delegation is not working with SNI-enabled server due to Squid proxy server

Categories

(bugzilla.mozilla.org :: API, defect)

Production
defect
Not set
normal

Tracking

()

RESOLVED WONTFIX

People

(Reporter: kohei, Assigned: dylan)

References

Details

Attachments

(3 files, 2 obsolete files)

:bwinton reported that Bugzilla Authentication Delegation didn't work for my BzDeck app and BMO, resulting in the following error:

> Auth delegation received an HTTP response other than 200 OK 
> from auth consumer. Code: 503 

For some reason the BzDeck server log shows the last request from Bugzilla was made on February 15:

> 63.245.214.162 - - [15/Feb/2016:18:41:33 -0500] "POST /integration
> /bugzilla-auth-callback/ HTTP/1.1" 200 5056 "-" "libwww-perl/5.837"

The server has been upgraded to HTTP/2 on Feb 19, but I'm not sure it's related. Can you please check Bugzila's log? Thanks.
Assignee: nobody → dylan
This is a more serious problem. the config has been overwritten with http://squid.proxy.service.consul:3128/,
which I believe to the value used for AWS. 

Is this address supposed to work from inside SCL3?
Flags: needinfo?(klibby)
Summary: Bugzilla Auth Delegation is not working for BzDeck → BMO Proxy config overwritten with AWS value
Nope, it appears that does exist in the hosts file and a manual curl works. In addition, proxying is used in github auth and that still works.
Flags: needinfo?(klibby)
Summary: BMO Proxy config overwritten with AWS value → Bugzilla Auth Delegation is not working for BzDeck
yeah, we added that in SCL3 to keep the config (more) simple for failover.
Sorry I'm late here. I have just tested this again. Downgrading to HTTP/1.1 didn't work. The access (and error) log still doesn't have any output when I accept auth delegation on https://bugzilla.mozilla.org/auth.cgi

Another possibility is...

1. the TLS cert? https://www.bzdeck.com/ uses Let's Encrypt since Oct 30, 2015. Does the Bugzilla server have the latest ca-certificates package?

2. auth delegation permission? Does Bugzilla use a whitelist or blacklist of sites (dis)allowing auth delegation?
(In reply to Kohei Yoshino [:kohei] from comment #4)

> 1. the TLS cert? https://www.bzdeck.com/ uses Let's Encrypt since Oct 30,
> 2015. Does the Bugzilla server have the latest ca-certificates package?

yes. it's ca-certificates-2015.2.6-65.0.1.el6_7, which appears to be just a few weeks old (cf https://bugzilla.redhat.com/show_bug.cgi?id=1297893 and https://bugzilla.redhat.com/show_bug.cgi?id=1299863)

And from a curl -v to https://www.bzdeck.com/ from a web head:

* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* 	subject: CN=bzdeck.com
* 	start date: Jan 18 23:38:00 2016 GMT
* 	expire date: Apr 17 23:38:00 2016 GMT
* 	common name: bzdeck.com
* 	issuer: CN=Let's Encrypt Authority X1,O=Let's Encrypt,C=US
If someone can give me more info I'm happy to go looking through logs, but :dylan may have already done so.
I can confirm that bzdeck works fine when I run it on my own server.
Attached file proxy-test.pl
squid is not able to connect to bzdeck, with the following error:

"This proxy and the remote host failed to negotiate a mutually acceptable security settings for handling your request. It is possible that the remote host does not support secure connections, or the proxy is not satisfied with the host security credentials."

Attached is a script that generates this error. Note the API key is fake.
Flags: needinfo?(klibby)
Attached file bzdeck-ciphers.txt (obsolete) —
ciphers bzdeck is using (from :ulfr's cipherscan, https://github.com/jvehent/cipherscan)
ciphers reviewboard is using (which auth delegation is able to work with)
Attached file bzdeck-ciphers-latest.txt (obsolete) —
The SSL conf was the server's default :/ Now following https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=apache-2.4.18&openssl=1.0.2g&hsts=yes&profile=modern

Also renewed the cert and rebooted Apache. Can you please test w/ squid again?

https://www.ssllabs.com/ssltest/analyze.html?d=www.bzdeck.com&hideResults=on
Attachment #8740981 - Attachment is obsolete: true
Nah, I still get a 503 error while accepting auth delegation request :(
Changed to the Generator's "intermediate profile" but the error persists.
Attachment #8741001 - Attachment is obsolete: true
Can you please consider replacing Squid with an alternative? As IE6 is dead, SNI becomes a practical solution. I can technically create a new virtual server but can't afford to pay another $5/month for now because BzDeck (nor the Firefox Site Compatibility project that shares the same server) is not a business. Thank you!
sorry, that's a considerably larger issues as the entire data center uses it for outbound traffic; the proxy isn't a BMO specific thing. /cc :digi

it may be worth figuring out if we can stand up bzdeck somewhere, but that's a managerial call ($, people).
Flags: needinfo?(klibby)
If your squid server is http://squid.proxy.service.consul:3128/ this suggests it's part of the nubis project, which is outside of my purview. If your proxy endpoint is proxy.dmz.scl3 or proxy.dmz.phx1 I can help work with you to find a solution. the nubis team is responsible for the supporting infrastructure of applications.
Sorry, I was just informed that the proxy endpoint is a red herring. Configs are identical across AWS and SCL3. Replacing squid isn't on our current roadmap, so the only alternative we can offer is for you to bypass squid and access BzDeck directly if our infosec team is ok with that.
Good news! I have just solved the issue myself. Not a new virtual server. Let's Encrypt allows us to request one single certificate for multiple domains, not only subdomains under the same domain, so I extended an existing cert to include all the domains and subdomains I have:

> ./letsencrypt-auto auth --agree-tos --apache \
> -d britegrid.com -d www.britegrid.com -d analytics.britegrid.com \
> -d fxsitecompat.com -d www.fxsitecompat.com \
> -d bzdeck.com -d www.bzdeck.com 

Then updated the server config, rebooted Apache, and problem solved!

Thank you so much :dylan and :fubar for your investigation!! Not yet FIXED by the Bugzilla side, so closing as WORKSFORME so far.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WORKSFORME
Summary: Bugzilla Auth Delegation is not working for BzDeck → Bugzilla Auth Delegation is not working with SNI-enabled server due to Squid proxy server
I mean WONTFIX.
Resolution: WORKSFORME → WONTFIX
See Also: → 1264667
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: