1566548 - onAuthRequired doesn't fire for HTTPS proxy requiring auth

Status: RESOLVED FIXED

(WebExtensions :: General, defect, P2)

Description

[jimmyz.z]

Attachment: onAuthRequired-never-fire.zip

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36

Steps to reproduce:

I'm writing an extension to utilize HTTPS proxy with authorization using proxy.onRequest and webRequest.onAuthRequired, but the problem is onAuthRequired never fire.

If it's a HTTPS proxy without auth, it works.
If it's a HTTP proxy with auth, onAuthRequired is fired and it works too.

The attached file is a minimized example, proxy host/port/credentials are hard coded.

HTTPS proxy server is squid, tested OK with curl and Chrome with Proxy SwitchyOmega.

BTW, Proxy SwitchyOmega has a firefox port and it behaves exactly the same, I'm not quite sure if it's the same bug though.

Comment 1

[Shane Caraveo (:mixedpuppy)]

Attachment: Bug 1566548 https proxy auth does not work with webRequest

Comment 2

[Shane Caraveo (:mixedpuppy)]

:mayhemer Any thoughts on why the attached test would fail? The authprompt code in WebRequest.jsm is never called for https requests.

Comment 3

[Honza Bambas (:mayhemer)]

The test is wrong.

For the non-secure request it works because xpcshell tests use PAC to simulate various hosts. That means that end server response may be a 407. But for secured requests (https) you need to support CONNECT handler, so the request never reaches the server. See test_http1-proxy.js and test_http2-proxy.js for how to do this. The latter is currently disabled because the node version on our test infra is too old to provide full h2 support in moz-http2.js .

As a side note, when there is no proxy configured and an end server responses with 407, it's considered a malformed response and is ignored (we never try to gather creds for obvious security reasons)

What you are trying to test here is an h2 proxy auth, right? It's known broken, see bug 1554218.

Comment 4

[Honza Bambas (:mayhemer)]

Why is this a P1, if I may ask?

Comment 5

[Shane Caraveo (:mixedpuppy)]

(In reply to Honza Bambas (:mayhemer) from comment #4)

Why is this a P1, if I may ask?

The report is that multiple proxy extensions on AMO are not receiving onAuthRequired when the proxy server is HTTPS. That's pretty broken.

I have to get the test right though to verify it.

For the non-secure request it works because xpcshell tests use PAC

BTW, this is a mochitest.

What you are trying to test here is an h2 proxy auth, right?

No, not necessarily. I presume that h2 proxies were not available in firefox at the time any of the extensions OP mentions were written.

Comment 6

[Honza Bambas (:mayhemer)]

(In reply to Shane Caraveo (:mixedpuppy) from comment #5)

(In reply to Honza Bambas (:mayhemer) from comment #4)

Why is this a P1, if I may ask?

The report is that multiple proxy extensions on AMO are not receiving onAuthRequired when the proxy server is HTTPS. That's pretty broken.

I have to get the test right though to verify it.

For the non-secure request it works because xpcshell tests use PAC

BTW, this is a mochitest.

Same for those.

What you are trying to test here is an h2 proxy auth, right?

No, not necessarily. I presume that h2 proxies were not available in firefox at the time any of the extensions OP mentions were written.

We have

• h2 (implied secured) proxy (disabled because of https://github.com/mozilla/release-services/issues/2150)
• h1 secure proxy (see https://searchfox.org/mozilla-central/rev/40e889be8ff926e32f7567957f4c316f14f6fbef/netwerk/test/unit/test_http1-proxy.js#136)
• h1 insecure (plain) proxy 407 responses can be made in the normal request handler (see https://searchfox.org/mozilla-central/rev/40e889be8ff926e32f7567957f4c316f14f6fbef/netwerk/test/unit/test_auth_proxy.js#400)

Comment 7

[Shane Caraveo (:mixedpuppy)]

• h1 secure proxy (see https://searchfox.org/mozilla-central/rev/40e889be8ff926e32f7567957f4c316f14f6fbef/netwerk/test/unit/test_http1-proxy.js#136)

Ok, I'll port this over to an extension test.

• h1 insecure (plain) proxy 407 responses can be made in the normal request handler (see https://searchfox.org/mozilla-central/rev/40e889be8ff926e32f7567957f4c316f14f6fbef/netwerk/test/unit/test_auth_proxy.js#400)

That is already done in extension tests.

Comment 8

[Shane Caraveo (:mixedpuppy)]

Attachment: Bug 1566548 add test for https onAuthRequired

A passing test for https requests over http/1 proxy

Comment 9

[Shane Caraveo (:mixedpuppy)]

Leaving open for addition of h2 proxy test once it is fixed on platform.

Comment 10

[jimmyz.z]

I'm not sure of your test environment, but from what I've seen, it looks like the return value from proxy.onRequest is hard coded type = "http", and you a testing a HTTPS request through a regular HTTP proxy, My report said the bug is for HTTPS proxy specifically, and I also said HTTP proxy does not suffer from this bug.

There is a long history of mistakenly calling a CONNECT capable HTTP proxy a HTTPS proxy, I'm not referring to that.

Comment 11

[Honza Bambas (:mayhemer)]

(In reply to jimmyz.z from comment #10)

I'm not sure of your test environment, but from what I've seen, it looks like the return value from proxy.onRequest is hard coded type = "http", and you a testing a HTTPS request through a regular HTTP proxy, My report said the bug is for HTTPS proxy specifically, and I also said HTTP proxy does not suffer from this bug.

There is a long history of mistakenly calling a CONNECT capable HTTP proxy a HTTPS proxy, I'm not referring to that.

Oh, no!! I wrote it wrong in https://bugzilla.mozilla.org/show_bug.cgi?id=1566548#c6. Yes, you are right, jimmyz. I got lost in all the combinations and forgot that the test I refer to as HTTPS/1 is actually HTTP/1-CONNECT test... Sorry for that. There is a long outstanding bug 466524, we don't have https for httpd in xpcshell tests...

Sorry, Shane to waste your time here... Anyway. The test can be easily converted for an h2 (secured) proxy, so it was not that big waste after all.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/91872ba99fcd
add test for https onAuthRequired r=mayhemer

Comment 13

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/91872ba99fcd

Comment 14

[jimmyz.z]

Some times fx will access "http://detectportal.firefox.com/success.txt" on its own, and this request will trigger onAuthRequired, and I believe fx then cached the credential and I can navigate other sites normally via the proxy.

If there's some API for extensions to trigger this portal check, that would be an acceptable workaround for me.

Comment 15

[jimmyz.z]

I believe I've found the workaround, and my report was not accurate, while accessing HTTP websites over HTTPS proxy, onAuthRequired will fire, all my previous tests were HTTPS websites over HTTPS proxy.

Comment 16

[BugBot [:suhaib / :marco/ :calixte]]

The leave-open keyword is there and there is no activity for 6 months.
:mixedpuppy, maybe it's time to close this bug?