1506520 - Referrer policy incorrectly shown

Status: NEW

(DevTools :: Netmonitor, defect, P3)

Description

[Ruben89]

Attachment: wrong_referrer_policy.png

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:65.0) Gecko/20100101 Firefox/65.0

Steps to reproduce:

Bug #1496742 introduced displaying the referrer-policy of a website in Firefox -Nightly (Firefox 65.0a1). 

At https://www.bonnieradvocaten.nl we use a same-origin referrer-policy, but this is not correctly recognized for the first response (HTML). 


Actual results:

Firefox 65 shows the referrer-policy as "no-referrer-when-downgrade". 


Expected results:

Firefox 65 should have shown the actual referrer policy for https://www.bonnieradvocaten.nl which is referrer-policy: same-origin.

Comment 1

[Jan Honza Odvarko [:Honza] (always need-info? me)]

@Heng Yeow: can you please look at this?

Honza

Comment 2

[Thomas Nguyen (:tnguyen)]

From a note in bug 1496742
Note: 
If the network.http.referer.hideOnionSource is set and the referrer is .onion ended
The "Referrer Policy" in devtool always shows "no-referrer-when-downgrade" which is default.

Comment 3

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Thomas Nguyen from comment #2)

From a note in bug 1496742
Note:
If the network.http.referer.hideOnionSource is set and the referrer is
.onion ended
The "Referrer Policy" in devtool always shows "no-referrer-when-downgrade"
which is default.

I can reproduce the bug and I have network.http.referer.hideOnionSource set to false.

@junior: do you know what's the problem here?

Honza

Comment 4

[Thomas Nguyen (:tnguyen)]

Added a simple debugging tests
[ReferrerPolicy.h:135] DEBUGGING STRING ==> 5 6 same-origin
[ReferrerPolicy.h:135] DEBUGGING STRING ==> 5 6 same-origin
[ReferrerPolicy.h:135] DEBUGGING STRING ==> 5 6 same-origin
[ReferrerPolicy.h:135] DEBUGGING STRING ==> 5 1 no-referrer-when-downgrade
[ReferrerPolicy.h:135] DEBUGGING STRING ==> 5 1 no-referrer-when-downgrade

And it is weird that only the first request got wrong referrer policy, the second one got it correctly.
Did we get a wrong channel?
https://searchfox.org/mozilla-central/rev/465dbfe030dfec7756b9b523029e90d48dd5ecce/devtools/server/actors/network-monitor/network-observer.js#499
devtool guys could have a better answer.

Comment 5

[Thomas Nguyen (:tnguyen)]

Attachment: Screen Shot 2019-01-25 at 17.32.20.png

The second request

Comment 6

[Thomas Nguyen (:tnguyen)]

Attachment: Screen Shot 2019-01-25 at 17.32.26.png

The first request

Comment 7

[Junior [inactive]]

Thomas knows more about Referrer-Policy than me.

Comment 8

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Thomas Nguyen from comment #4)

Added a simple debugging tests
[ReferrerPolicy.h:135] DEBUGGING STRING ==> 5 6 same-origin
[ReferrerPolicy.h:135] DEBUGGING STRING ==> 5 6 same-origin
[ReferrerPolicy.h:135] DEBUGGING STRING ==> 5 6 same-origin
[ReferrerPolicy.h:135] DEBUGGING STRING ==> 5 1 no-referrer-when-downgrade
[ReferrerPolicy.h:135] DEBUGGING STRING ==> 5 1 no-referrer-when-downgrade

And it is weird that only the first request got wrong referrer policy, the second one got it correctly.
Did we get a wrong channel?
I don't see why the channel would be wrong (and only for the first case)

Can you please post your debugging test (to understand the output)

Honza

Comment 9

[Thomas Nguyen (:tnguyen)]

diff --git a/netwerk/base/ReferrerPolicy.h b/netwerk/base/ReferrerPolicy.h
index 9cd36f0c7d7d..b044f70c2421 100644
--- a/netwerk/base/ReferrerPolicy.h
+++ b/netwerk/base/ReferrerPolicy.h
@@ -132,6 +132,7 @@ inline ReferrerPolicy AttributeReferrerPolicyFromString(
}

inline const char* ReferrerPolicyToString(ReferrerPolicy aPolicy) {

• printf_stderr("\n[ReferrerPolicy.h:135] DEBUGGING STRING ==> %d %d %s\n\n", 5, aPolicy, kReferrerPolicyString[static_cast<uint32_t>(aPolicy)]);
  return kReferrerPolicyString[static_cast<uint32_t>(aPolicy)];
  }

If no one in your team could take a look at the moment, just assign to me. I will take a look 1-2 weeks later
Thanks

Comment 10

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Thomas Nguyen from comment #9)

If no one in your team could take a look at the moment, just assign to me. I will take a look 1-2 weeks later
Done, thanks!

Honza

Comment 11

[Hengyeow (:tanhengyeow)]

Seems like a platform issue, but would be glad to follow up if there are any JS stuff that needs help after the issue has been investigated :)

Comment 12

[Thomas Nguyen (:tnguyen)]

The channel was not updated the referrer policy after found "referrer-policy". That's is only for loading document case. I will decide to fix it after exposing referrer policy in channel.

Comment 13

[Thomas Nguyen (:tnguyen)]

(In reply to Thomas Nguyen from comment #12)

The channel was not updated the referrer policy after found "referrer-policy". That's is only for loading document case. I will decide to fix it after exposing referrer policy in channel.

The problem is, the showing referrer policy is the real (and old) referrer policy which channel used to compute referer header and sent to the network.
I don't quite sure what is going to show, channel request's referrer policy (1) or document's (or response's) referrer policy (2)?

• If (1), I don't see any spec point that the request have to update referrer policy from Referrer-Policy header. I assume we are doing right. Except redirect:
  https://w3c.github.io/webappsec-referrer-policy/#parse-referrer-policy-from-header
• If (2), we probably have to get response's header or document's referrer policy instead of channel.referrerpolicy
  https://searchfox.org/mozilla-central/rev/dd7e27f4a805e4115d0dbee70e1220b23b23c567/devtools/server/actors/network-monitor/network-observer.js#499
  Hi Honza, could someone in your team could anwser my question?
  Thanks

Comment 14

[Jan Honza Odvarko [:Honza] (always need-info? me)]

@Christoph, can you please take a look at the question in comment #13?

Thanks,
Honza

Comment 15

[Thomas Nguyen (:tnguyen)]

I am biased to keep our implementation the same (as we are observing the channel we really sent to network).

Comment 16

[Christoph Kerschbaumer [:ckerschb, back on Aug 10th}]

Looking at comment 0 I see the following. I see request headers and response headers:

• Since this is the initial top-level request I assume we display the default origin policy which is 'no-referrer-when-downgrade' (since no referrer-policy has been transmitted to the user agent yet).
• The response header includes 'same-origin' so I assume subsequent subresouce loads include the right origin policy.

I guess the right thing to do is to exclude loads of TYPE_DOCUMENT when displaying the referrer policy in the request header.

Does that sound right Thomas?

Comment 17

[Junior [inactive]]

Per 4.1.2.6 of Fetch [1], we would set no-referrer-when-downgrade as a default policy.

[1] https://fetch.spec.whatwg.org/#main-fetch

Comment 18

[Christoph Kerschbaumer [:ckerschb, back on Aug 10th}]

(In reply to Junior [:junior] from comment #17)

Per 4.1.2.6 of Fetch [1], we would set no-referrer-when-downgrade as a default policy.

Oh thanks - in that case I guess there is no bug here, or am I missing something?

Comment 19

[Thomas Nguyen (:tnguyen)]

(In reply to Christoph Kerschbaumer [:ckerschb] from comment #18)

(In reply to Junior [:junior] from comment #17)

Per 4.1.2.6 of Fetch [1], we would set no-referrer-when-downgrade as a default policy.

Oh thanks - in that case I guess there is no bug here, or am I missing something?

Let me clarify it:

• Given a webpage A.html with <meta name="referrer" content="origin">, then user opens a new window B.html from a link (Right click, and click Open New Window in context menu). Then the channel opening new window will get the referrer policy of document A (origin in this case).
• Supposed that when loading B.html, we get a Referrer-Policy response = "no-referrer".
  What will happen is:
• The first (top level) load used the old referrer policy, this should be "origin" and other loads of B.html should use new referrer policy: "no-referrer"

I understand Chrome will show "no-referrer" but it seems not reasonable. I think showing "origin" for the first load and "no-referrer" for other loads is correct behavior.