1385576 - Tracking Protection broken in Nightly.

Status: NEW

(Toolkit :: Safe Browsing, defect, P3)

Description

[Eric Rescorla (:ekr)]

Attachment: PB in Nightly

Comment 1

[Eric Rescorla (:ekr)]

Attachment: PB in Release

Note how Release has the TP shield *and* no banner ad.

Comment 2

[Honza Bambas (:mayhemer)]

Tentatively marking as blocker of bug 1360581 that has changed this recently.  Kershaw?

Interesting we don't have a single test that would catch this...

Note: the second screenshot shows the page is still loading, :ekr, are you sure the ad was not just loading slowly?

Comment 3

[Eric Rescorla (:ekr)]

Attachment: Screen Shot 2017-07-29 at 8.33.26 AM.png

Comment 4

[Eric Rescorla (:ekr)]

I'm not sure, but here's a finished loading screenshot of the same page from Nightly (which has mysteriously started working again) and note how it doesn't show the banner ad either, so I think it's pretty likely that the banner is from a tracker. Also, shouldn't the *shield* start showing immediately because we block prior to domain resolution.

Comment 5

[Kershaw Chang [:kershaw]]

I can't reproduce this at my side.

:ekr, if possible, could you try to capture the log by setting MOZ_LOG as below.

MOZ_LOG=timestamp,rotate:200,nsHttp:5,nsChannelClassifier:5

Also, I'd like to know more details about how to reproduce this. Thanks.

Comment 6

[Eric Rescorla (:ekr)]

As noted in c4, it has started working again. But I think these screenshots are fairly strong evidence there is a bug

Comment 7

[Honza Bambas (:mayhemer)]

Could it be that that one advertisement, for one time, was simply not coming from a tracker domain?  That we simply miss it on the list?

Comment 8

[Honza Bambas (:mayhemer)]

:ekr, I'll also ask you:
- how many times have you reloaded the page and also reproduced the problem?
- or in general, how many times/how long have you seen this problem?
- have you refreshed the page with pressing enter in the address bar or with F5 or with Ctrl-F5?

Comment 9

[Eric Rescorla (:ekr)]

@c8: None of these. It happened once so I reported it.

WRT @c7: that's possible, I suppose, but as noted, sites are generally full of trackers, so it seems unlikely it suddenly didn't have any.

Comment 10

[Honza Bambas (:mayhemer)]

(In reply to Eric Rescorla (:ekr) from comment #9)
> @c8: None of these. It happened once so I reported it.

Have you restarted the browser in between?  Or you just went to the page again later (in a new tab) and then it worked?  (Sorry, but I really would appreciate few more details when you report a bug...)

> 
> WRT @c7: that's possible, I suppose, but as noted, sites are generally full
> of trackers, so it seems unlikely it suddenly didn't have any.

Thanks.  Looking at how many trackers are there, and the missing shield icon, apparently something was wrong.  Thinking of it more, this is probably happening on a higher level than in the code bug 1360581 has modified.  Since I would more tend to believe it would be happening more randomly.

Anyway, I believe there is (was at the moment) a bug, but I have no idea at the moment how to investigate further.  Since I use Nightly with TP always on as well, I will keep a closer eye on this since now.

Comment 11

[Eric Rescorla (:ekr)]

(In reply to Honza Bambas (:mayhemer) from comment #10)
> (In reply to Eric Rescorla (:ekr) from comment #9)
> > @c8: None of these. It happened once so I reported it.
> 
> Have you restarted the browser in between?  Or you just went to the page
> again later (in a new tab) and then it worked? 

It just worked.


> (Sorry, but I really would
> appreciate few more details when you report a bug...)

Those details were not available when I reported the bug, because it was broken.

Comment 12

[Patrick McManus [:mcmanus]]

honza, it seems likely that the interaction here is between the classic tracking-protection-cancel feature (which is enabled in PB mode iirc), and the tracking-protection-priority feature (which is new and part of your work).

The bigger bug here seems to be that something probably made the TP-cancel feature not work in the PB-nightly screenshot (it is showing what is generally a tracker on that page.. but given that we don't know the URIs involved we cannot be 100% certain of that).

It might be specific to TP-cancel, or it might just be a bug involving cancel() generally and the reprioritization. The lack of shield is almost certainly as you say because the loadgroup isn't complete.. which smells like a book-keeping on cancel problem.

does that help give something to go on? It might require some due dilligence. Its all wfm as well.

ekr, next time you're on vox (as you're the only known repro for this) can you just do a few reloads so we can see if its a common thing for your config? if it is - you can turn a log on dynamically via about:networking to capture it. That would be helpful.

Comment 13

[Honza Bambas (:mayhemer)]

(In reply to Patrick McManus [:mcmanus] from comment #12)
> honza, it seems likely that the interaction here is between the classic
> tracking-protection-cancel feature (which is enabled in PB mode iirc), and
> the tracking-protection-priority feature (which is new and part of your
> work).

That was my first suspicion too, yes.

> 
> The lack of shield is almost
> certainly as you say because the loadgroup isn't complete.. 

No.  We show the shield the first moment we encounter any tracker on a page (lg), long before all lg requests are finished.  Hence, if it is missing, we missed ALL the TP listed resources, which is weird and definitely wrong.

> which smells
> like a book-keeping on cancel problem.
> 
> does that help give something to go on? It might require some due
> dilligence. Its all wfm as well.

My knowledge of the TP code is limited, but I believe Kershaw could try checking the code paths as he knows this area currently the best of the Necko team.


I might miss the very short and abbreviated use of "PB" in the screenshot names.  Does that mean this was observed in Private Browsing windows?  ekr, do you have the default Tracking Protection settings? (i.e. enable Tracking Protection: Only in Private Windows?)  If so, could you just accidentally open the tab in a normal window instead?  The PB markings are not visible in the screenshot, so it could be that.

Comment 14

[Eric Rescorla (:ekr)]

I always run TP, but I specifically opened a PB window to test it because I saw it on a regular window. I'll make some attempt to repro it, but I go to vox a lot, and haven't seen it again.

Comment 15

[François Marier [:francois]]

Ekr, do you have any content blocking addons installed? (e.g. NoScript, Privacy Badger, uBlock, AdblockPlus, etc.)

Comment 16

[Honza Bambas (:mayhemer)]

Hmm.. I have a page that during one session was showing the shield.  In another browser session it doesn't.  But we definitely block trackers according the console and also local debugging that instance.

However, I can see that requests to e.g. http://www.googletagservices.com/tag/js/gpt.js are sometimes blocked by the classifier and sometimes are not.  Those that are not are coming from the cache after server validation (on plain F5, apparently), we apparently don't even start the classifier for them.

Michal, RCWN?


Then another thing - after reload (plain F5) I can see the following in the console:

Loading failed for the <script> with source “http://www.googletagservices.com/tag/js/gpt.js”.
ads-position-2.html:1
Loading failed for the <script> with source “http://gacz.hit.gemius.pl/xgemius.js”.
mld.fpl:1
Loading failed for the <script> with source “http://www.google-analytics.com/ga.js”.
mld.fpl:98
Loading failed for the <script> with source “https://ads.rubiconproject.com/ad/10900.js”.
ads-position-1.html:9
Loading failed for the <script> with source “http://ads.rubiconproject.com/ad/10900.js”.
ads-position-1.html:9


But I don't see any mentioning of tracking protection and also don't see the shield in the addres bar.

But from time to time (plain F5) I also see additional lines in the console:

The resource at “http://cpex.demdex.net/event?d_nsid=8&d_ld=_ts%3D1501868706304&d_rtbd=json&d_jsonv=1&d_dst=1&d_cb=demdexRequestCallback_8_1501868706304&c_pagehostname=www.slovnik.cz&c_pagetitle=slovnik.cz%20-%20Multilingual%20Dictionary&c_pageurl=http%3A%2F%2Fwww.slovnik.cz%2F&c_pagekeywords=slovnik.cz%2Cmultilingual%2Cdictionary&c_pagedescription=slovnik.cz%20-%20Multilingual%20Dictionary&c_publisher=CPEx” was blocked because tracking protection is enabled.
[Learn More]
www.slovnik.cz
Loading failed for the <script> with source “http://cpex.demdex.net/event?d_nsid=8&d_ld=_ts%3D1501868706304&d_rtbd=json&d_jsonv=1&d_dst=1&d_cb=demdexRequestCallback_8_1501868706304&c_pagehostname=www.slovnik.cz&c_pagetitle=slovnik.cz%20-%20Multilingual%20Dictionary&c_pageurl=http%3A%2F%2Fwww.slovnik.cz%2F&c_pagekeywords=slovnik.cz%2Cmultilingual%2Cdictionary&c_pagedescription=slovnik.cz%20-%20Multilingual%20Dictionary&c_publisher=CPEx”.

after which I can see the shield.

Comment 17

[Michal Novotny [:michal]]

(In reply to Honza Bambas (:mayhemer) from comment #16)
> However, I can see that requests to e.g.
> http://www.googletagservices.com/tag/js/gpt.js are sometimes blocked by the
> classifier and sometimes are not.  Those that are not are coming from the
> cache after server validation (on plain F5, apparently), we apparently don't
> even start the classifier for them.
> 
> Michal, RCWN?

AFAICS, classifying is done before nsHttpChannel::Connect is called, so this happens before we consider racing. Btw, I don't see the tracking protection icon on nightly even if I disable RCWN in about:config, so I doubt RCWN is anyhow involved.

Comment 18

[Kershaw Chang [:kershaw]]

(In reply to Michal Novotny (:michal) from comment #17)
> (In reply to Honza Bambas (:mayhemer) from comment #16)
> > However, I can see that requests to e.g.
> > http://www.googletagservices.com/tag/js/gpt.js are sometimes blocked by the
> > classifier and sometimes are not.  Those that are not are coming from the
> > cache after server validation (on plain F5, apparently), we apparently don't
> > even start the classifier for them.
> > 
> > Michal, RCWN?
> 
> AFAICS, classifying is done before nsHttpChannel::Connect is called, so this
> happens before we consider racing. Btw, I don't see the tracking protection
> icon on nightly even if I disable RCWN in about:config, so I doubt RCWN is
> anyhow involved.

It's actually depended on whether the channel's uri is in local TP list. If it's in the list, classifier is started before nsHttpChannel::Connect is called. If not, classifying is done after nsHttpChannel::Connect.

Honza, could you provide me the steps for reproducing this issue as you said in comment#16? I'd like to have a look. Thanks.

Comment 19

[Honza Bambas (:mayhemer)]

(In reply to Kershaw Chang [:kershaw] from comment #18)
> Honza, could you provide me the steps for reproducing this issue as you said
> in comment#16? I'd like to have a look. Thanks.

There are no STR.  I was only loading http://www.slovnik.cz/ while TP was always on (not just in PB mode) in different browser sessions across different time spans.  It's changing the ads periodically what makes the shield sometimes to show and sometimes not.

Comment 20

[Michal Novotny [:michal]]

Kershaw, try loading https://www.novinky.cz

I always get tracking protection notification in 55.0b14, but never in 56.0b1.

Comment 21

[Kershaw Chang [:kershaw]]

(In reply to Michal Novotny (:michal) from comment #20)
> Kershaw, try loading https://www.novinky.cz
> 
> I always get tracking protection notification in 55.0b14, but never in
> 56.0b1.

I've tried, but I can't see this on my nightly.
Could you try to get the log as I said in comment#5? Thanks.

Comment 22

[Michal Novotny [:michal]]

Attachment: log.xz

Comment 23

[Kershaw Chang [:kershaw]]

The log file looks fine and the reason why tracking protection is not working is because that tracking protection is disabled. Looking into the log file and you can see that tpEnabled is always 0. FWIW, channel annotation is working perfectly.

So, I'd like to know more details about why TP is disabled.
Michal, could you please check the following things:
1. Did you load the page in private browsing mode?
2. Please check the value of prefs "privacy.trackingprotection.enabled" and "privacy.trackingprotection.pbmode.enabled".
3. Do you have any addons installed as Francois said in comment#15?

Thanks.

Comment 24

[Michal Novotny [:michal]]

(In reply to Kershaw Chang [:kershaw] from comment #23)
> So, I'd like to know more details about why TP is disabled.
> Michal, could you please check the following things:
> 1. Did you load the page in private browsing mode?

No

> 2. Please check the value of prefs "privacy.trackingprotection.enabled" and

This is false. When changing it to true TP works correctly. So ignore my comments and sorry for the confusion. I tested the older version with a different profile where this pref is enabled, but I'm pretty sure I didn't enable it myself.