1512027 - Disabling Content Blocking for a site hides the tracking icons in Netmonitor

Status: RESOLVED FIXED

(DevTools :: Netmonitor, defect, P2)

Description

[:Harald Kirschner :digitarald]

Attachment: image.png

STR:
- Open CNN.com
- Disable Content Blocking in the mission control
- Open Network panel and reload

AR: Trackers are not marked as trackers.
ER: Tracking icon is applied liberally to the many tracker JS resources.

Comment 1

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Thanks for the report Harald!

I can reproduce this is my machine (OSX)

Honza

Comment 2

[Jan Honza Odvarko [:Honza] (always need-info? me)]

François, I tested this (using STR in comment #0) and nsIHttpChannel.isThirdPartyTrackingResource is false for all resources. Could this be a problem with the platform API?

Honza

Comment 3

[François Marier [:francois]]

I don't work at Mozilla anymore, so I'll redirect to baku.

Comment 4

[(no longer active)]

This isn't a problem, it's very intentional, and it's happening because of this check: https://searchfox.org/mozilla-central/rev/64ef7bc9fb478ba7c292f9fa57813d3fe864201e/netwerk/url-classifier/UrlClassifierCommon.cpp#135

Comment 5

[:Harald Kirschner :digitarald]

If this is intentional, it is probably not the right source for the network panel; which should always represent some insight into which resources might be affected.

Given that CB now has more levels of granularity, maybe Network panel should always check against the strictest/largest set. The objective of the feature is to make developers aware of potential failures.

Is there another function that we can use to match resources against the TP list?

Comment 6

[Andrea Marchesini [:baku]]

Is there another function that we can use to match resources against the TP list?

Yes, you can do this:

let feature = classifierService.getFeatureByName("tracking-annotation");
if (feature) {
  classifierService.asyncClassifyLocalWithFeatures(uri, [feature],
    Ci.nsIUrlClassifierFeature.blacklist,
    list => {
      // list is an array. if empty, the URL is not in the tracking-protection list
    });
}

Comment 7

[(no longer active)]

Comment 6 I think is part of the answer but may not be the full answer... For example, it doesn't take into account whether the URL is a third-party tracking resource or not (e.g. doubleclick.net isn't considered a third-party tracking resource on youtube.com because both are owned by Google).

Andrea, is there a way to query the url classifier service taking the entity list into consideration?

Comment 8

[Andrea Marchesini [:baku]]

I don't think so. We can add something, of course. Let's ask dimi.

Comment 9

[Dimi Lee [:dimi]]

(In reply to Andrea Marchesini [:baku] from comment #8)

I don't think so. We can add something, of course. Let's ask dimi.

If you are asking if URL Classifer has a simple API to do that, I would say no.

Please correct me if I am wrong.
I think what we really need, is not an URL Classifer API, is to separate this use case from tracking annotation. One of the requirement(or goal) for the listmanager revamp, is to support this.

Originally, the tracking annotation is used by all the new features and it is hard to maintain and evolve. With baku's work recently, we can easily add a new Feature(ex. FeatureTrackerInfo) in netwerk/url-classifer , with the same tables as tracking annotation and without checking ::ShouldEnableClassifier[1]. The ultimate goal, is every feature uses its own "url classifier feature".

So I guess what we need to do, is:

1. Add a new UrlClassifierFeature[2], add a new preference to turn it on/off.
   • most of the code should be the same as FeatureTrackingAnnotation[3]
2. Write the classified information somewhere as we did in tracking annotation.
3. Network panel read the information.

Does this make sense?

[1] https://searchfox.org/mozilla-central/rev/bee8cf15c901b9f4b0c074c9977da4bbebc506e3/netwerk/url-classifier/UrlClassifierCommon.cpp#90
[2] https://searchfox.org/mozilla-central/rev/bee8cf15c901b9f4b0c074c9977da4bbebc506e3/netwerk/url-classifier/UrlClassifierFeatureBase.h#17
[3] https://searchfox.org/mozilla-central/rev/bee8cf15c901b9f4b0c074c9977da4bbebc506e3/netwerk/url-classifier/UrlClassifierFeatureTrackingAnnotation.cpp

Comment 10

[(no longer active)]

That sounds like a good plan I think. It will allow us to also make sure we're using the basic list not the strict list, which is another problem the network monitor had come across (not sure if there is a bug on file for it, I can't find one right now...)

Comment 11

[:Harald Kirschner :digitarald]

Pardon if this is completely now how the new URL-Classifier and the features work. My assumption would be that a URL can be annotated by various features, as shown in about:url-classifier.

Could we provide the same annotations for a given URL to DevTools. With this list, the Network panel can then highlight the important ones (and provide the others for debugging purposes). This could also link to about:url-classifier for any advanced inspection/editing capabilities.

Comment 12

[Dimi Lee [:dimi]]

(In reply to :Harald Kirschner :digitarald from comment #11)

Pardon if this is completely now how the new URL-Classifier and the features work. My assumption would be that a URL can be annotated by various features, as shown in about:url-classifier.

Yes

Could we provide the same annotations for a given URL to DevTools. With this list, the Network panel can then highlight the important ones (and provide the others for debugging purposes). This could also link to about:url-classifier for any advanced inspection/editing capabilities.

I think this is also Yes.

Comment 13

[Cristian Comorasu, QA [:ccomorasu], Releae Desktop QA - Please contact [e-mail]]

Hello!
Is a regression window still required? Based on comment 7 I am a bit confused.
Thank you!

Comment 14

[(no longer active)]

No, I don't think so. Thanks!

Comment 15

[Jan Honza Odvarko [:Honza] (always need-info? me)]

The severity isn't "major" here, changing to S3

But, it would still be great to properly mark all trackers in the Network panel and perhaps even more - show annotations for individual URLs in the panel.

Dimi it's been a while, do we have good APIs to read all anotations for given URL and show them in the DevTools Network panel now?

I am looking at API like isTracker here: https://searchfox.org/mozilla-central/rev/31368c7795f44b7a15531d6c5e52dc97f82cf2d5/toolkit/components/antitracking/PurgeTrackerService.jsm#74

But according to the comment #7 this isn't enough and we need more, is that still correct?

Comment 16

[Dimi Lee [:dimi]]

(In reply to Jan Honza Odvarko [:Honza] (always need-info? me) from comment #15)

Dimi it's been a while, do we have good APIs to read all anotations for given URL and show them in the DevTools Network panel now?

I am looking at API like isTracker here: https://searchfox.org/mozilla-central/rev/31368c7795f44b7a15531d6c5e52dc97f82cf2d5/toolkit/components/antitracking/PurgeTrackerService.jsm#74

But according to the comment #7 this isn't enough and we need more, is that still correct?

I think we can use these APIs now. If i remember correctly, we still set classification flag even if you turn off ETP for a site.

Comment 17

[Hubert Boma Manilla (:bomsy)]

(In reply to Dimi Lee [:dimi][:dlee] from comment #16)

(In reply to Jan Honza Odvarko [:Honza] (always need-info? me) from comment #15)

Dimi it's been a while, do we have good APIs to read all anotations for given URL and show them in the DevTools Network panel now?

I am looking at API like isTracker here: https://searchfox.org/mozilla-central/rev/31368c7795f44b7a15531d6c5e52dc97f82cf2d5/toolkit/components/antitracking/PurgeTrackerService.jsm#74

But according to the comment #7 this isn't enough and we need more, is that still correct?

I think we can use these APIs now. If i remember correctly, we still set classification flag even if you turn off ETP for a site.

Thanks Dimi. I'll try this out and see.

Comment 18

[Kershaw Chang [:kershaw]]

Hi Hubert,
I've looked at all comments and try to figure out how necko team can help, but it's not clear to me what kind of support you need from necko. Could you explain more?
Thanks.

Comment 19

[Valentin Gosu [:valentin] (he/him)]

(In reply to Hubert Boma Manilla (:bomsy) from comment #17)

I think we can use these APIs now. If i remember correctly, we still set classification flag even if you turn off ETP for a site.

Thanks Dimi. I'll try this out and see.

Not sure if this bug is about more than what devtools already does here.
If anyone has more info let me know.

Comment 20

[Hubert Boma Manilla (:bomsy)]

Hi Kershaw and Valentin,

Thanks for taking a look.
So the original issue seems to be fixed as there are already tracking icons for most tracker resources in the netmonitor for cnn.com.

But the outstanding issue is that mentioned in the example in Comment 7 where doubleclick.net isn't considered a third-party tracking resource on youtube.com, as double click is owned by google.

I think might be best to close this bug as done and open another that focuses on the specific issue (i'll do that).

Comment 21

[Hubert Boma Manilla (:bomsy)]

Closing this as the related main issue has been fixed. Opened Bug 1795649 for the outstanding issue.