1537627 - Expose anti-tracking classification flags in devtools

Status: ASSIGNED

(DevTools :: Netmonitor, enhancement, P3)

Description

[Steven Englehardt [:englehardt]]

Classification flags landed in Bug 1533312 and Bug 1533074. These make it possible to query the classification status of a channel. These include: CLASSIFIED_FINGERPRINTING, CLASSIFIED_CRYPTOMINING, CLASSIFIED_TRACKING.

Right now, devtools only labels domains in the network tab if they have a tracking annotation. We should expand this to include the new classifications, possibly with new icons.

https://senglehardt.com/test/trackingprotection/test_pages/fingerprinting_and_cryptomining_and_trackers.html is a useful test page for this.

Comment 1

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Thanks for the report!

@Harald: are there any existing icons for CLASSIFIED_FINGERPRINTING and CLASSIFIED_CRYPTOMINING?

Honza

Comment 2

[:Harald Kirschner :digitarald]

v0 of this could be exposing the classification in the network's sidepanel, as the icons have the risk of seeming arbitrary without the tracker context.

Could we, for now, expand the existing annotation to all those classifications and at least show the shield; or which I assume is still the generic Content Blocking icon?

Comment 3

[Steven Englehardt [:englehardt]]

(In reply to :Harald Kirschner :digitarald from comment #2)

Could we, for now, expand the existing annotation to all those classifications and at least show the shield; or which I assume is still the generic Content Blocking icon?

It is, and that would make sense to me as a first step.

Comment 4

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Attachment: image.png

Sounds good to me.

Attach a mockup indicating how this could look in the Side panel UI.

Honza

Comment 5

[Hengyeow (:tanhengyeow)]

Hi Honza

Would love to work on this bug if there are no other bugs blocking work for it :)

Comment 6

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Heng Yeow (:tanhengyeow) from comment #5)

Would love to work on this bug if there are no other bugs blocking work for it :)

Assigned to you, thanks for the help!
AFAIK there should be no blockers

Honza

Comment 7

[Hengyeow (:tanhengyeow)]

Attachment: Bug 1537627 - Expose anti-tracking classification flags in devtools. r=Honza

Comment 8

[Hengyeow (:tanhengyeow)]

Attachment: initial-patch-classificationFlags.png

Hi Honza

I've attached an initial patch for this bug. Fetching from the backend and displaying the value on the UI looks fine but the value displayed isn't very useful to the user.

The closest useful value I could find from the .idl file is this attribute but it is a long value.

Perhaps we need something that can return us the string value of the classification flag?

Comment 9

[Jan Honza Odvarko [:Honza] (always need-info? me)]

@tanhengyeow: What page are you using for testing. I tried (from comment #0)
https://senglehardt.com/test/trackingprotection/test_pages/fingerprinting_and_cryptomining_and_trackers.html

... but the classificationFlags is always 0 for me

Honza

Comment 10

[Hengyeow (:tanhengyeow)]

@Honza Same for me too. I used the normal Google home page and was able to see the value for some requests. Seems like the issue might lie with not fetching the correct value from the backend API.

Comment 11

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Heng Yeow (:tanhengyeow) from comment #10)

@Honza Same for me too. I used the normal Google home page and was able to see the value for some requests. Seems like the issue might lie with not fetching the correct value from the backend API.

I see, thank for the update!

@Steven, we are testing the attached patch with this page:
https://senglehardt.com/test/trackingprotection/test_pages/fingerprinting_and_cryptomining_and_trackers.html

But the classificationFlags is always 0. Is that expected?

Honza

Comment 12

[Jan Honza Odvarko [:Honza] (always need-info? me)]

@Steven, the comment for classificationFlags; (in nsIHttpChannel.idl file [1]) says:
This value is equal to "firstPartyClassificationFlags || thirdPartyClassificationFlags".

Should we define our own (localized) strings to display the value in the UI or we should use some existing strings?

[1] https://searchfox.org/mozilla-central/rev/aba472751e24763d0c18bae8408e9d7106e9acea/netwerk/protocol/http/nsIHttpChannel.idl#528

Honza

Comment 13

[Jan Honza Odvarko [:Honza] (always need-info? me)]

@Harald, the classificationFlags has two values:

• firstPartyClassificationFlags
• thirdPartyClassificationFlags

What wording we should use in the side panel? (and as tooltips for the shield icon I guess)
Honza

Comment 14

[(no longer active)]

(In reply to Jan Honza Odvarko [:Honza] (always need-info? me) from comment #11)

(In reply to Heng Yeow (:tanhengyeow) from comment #10)

@Honza Same for me too. I used the normal Google home page and was able to see the value for some requests. Seems like the issue might lie with not fetching the correct value from the backend API.

I see, thank for the update!

@Steven, we are testing the attached patch with this page:
https://senglehardt.com/test/trackingprotection/test_pages/fingerprinting_and_cryptomining_and_trackers.html

But the classificationFlags is always 0. Is that expected?

(Steven is on PTO, baku or I can probably help answer questions too FWIW.)

No, that isn't expected. Make sure the privacy.trackingprotection.fingerprinting.enabled, privacy.trackingprotection.fingerprinting.annotate.enabled, privacy.trackingprotection.cryptomining.enabled and privacy.trackingprotection.cryptomining.annotate.enabled prefs are all set to true (should be the default on Nightly I believe). When you're on that page you should see:

Test for each list will say "Blocked" if Firefox is blocking resources from that list.

Fingerprinting (base-fingerprinting-track-digest256):
BLOCKED

Cryptomining (base-cryptomining-track-digest256):
BLOCKED

Basic Tracking Protection List (base-track-digest256):
NOT BLOCKED

(In reply to Jan Honza Odvarko [:Honza] (always need-info? me) from comment #12)

@Steven, the comment for classificationFlags; (in nsIHttpChannel.idl file [1]) says:
This value is equal to "firstPartyClassificationFlags || thirdPartyClassificationFlags".

BTW are you sure you want to use classificationFlags and not thirdPartyClassificationFlags? Gecko doesn't really treat content loaded as first-party differently based on these classifications...

Should we define our own (localized) strings to display the value in the UI or we should use some existing strings?

I don't think we have pre-existing strings that are suitable to be reused here.

Comment 15

[Steven Englehardt [:englehardt]]

Thanks for answering while I was out Ehsan.

Comment 16

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Attachment: image.png

I am still unable to get non-zero classification flag - even if using channel.thirdPartyClassificationFlags

I am attaching a screenshot of my Browser Console

• I can see content blocking messages for https://base-cryptomining-track-digest256.dummytracker.org/test_not_blocked.png resource

• But the classification flag is zero

I am testing with:
https://senglehardt.com/test/trackingprotection/test_pages/fingerprinting_and_cryptomining_and_trackers.html

All the prefs mentioned in comment #14 are properly set to true (I am on Nightly).

What am I doing wrong?

Honza

Comment 17

[(no longer active)]

(In reply to Jan Honza Odvarko [:Honza] (always need-info? me) from comment #16)

Created attachment 9070536 [details]
image.png

I am still unable to get non-zero classification flag - even if using channel.thirdPartyClassificationFlags

I am attaching a screenshot of my Browser Console

• I can see content blocking messages for https://base-cryptomining-track-digest256.dummytracker.org/test_not_blocked.png resource

• But the classification flag is zero

I am testing with:
https://senglehardt.com/test/trackingprotection/test_pages/fingerprinting_and_cryptomining_and_trackers.html

All the prefs mentioned in comment #14 are properly set to true (I am on Nightly).

What am I doing wrong?

Please note that I am trying to read between the lines based on the screenshots you are posting. I actually don't really know what it is that you are trying to test here so there is a ton of guesswork on my part trying to read into what you're hoping to achieve here. :-) It would be a lot easier if you could explain the goal you are trying to achieve.

But anyways, based on the screenshot:

• The channel for base-cryptomining-track-digest256.dummytracker.org is blocked. So there is no channel left to classify here, therefore no classificationFlags to look at.
• The channel for base-fingerprinting-track-digest256.dummytracker.org is blocked. So there is no channel left to classify here, therefore no classificationFlags to look at.
• The channel for base-track-digest256.dummytracker.org is not blocked. Depending on your content blocking settings our tracking annotations must be annotating it, so with the right prefs the classificationFlags will be non-zero. classificationFlags being 0 suggests your network.cookie.cookieBehavior is probably 0? Setting it to 4 would probably help change the classificationFlags you are observing here. In addition privacy.trackingprotection.annotate_channels should be true.

If that didn't help you can try to use about:url-classifier to debug this. You can type any of these URLs into the "URL" field there and press Start searching to see what URL classifier tables it is found in. For example, "http://base-track-digest256.dummytracker.org" matches the following for me:

tracking-protection	
URI: http://base-track-digest256.dummytracker.org/
List of tables: base-track-digest256
tracking-annotation	
URI: http://base-track-digest256.dummytracker.org/
List of tables: base-track-digest256

The annotation tables are used to populate the classification flags.

Hope this helps.

Comment 18

[(no longer active)]

(BTW you can also turn the anti-tracking logs on by setting MOZ_LOG=AntiTracking:5 to see what's going wrong if none of the above helps. The log message will probably tell you why the anti-tracking module thinks it should not annotate "http://base-track-digest256.dummytracker.org" and apply any classificationFlags to it for you.)

Comment 19

[:Harald Kirschner :digitarald]

I don't think we have pre-existing strings that are suitable to be reused here.

Lets circle back with the copy team at all hands about naming.

Comment 20

[Patrick Brosset <:pbro>]

Is there any update here? This bug has been inactive for a few months, however it received a patch and some good discussions. Is there a way to make it move forward again? Or should it become unassigned if it's blocked?

Comment 21

[Steven Englehardt [:englehardt]]

Now that Bug 1505917 has landed the devtools network panel will only show the shield for things that will be blocked by tracker blocking (i.e., Strict mode or PB windows). In light of that, I suggest that we add an annotation for cookie blocking in addition to fingerprinting and cryptomining.

As an alternative, we could also consider reframing the indicators to show the expected restriction that will be placed on the resource, rather than the exact feature that matched the resource. Tracker blocking (i.e., "legacy" TP), Fingerprinting Blocking, and Cryptomining Blocking all block resources. ETP blocks cookies. So, as an alternative, we could show icons that indicate resource blocking or cookie blocking.

Comment 22

[Jan Honza Odvarko [:Honza] (always need-info? me)]

@Heng Yeow: The classification flag works for me now so, the patch is close to landing!

It just needs to be rebased. Do you want to look at that?

Please ping me either way so, I can reassign if needed.

Honza

Comment 23

[:Harald Kirschner :digitarald]

Honza, how close is the patch to landing? Could we help?

Comment 24

[Jan Honza Odvarko [:Honza] (always need-info? me)]

The patch was working for me already and I could see the classification flag just fine. It just needs to be rebased (also due to the work done on the side bar refactoring)

@Hubert, this patch adds a new "classification flag" in the the general section in the Headers panel. See my mockup in comment #4
Can you please finish this bug (rebase the patch and test), thanks!

This is a test page:
https://senglehardt.com/test/trackingprotection/test_pages/fingerprinting_and_cryptomining_and_trackers.html

Honza

Comment 25

[Hubert Boma Manilla (:bomsy)]

Honza,
Will do thanks!

Comment 26

[:Harald Kirschner :digitarald]

Just for reference, the extension API webRequest/onBeforeRequest exposes urlClassification, split in both firstParty and thirdParty: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/onBeforeRequest and explains all the values.

Comment 27

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Bomsy, looks like we dropped a ball here a while ago.
But, the patch is almost ready, should we rebase and finish?

Honza

Comment 28

[Hubert Boma Manilla (:bomsy)]

Oh sure! i'll rebase and finish this.