1593083 - Tracking channels are improperly classified as non-tracking in the child process

Status: RESOLVED FIXED

(Core :: Privacy: Anti-Tracking, defect, P1)

Description

[Steven Englehardt [:englehardt]]

Attachment: Screenshot from 2019-10-29 15-00-38.png

STR:

1. On Nightly, go to https://www.lemonde.fr/ and open up the devtools network panel
2. Scroll down the page a bit to trigger the loading of third parties
3. Filter network panel to adnxs.com

Expected: All adnxs.com resources have cookies stripped
Actual: Only some adnxs.com resources have cookies stripped.

I started digging into the cause and I believe it's related to a mismatch in the channel classification flags between the parent and child channels. In the parent process, the resources are classified as expected:

[Parent 24613: URL Classifier]: D/UrlClassifierDbService Checking table ads-track-digest256, URL is ib.adnxs.com/ut/v3/prebid
[Parent 24613: URL Classifier]: D/UrlClassifierDbService Found a result in fragment adnxs.com/, hash EE75F998B5C372274469BA10C15D55272633342053B6403CD62B8408C5D8E45A (98F975EE)
[Parent 24613: URL Classifier]: D/UrlClassifierDbService Result confirmed., match 32-bytes prefix
[Parent 24613: URL Classifier]: D/UrlClassifierDbService Found 1 results.
[Parent 24613: URL Classifier]: I/nsChannelClassifier TableData::DoLookup[0x7f77a88f3790] - lookup completed. Matches: 1
[Parent 24613: URL Classifier]: I/nsChannelClassifier FeatureData::DoLookup[0x7f77aa9b4348] - blacklisted before whitelisting: 1
[Parent 24613: URL Classifier]: I/nsChannelClassifier TableData::DoLookup[0x7f77a75ca9a0] - starting lookup
[Parent 24613: URL Classifier]: D/UrlClassifierDbService Checking table mozstd-trackwhite-digest256, URL is www.lemonde.fr/?resource=adnxs.com
[Parent 24613: URL Classifier]: D/UrlClassifierDbService Found 0 results.
[Parent 24613: URL Classifier]: I/nsChannelClassifier TableData::DoLookup[0x7f77a75ca9a0] - lookup completed. Matches: 0
[Parent 24613: URL Classifier]: I/nsChannelClassifier TableData::DoLookup[0x7f77a88f3d90] - starting lookup
[Parent 24613: URL Classifier]: D/UrlClassifierDbService Checking table moztest-trackwhite-simple, URL is www.lemonde.fr/?resource=adnxs.com
[Parent 24613: URL Classifier]: D/UrlClassifierDbService Found 0 results.
[Parent 24613: URL Classifier]: I/nsChannelClassifier TableData::DoLookup[0x7f77a88f3d90] - lookup completed. Matches: 0
[Parent 24613: URL Classifier]: I/nsChannelClassifier FeatureData::DoLookup[0x7f77aa9b4348] - blacklisted
[Parent 24613: URL Classifier]: I/nsChannelClassifier FeatureTask::DoLookup[0x7f77a74ead00] - lookup completed
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureData::MaybeCompleteClassification[0x7f77aa9b4208] - completing classification for channel 0x7f77aa9ed040
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureData::MaybeCompleteClassification[0x7f77aa9b4208] - no match. Let's move on
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureData::MaybeCompleteClassification[0x7f77aa9b4248] - completing classification for channel 0x7f77aa9ed040
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureData::MaybeCompleteClassification[0x7f77aa9b4248] - no match. Let's move on
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureData::MaybeCompleteClassification[0x7f77aa9b4288] - completing classification for channel 0x7f77aa9ed040
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureData::MaybeCompleteClassification[0x7f77aa9b4288] - no match. Let's move on
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureData::MaybeCompleteClassification[0x7f77aa9b42c8] - completing classification for channel 0x7f77aa9ed040
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureData::MaybeCompleteClassification[0x7f77aa9b42c8] - no match. Let's move on
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureData::MaybeCompleteClassification[0x7f77aa9b4308] - completing classification for channel 0x7f77aa9ed040
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureData::MaybeCompleteClassification[0x7f77aa9b4308] - no match. Let's move on
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureData::MaybeCompleteClassification[0x7f77aa9b4348] - completing classification for channel 0x7f77aa9ed040
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureData::MaybeCompleteClassification[0x7f77aa9b4348] - blacklisted
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureData::MaybeCompleteClassification[0x7f77aa9b4348] - process channel 0x7f77aa9ed040
[Parent 24613: Main Thread]: D/thirdPartyUtil ThirdPartyUtil::IsThirdPartyChannel [channel=0x7f77aa9ed040]
[Parent 24613: Main Thread]: D/thirdPartyUtil ThirdPartyUtil::IsThirdPartyInternal lemonde.fr =? adnxs.com
[Parent 24613: Main Thread]: D/thirdPartyUtil ThirdPartyUtil::IsThirdPartyInternal adnxs.com =? lemonde.fr
[Parent 24613: Main Thread]: I/nsChannelClassifier UrlClassifierCommon::AnnotateChannel, annotating channel[0x7f77aa9ed040]
[Parent 24613: Main Thread]: I/nsChannelClassifier FeatureTask::CompleteClassification[0x7f77a74ead00] - exec callback

However, in the child process the channel for this URI appears to no longer have the classification flag:

[Child 24900: Main Thread]: D/AntiTracking Computing whether channel 0x7f99fc8a9070 has access to URI https://ib.adnxs.com/ut/v3/prebid
[Child 24900: Main Thread]: D/AntiTracking Deciding whether the user has overridden content blocking for https://www.lemonde.fr/
[Child 24900: Main Thread]: D/AntiTracking No user override found
[Child 24900: Main Thread]: D/thirdPartyUtil ThirdPartyUtil::IsThirdPartyChannel [channel=0x7f99fc8a9070]
[Child 24900: Main Thread]: D/thirdPartyUtil ThirdPartyUtil::IsThirdPartyInternal lemonde.fr =? adnxs.com
[Child 24900: Main Thread]: D/AntiTracking Our channel isn't a third-party tracking channel

That last message is the result of the check here: https://searchfox.org/mozilla-central/rev/ce02064d8afc8673cef83c92896ee873bd35e7ae/toolkit/components/antitracking/AntiTrackingCommon.cpp#1692-1694

Comment 1

[Gary Chen [:xeonchen]]

Attachment: Bug 1593083 - Ensure mTopWindowURI & mContentBlockingAllowListPrincipal are set correctly during HTTP redirects;

Comment 2

[Honza Bambas (:mayhemer)]

Gary, please add a good explanation of your patch, the cause of the bug etc... Thanks.

Comment 3

[Gary Chen [:xeonchen]]

(In reply to Honza Bambas (:mayhemer) from comment #2)

Gary, please add a good explanation of your patch, the cause of the bug etc... Thanks.

Just added, sorry for not having a good description of that.

Comment 4

[(no longer active)]

Bug 1516309 is an existing bug which is related to the classification flags being accessed before a classification is done on the post-redirect channel... I wonder if you're seeing something similar here, in case it helps.

Comment 5

[Gary Chen [:xeonchen]]

I think this could be an intermittent bug.

Yesterday :englehardt told me it was good, so I use mozregression today to find out regression build.
But I can't find the exact revision because the result doesn't make any sense.

Here's what I found:

1. It can be good on today's (2019-11-13) build.
2. Usually there are only connections to ib.adnxs.com, and this is almost good except on some reload case.
3. Except case 2, if something get unclassified, it's <country>-ib.adnxs.com, in my case <country> is ams1 and in the bug description, it is lax1. But connection to this host doesn't always happen.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug is a regression, but please revert this change in case of error.

Comment 7

[(no longer active)]

(In reply to Gary Chen [:xeonchen] from comment #5)

I think this could be an intermittent bug.

Yesterday :englehardt told me it was good, so I use mozregression today to find out regression build.
But I can't find the exact revision because the result doesn't make any sense.

Here's what I found:

1. It can be good on today's (2019-11-13) build.
2. Usually there are only connections to ib.adnxs.com, and this is almost good except on some reload case.
3. Except case 2, if something get unclassified, it's <country>-ib.adnxs.com, in my case <country> is ams1 and in the bug description, it is lax1. But connection to this host doesn't always happen.

The intermittency is probably the result of different things coming down the network, no? It may be a good idea to use a record and replay proxy (like mitmproxy) to record a version of the page that reproduces the bug and then use mozregression on the recording...

Comment 8

[Gary Chen [:xeonchen]]

It looks like UrlClassifierCommon::ShouldEnableClassifier returns false because it is called in parent process, where GetTopWindowURI is invalid.

By setting pref dom.serviceWorkers.parent_intercept to false, will seem to fix this because the above checking will be done in child process.
The pref is enabled in bug 1456995.

Steven, could you help to test if the pref fixes the issue on your platform?

Comment 9

[Steven Englehardt [:englehardt]]

Yes, setting that pref to false fixes the issue for me. Good find!

Comment 10

[Gary Chen [:xeonchen]]

(In reply to Steven Englehardt [:englehardt] from comment #9)

Yes, setting that pref to false fixes the issue for me. Good find!

Thanks. I think this is probably regressed by bug 1456995, where ServiceWorker interception was moved from child to parent process, therefore a classification is not possible.

Perry and Dimi, any idea?

Comment 11

[Dimi Lee [ooo 16.04 - 21.04］]

(In reply to Gary Chen [:xeonchen] from comment #8)

It looks like UrlClassifierCommon::ShouldEnableClassifier returns false because it is called in parent process, where GetTopWindowURI is invalid.

This is because Bug 1437626

After talking with Gary, we think Bug 1437626 may NOT help fix this issue(Although that is still something we need to fix).
As gary explained to me, the flow here is:

1. visit channel tracker.com
2. channel tracker.com is intercepted by ServiceWorker
3. SW decides not to intercept this channel and redirects it back to the original channel tracker.com

In step 1, tracker.com is annotated as a tracker no matter dom.serviceWorkers.parent_intercept is on or off
In step 2, intercepted tracker.com is NOT annotated as a tracker no matter dom.serviceWorkers.parent_intercept is on or off (Because of Bug 1437626)
In Step 3. tracker.com is annotated as a tracker when the pref is off and is NOT annotated as a a tracker when the pref is on.

This bug relies on step3 to annotate channels properly, we need to figure out what's the difference between the pref is on and the pref is off in step3. gary, please correct me if anything is wrong here, thx

Comment 12

[Ethan Tseng [:ethan]]

Considering the impact, this bug should be a top priority.

Comment 13

[(no longer active)]

It sounds like the fix here is to fix bug 1437626.

Comment 14

[Perry Jiang [:perry] [no longer employee, use ni?]]

Clearing ni? because it looks like the underlying bug is identified.

Comment 15

[Gary Chen [:xeonchen]]

Just updated the patch. Copying mTopWindowURI while redirecting to InterceptedHttpChannel will make UrlClassifier work again.
I think fixing bug 1437626 may potentially fix this bug.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7f37a007a205
Ensure mTopWindowURI & mContentBlockingAllowListPrincipal are set correctly during HTTP redirects; r=mayhemer,Ehsan

Comment 17

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/7f37a007a205