1709612 - The page does not show the article with ETP set to STRICT at reuters.com

Status: VERIFIED FIXED

(Core :: Privacy: Anti-Tracking, defect, P3)

Description

[Raul Bucata]

Attachment: Screenshot 2021-05-05 170253.png

Environment:
Operating system: Windows 10 PRO x64
Firefox version: :Firefox Nightly 90.0a1 (2021-05-05)

Preconditions:
ETP set to STRICT
Clean profile

Steps to reproduce:

1. Navigate to: https://www.reuters.com/world/us/us-supreme-court-hear-peeved-cheerleaders-free-speech-case-2021-04-28/
2. Observe the page

Expected Behavior:
The article is loaded

Actual Behavior:
The article loads briefly, then disappears

Notes:

• Not reproducible with ETP set to STANDARD.
• Works as expected using Chrome.

Comment 1

[Thomas Wisniewski [:twisniewski]]

There are some ad scripts being blocked here, but this error is particularly interesting:

TypeError: l.safetyDataAvailable is not a function

This is from this code, accessing Moat's API:

              l = window.moatPrebidApi;
              if (l && l.safetyDataAvailable() && 'function' == typeof l.getMoatTargetingForPage) try {
                c = window.moatPrebidApi.getMoatTargetingForPage()

And indeed, my candidate shim for Moat I developed for bug 1651383 resolves this if I have it apply for moatheader.js.

Comment 2

[Oana Arbuzov [:oanaarbuzov]]

Also reproducible for URL: https://www.reuters.com/technology/musk-tweet-dents-bitcoin-weekly-gain-prospect-2021-06-04/
https://prnt.sc/14up3qn

Environment:
Firefox version: Firefox Nightly 91.0a1 (2021-06-08)
Operating system: Windows 10 PRO x64

Comment 3

[Simona Badau, Desktop QA]

I can still reproduce this issue on Firefox 92 beta 5 and Nightly 93.0a1, the steps are a little different than the ones from the Description:

Steps to reproduce:

1. Open Firefox with a new profile
2. Set ETP to Strict
3. Navigate to: https://www.reuters.com/world/us/us-supreme-court-hear-peeved-cheerleaders-free-speech-case-2021-04-28/
4. Restart Firefox
5. Go back to the tab from step 3.

Actual results:
In step 5 the video is no longer displayed.

Comment 4

[Thomas Wisniewski [:twisniewski]]

I take it you're missing a step 2.5, to set Firefox to restore your previous tab session on startup?

Either way, I can't reproduce this on the most recent nightly build. Are there any interesting messages in the web console, perhaps? Does the console at least mention whether anything is being shimmed by Firefox?

Comment 5

[Simona Badau, Desktop QA]

When the Reuters permissions pop-up is displayed I click on "Allow All". I've made a screencast with the exact steps I'm taking to reproduce this issue:
https://drive.google.com/file/d/1pFDSlfBD0p9O8dBe7EtbcAGZLcLSi42m/view

In the web console, I'm seeing some shimmed mentions but do not seem related:

Moat is being shimmed by Firefox. See https://bugzilla.mozilla.org/show_bug.cgi?id=1713704 for details. sandbox eval code:1:9
Integral Ad Science PET is being shimmed by Firefox. See https://bugzilla.mozilla.org/show_bug.cgi?id=1713701 for details. sandbox eval code:1:9
Google Analytics and Tag Manager is being shimmed by Firefox. See https://bugzilla.mozilla.org/show_bug.cgi?id=1713687 for details. sandbox eval code:1:9
Chartbeat is being shimmed by Firefox. See https://bugzilla.mozilla.org/show_bug.cgi?id=1713699 for details. sandbox eval code:1:9
Google Publisher Tags is being shimmed by Firefox. See https://bugzilla.mozilla.org/show_bug.cgi?id=1713685 for details. sandbox eval code:1:9
Amazon Transparent Ad Marketplace is being shimmed by Firefox. See https://bugzilla.mozilla.org/show_bug.cgi?id=1713698 for details. sandbox eval code:1:9

​And I'm also seeing an interesting message about "reuters":
Request to access cookie or storage on “https://ad.wsod.com/site/dc54d4678e62010da03e468039cfe826/1.0.async/1629444562;0,0,0;1792x1120x2;https%3A_@[e-mail]_@2Fworld_@2Fus_@2Fus-supreme-court-hear-peeved-cheerleaders-free-speech-case-2021-04-28_@2F;;;” was blocked because it came from a tracker and content blocking is enabled.

Comment 6

[Thomas Wisniewski [:twisniewski]]

Interesting. I don't see the GDPR permissions popup at all, and restarting the page with about:profiles as you are isn't reproducing the issue. I have to presume that it is related to the cookie somehow, but that wouldn't be related to SmartBlock.

Paul, maybe you will have better luck reproducing this one and confirming if it's indeed related to cookies being blocked, rather than scripts?

Comment 7

[Paul Zühlcke [:pbz]]

I see the GRPR prompt, probably because I have an IP from Germany. However, I can't reproduce this issue. The article loads fine after accepting the GDPR prompt. Even after a restart the video element works every time.

Do we run shims on this site? If so, maybe something isn't injected correctly when we do session restore? Something along the lines of Bug 1733566.

Tested on Ubuntu 21.10 + Nightly 96.0a1 (2021-11-04) (64-bit).

Comment 8

[Thomas Wisniewski [:twisniewski]]

Yes, we do run shims on this site, at least the ones mentioned in comment #5. But I'm not sure how the STR Simona gave would involve session restore. Perhaps the bug has just been fixed recently? Simona, can you still reproduce the error now? Perhaps it depends on which ads the site is trying to load at the time...

Comment 9

[Simona Badau, Desktop QA]

I can still reproduce the missing video issue when Strict Mode is enabled on Nightly 96.0a1 on macOS Big Sur 11.6 - here is a screencast.

As soon as I disable Strict Mode and enable the Standard mode, I'm able to properly see the video.

Comment 10

[Thomas Wisniewski [:twisniewski]]

Thank you for patiently confirming that it still doesn't work, Simona!

Funny enough, I just tried it again and this time the video isn't loading for me as well, so let's hope I was just doing something silly and I'll be able to reproduce this consistently when I have more time to diagnose it.

I see that they are trying to load a tracking script (iasPET) from an unexpected location, but fixing that doesn't seem to be enough to get the video loading again. It looks like this might require a very involved diagnosis.

Comment 11

[Thomas Wisniewski [:twisniewski]]

It turns out that iasPET was indeed the culprit, but not only because it was being served from another domain. It turns out that it also accepts a queue parameter as other trackers do, which contains callbacks. If these are not run, the video never loads. I have a patch ready which fixes this oversight.

Comment 12

[Thomas Wisniewski [:twisniewski]]

Attachment: Bug 1709612 - Add support for command queuing to the SmartBlock iasPET shim; r?denschub

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c3f603ac7f1d
Add support for command queuing to the SmartBlock iasPET shim; r=denschub,webcompat-reviewers

Comment 14

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/c3f603ac7f1d

Comment 15

[Ardelean Oana, Desktop QA [:oardelean]]

Reproducible on a 2021-05-05 Nightly build on Windows 10 x64 PRO.
With ETP set to STRICT, Nightly 103.0a1 is working as expected. Both the page and the video load without issues. However, Firefox 102.0b4 with ETP set to STRICT will render the article visible, but the video is still missing. No errors in the browser console as well.
Should I re-open this issue since it is still partially reproducible on beta?

Comment 16

[Thomas Wisniewski [:twisniewski]]

Interesting, that makes me suspect that while it was working for me on 102, there were other cases which I missed that have since been fixed in 103 by other work I've done since for SmartBlock for 103. So it might be best to keep an eye on this and update the tracking flags instead to show that it's fully fixed in 103, rather than 102.

Comment 17

[Ardelean Oana, Desktop QA [:oardelean]]

Reproduced on a 2022-05-03 Nightly build on macOS 12; Verified as fixed on Firefox 107.0(build ID: 20221107173030) and Nightly 108.0a1(build ID: 20221108213602) on macOS 12, Windows 10, Ubuntu 22.