1876149 - Canvas getImageData returns unexpected data in private browsing

Status: NEW

(Core :: Privacy: Anti-Tracking, defect)

Description

[Evan Nowak]

Attachment: Private result - wrong.png

Steps to reproduce:

1. Find a Windows PC running recent Firefox build
2. Enter private browsing
3. Navigate to https://cad.onshape.com/documents/88e4b854473d581b8ab58b2e/w/0486c9f160bc9ad7b4d4aa0d/e/4af56466ed64d19fe5dd1402

Actual results:

The text on the semi-transparent blue planes is clipped.

This text is derived from a rendering in a 2D Canvas. The bounds of the text is extracted from within the canvas so that a tightly-bound WebGL texture can be derived from it. This routine uses the getImageData to read back some rendered text from the canvas. Starting around Firefox 120, this data contains some "noise" in private mode. This noise ends up getting bounded, and the texture ends up placed incorrectly in the WebGL view.

We're making our bounding routine tolerant of this noise so that this will not be an issue in a future release. However, I wanted to report this in case it is of interest to Firefox developers.

FWIW, I attempted to create a self-contained reproducible case. However, the simpler case I created did not reproduce the issue. So there may be some more distant, upstream causes here.

Expected results:

In general I'd expect getImageData to not add unexpected values, whether in private mode or not.

Comment 1

[Evan Nowak]

Attachment: getImageData result in private mode

This is data extracted from getImageData in private mode that contains the noisy pixels. They're not noticeable to the naked eye, but the pixels that contain the aberration have a higher alpha value

Comment 2

[Evan Nowak]

Attachment: private getImageData mask

Here's a "mask" image showing the pixels that exceeded our threshold check. Notice the islands of seemingly random pixels that have a higher alpha value.

Comment 3

[Evan Nowak]

Attachment: Not private result - correct

This is how the scene should look when the Canvas is operating as expected.

Comment 4

[Evan Nowak]

Attachment: Not private getImageData - correct

Here's the data returned from getImageData from the same Firefox browser/machine, but not in private mode.

Comment 5

[Evan Nowak]

Comment on attachment 9376065 [details]
Private result - wrong.png

Here is the incorrect result in a private window. The transparent blue plane labels are missing or cut off, as is the Z axis label.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox::Private Browsing' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 7

[1]

We have also encountered this bug, we are using the data from getImageData in a script which now fails due to the data differences. We also can only observe the wrong data in a Private Browsing tab.
The bug is not present anymore in current Firefox Developer edition, so it seems to have been fixed between v122.0 and v123.0b1.

Comment 8

[Robert Longson [:longsonr]]

See https://support.mozilla.org/en-US/kb/firefox-protection-against-fingerprinting and bug 1816189

Comment 9

[Evan Nowak]

Interesting. This is intentional, then.

FWIW, I was never able to reproduce the behavior on my M1 Mac in private mode, even after fiddling with the fingerprinting settings in Firefox 122. Not sure why it isn't consistent across platforms.

If this is the behavior in Firefox going forward, I'm fine having this bug resolved. (Is WONTFIX appropriate?)

Comment 10

[1]

If this behaviour is intentional, how should we handle production code that now breaks because it relied on getting the correct image data?
Is there a browser API we can check to see if the Fingerprint Protection is active, so we could at least warn users that our site might be broken because of this. Note that the settings linked above do instruct the user to disable it when a site is broken, but in our case the users were not aware that this setting even existed because it was set by the managing organisation.

Comment 11

[Rares Doghi, Desktop QA]

I was able to reproduce this issue as well and get a regression range for it:

211:23.53 INFO: Last good revision: e5d78a9ddcb4f26912e6e8ef797cb6556599b9b8
211:23.53 INFO: First bad revision: eae33625ba1ee4a773b3c378279f52e10f785e66
211:23.53 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=e5d78a9ddcb4f26912e6e8ef797cb6556599b9b8&tochange=eae33625ba1ee4a773b3c378279f52e10f785e66

Looks like Bug 1852541 is the culprit here.
@Tom can you please take a look at this issue ?

Comment 12

[Tom Ritter [:tjr]]

(In reply to 1 from comment #10)

If this behaviour is intentional, how should we handle production code that now breaks because it relied on getting the correct image data?
Is there a browser API we can check to see if the Fingerprint Protection is active, so we could at least warn users that our site might be broken because of this. Note that the settings linked above do instruct the user to disable it when a site is broken, but in our case the users were not aware that this setting even existed because it was set by the managing organisation.

So this behavior is generally intentional. I'm not sure why it didn't manifest in Evan's Mac - it is random, so between session restarts you might get lots of random noise or much less. We do try to put some in all the time though.

Fingerprinting Protection (FPP) is not designed to be directly queryable, but it can be inferred from side channels. You could render a known pattern to a canvas (it'd need to be a pattern, a constant color does not have noise added) and look for the noise.

Although not queryable, it is disable-able. A user can disable Enhanced Tracking Protection and that will disable the canvas noise. You could warn the user and provide instructions for how to handle it.

Presently, FPP is enabled in PBM and when users have opted into ETP Strict.