1506680 - Baltimore Sun & Boston Globe detect whether you're in Private Browsing mode, & put up a paywall if so

Status: RESOLVED DUPLICATE of bug 781982

(Firefox :: Private Browsing, defect)

Description

[Daniel Holbert [:dholbert]]

STR:
 1. Open a private browsing mode window (File|New Private Window)
 2. Visit https://www.baltimoresun.com/business/bs-md-lime-recall-20181112-story.html or any other Baltimore Sun article

ACTUAL RESULTS:
The article won't load -- I get a popover that says:
> You're using a browser set to private or incognito mode.
> Only subscribers can read articles in this mode.
> To proceed, log in or SUBSCRIBE NOW!

EXPECTED RESULTS:
The site shouldn't have been able to detect that I was in Private Browsing mode.

(At least, I recall that being a goal of PB mode at one point; not sure if it's attainable or not.)

Comment 1

[Daniel Holbert [:dholbert]]

Note: in non-private-browsing windows, I can dial up Firefox Nightly's built-in content blocking to its maximum level on about:preferences, and the article still loads just fine. In particular, I made these selections without breaking the article:
 Choose what to block:
   Trackers: always
   Third party cookies: all third party cookies
   Send "Do Not Track": always

So, these ^^ aren't the signals that the site is using to identify PB mode & to put up the paywall.

Comment 2

[Daniel Holbert [:dholbert]]

Apparently the Boston Globe does this as well. Sample article there (which loads fine in a normal window but hits a paywall in a private browsing window):
https://www.bostonglobe.com/metro/2018/11/12/you-listen-christmas-music-november-you-are-psychopath/0im3VBKmdqyhD11GsH9vuM/story.html

And here's a (working just fine :)) article from last year, describing the Globe's behavior on this:
https://arstechnica.com/information-technology/2017/05/boston-globe-website-no-longer-lets-you-read-articles-in-private-mode/

That Ars article says that Local Storage API seems to be what the Globe was testing for (because the paywall comes up if you simply disable local storage).  Perhaps it'd be possible for us to add a stubbed-out local storage option to thwart this method of detecting a user's private/non-private status?

Comment 3

[Daniel Holbert [:dholbert]]

(In reply to Daniel Holbert [:dholbert] from comment #2)
> That Ars article says that Local Storage API seems to be what the Globe was
> testing for (because the paywall comes up if you simply disable local
> storage).  Perhaps it'd be possible for us to add a stubbed-out local
> storage option to thwart this method of detecting a user's
> private/non-private status?

Actually, I'm not 100% sure that this hint from Ars is actually accurate, because:
 (1) LocalStorage still seems to work in PB mode, at least in this testcase:
https://bug1410701.bmoattachments.org/attachment.cgi?id=8920859 
 (2) I was still able to load the boston globe article in "normal" windows after turning off the pref dom.storage.enabled (which I think controls localstorage).

So I don't know for sure what these news sites are using to test for PB mode.

Comment 4

[Johann Hofmann [:johannh]]

FWIW the site also geo-blocks european countries :|

To figure out the PB mode maybe they test for IndexedDB, which should be disabled I think.

Comment 5

[Daniel Holbert [:dholbert]]

(In reply to Johann Hofmann [:johannh] from comment #4)
> To figure out the PB mode maybe they test for IndexedDB, which should be
> disabled I think.

Nope, that's not it either. I can load articles from both sites, in a normal (non-PB) browser window with these prefs set (plus a restart for good measure):
 dom.indexedDB.enabled = false
 dom.storage.enabled = false

Comment 6

[Daniel Holbert [:dholbert]]

OK, the JS in question is here:  https://www.tribdss.com/meter/baltngux.js

At the end there, you can see a JSON blob with "Incognito Paywall" and "privateMode" as member-data.  And privateMode is actually a reference to a JS function defined further up:
      privateMode: function(callback) {
        var isPrivate, ieMatch = /(?:MSIE|rv:)\s?([\d\.]+)/.exec(ua), tries = 3,
          pass = function() { isPrivate = false; }, fail = function() { isPrivate = true; };

        if ('private' in trb) {
          isPrivate = trb.private;
        } else if (window.webkitRequestFileSystem) {
          webkitRequestFileSystem(TEMPORARY, 1, pass, fail);
        } else if (/Firefox/.test(ua) && window.indexedDB) {
          var db = indexedDB.open('test'); db.onsuccess = pass; db.onerror = fail;
        } else if (/Edge/.test(ua) || (ieMatch && parseInt(ieMatch[1], 10) >= 10)) {
          isPrivate = !window.indexedDB;
        } else if (/Safari/.test(ua) && window.localStorage) try {
          isPrivate = !openDatabase(null, null, null, null);
          localStorage.setItem('test', 1); localStorage.removeItem('test');
        } catch (e) { isPrivate = true; }

        (function retry() {
          if (isPrivate != null) callback(trb.private = isPrivate); else tries-- ? setTimeout(retry, 50) : callback(false);
        })();
      }
    },

The bit of that that's relevant is the "Firefox" bit, combined with pass/fail functions defined up above. They are testing for the IndexedDB API, and they decide you're in private browsing mode if the API is present but indexedDB.open() triggers an onerror handler.

If I pref off indexedDB entirely (per the pref flip "dom.indexedDB.enabled = false" in previous comment), then the articles load just fine *in normal AND in private mode* because the "window.indexedDB" check fails here.

Comment 7

[Daniel Holbert [:dholbert]]

So johannh was absolutely right that they test for indexedDB -- and specifically, they're testing for "indexedDB exists but triggers an onerror handler for calls to .open()", and that makes them classify us as being in PB mode.

Comment 8

[Luke Crouch [:groovecoder]]

Just re: the priority of this bug ... from the wiki page:

Can an online adversary detect private browsing mode?

    From a purely technical standpoint, there are a few weak spots in the platform that make it impossible to hide private browsing mode effectively. Also, over the years, it has become even more difficult to protect from new threats (e.g., fingerprinting) AND hiding the protections themselves.


Not to say that we can't or shouldn't work on this ... just to say that we don't treat PBM detection vulnerabilities as "P1".

Comment 9

[Daniel Holbert [:dholbert]]

FWIW, https://www.bild.de/ does this as well. (If you view that site in PB mode, their front page directs you to https://www.bild.de/wa/ll/bild-de/privater-modus-unangemeldet-54578900.bild.html which is a "please-don't-view-us-in-PB-mode" wall. )

Just as noted at the end of comment 6, you can work around this by preffing off indexedDB entirely (and then it works in normal and PB mode).

Comment 10

[Mark Baysinger]

There's a tracing bug for this: https://bugzilla.mozilla.org/show_bug.cgi?id=1366318

Comment 12

[Mark Baysinger]

The New York Times, (nytimes.com) is doing this, now, too! This issue deserves a higher priority!

Comment 13

[Gingerbread Man]

Washington Post does this as well, according to bug 1558981.

Comment 15

[Randell Jesup [:jesup] (needinfo me)]

Since it's indexDB, this is a dup of bug 781982