Run browser/components/originattributes/test/browser tests with dfpi enabled
Categories
(Core :: Privacy: Anti-Tracking, task, P2)
Tracking
()
People
(Reporter: manuel, Unassigned)
References
Details
browser/components/originattributes/test/browser/browser.toml:66 (moved from browser.ini)
# We don't want to run tests using BEHAVIOR_REJECT_TRACKER_AND_PARTITION_FOREIGN
# (5 - aka Dynamic First Party Isolation) yet.
prefs = ["network.cookie.cookieBehavior=4"]
What this bug is asking for
browser/components/originattributes/test/browser/ is our coverage for origin-attribute-based isolation. Twenty-two browser-chrome tests load a third-party resource under two different first parties and assert whether the resulting state is shared or isolated, across: cookies, HTTP cache, image cache, CacheAPI, blob URLs, BroadcastChannel, localStorage, SharedWorkers, favicons, HTTP auth, client certs, permissions, postMessage, window.opener restrictions, and sanitization.
The whole directory is pinned to a non-default cookie configuration:
# Bug 2038682: We don't want to run tests using BEHAVIOR_PARTITION_FOREIGN
# (5 - aka Dynamic First Party Isolation) yet.
prefs = ["network.cookie.cookieBehavior=4"]
4 is BEHAVIOR_REJECT_TRACKER — block known trackers, but let ordinary third parties share state across sites. The shipping default is 5, BEHAVIOR_PARTITION_FOREIGN — dFPI / Total Cookie Protection, where every third party gets a separate storage bucket per top-level site.
That pin was added by bug 1634021 in May 2020, when browser_broadcastChannel.js failed under dFPI. It was a blanket, directory-wide workaround, and it has been in place for six years.
The irony is that dFPI is origin-attribute isolation. FPI sets the firstPartyDomain attribute; dFPI sets partitionKey. This directory is precisely where dFPI's isolation guarantees ought to be covered, and it is the one place that explicitly opts out of them.
Why this matters
Every isolation guarantee asserted here is asserted against a cookie behaviour no default profile has used since Firefox 86. A regression that only manifests under dFPI — the configuration essentially all users run — would not be caught by this suite. The coverage looks strong and is measuring the wrong thing.
Why it is not a one-line change
IsolationTestTools runs each participating test in three modes, and derives the expectation from the mode alone:
let shouldIsolate =
aMode !== TEST_MODE_NO_ISOLATION && tabSettingA !== tabSettingB;
...
result = shouldIsolate ? resultA !== resultB : resultA === resultB;
Working through the three modes with the two tabs set to different first parties (https://example.com vs https://example.org):
| Mode | Pref | Asserts | Under cookieBehavior=5 |
|---|---|---|---|
TEST_MODE_FIRSTPARTY |
privacy.firstparty.isolate=true |
isolated (resultA !== resultB) |
still isolated — firstPartyDomain already separates the tabs. Unaffected. |
TEST_MODE_NO_ISOLATION |
privacy.firstparty.isolate=false |
shared (resultA === resultB) |
premise invalid — dFPI partitions the third party by top-level site, so the two tabs cannot share. |
TEST_MODE_CONTAINERS |
privacy.userContext.enabled=true |
isolated | still isolated — userContextId separates the tabs. Unaffected. |
So flipping the pref does not surface incidental breakage. One of the three modes encodes a pre-dFPI world model: "FPI off and containers off implies no isolation at all." Under the shipping default that statement is simply false. The mode has to be redefined before the pin can come off, and its name is actively misleading for anyone reading the suite today.
This bounds the work usefully: the expected damage is confined to one mode of the 13 framework-driven tests. The other 9 are standalone, and most of those are browser_firstPartyIsolation* tests that enable FPI themselves — they inherit the pin but should not depend on it.
One nuance worth checking early: network-state partitioning is no longer pref-gated at all. The master privacy.partition.network_state pref is gone from the tree, leaving only privacy.partition.network_state.connection_with_proxy. So the cache-oriented tests (browser_cache.js, browser_imageCacheIsolation.js) may already be partitioned independently of cookieBehavior, which means the "no isolation" premise may already be strained for them today. Their current expectations should be read carefully rather than assumed correct.
Recommended approach
1. Measure before designing. Push to try with the prefs line removed and nothing else changed — full directory, --rebuild 5, across platforms. This turns the analysis above into a concrete failure list, and it is cheap.
2. Check the prediction. Failures should cluster in the "no isolation" mode. If the FPI or containers modes also fail, that is a separate and more interesting finding — either a real isolation gap under dFPI or a harness assumption — and should be triaged on its own before any test rewriting starts.
3. Redefine the third mode. Two designs:
- Reinterpret it as the default-configuration mode: assert dFPI's partition-by-top-level-site behaviour, i.e. isolated for anything dFPI partitions, and rename it (
TEST_MODE_NO_ISOLATION→ something likeTEST_MODE_DFPI, along withTEST_MODE_NAMES) so it does not mislead. - Split it, keeping an explicit
cookieBehavior=4legacy mode alongside a new dFPI mode.
Recommend the first. The point of the bug is to assert what ships; the second option preserves coverage of a configuration almost nobody runs, at the cost of doubling runtime for this suite.
4. Make expectations per-mechanism, not global. dFPI does not partition everything uniformly, and the suite already has exceptions — browser_blobURLIsolation.js explicitly sets privacy.partition.bloburl_per_partition_key=false. A single boolean derived from the mode will not survive contact with all 13 tests. Better to let each test declare what it expects under dFPI (a capability flag passed into runTests) than to keep widening one shared expression.
5. Remove the pin last, and narrowly. Rather than one patch flipping 13 tests × 3 modes at once, add a temporary per-test opt-in so tests can be migrated and landed incrementally, then delete the [DEFAULT] pin once the list is empty. Any surviving per-test cookieBehavior=4 pin should carry a comment and a bug reference, so the next person does not inherit another undated blanket workaround.
Success criterion: no cookieBehavior pin in [DEFAULT], the suite green under the shipping default, and the third mode named for what it actually tests.
The mode-by-mode table above is derived from reading the harness, not from a try run — I have not pushed one. Step 1 would confirm or correct it.
Follow-up: I went through the 13 framework-driven tests individually to scope the work, and found something that materially changes the urgency — the TEST_MODE_NO_ISOLATION mode is not merely at risk from dFPI, it has already been partly abandoned.
Two tests have silently stopped asserting anything in that mode
// browser_httpauth.js
function compareResult(shouldIsolate, a, b, mode) {
if (mode == TEST_MODE_NO_ISOLATION) {
todo(false, "This used to depend on no http partitioning");
return true;
}
return shouldIsolate ? a !== b : a === b;
}
// browser_imageCacheIsolation.js
function doCheck(shouldIsolate, a, b, mode) {
...
if (mode == TEST_MODE_NO_ISOLATION) {
todo(false, "This test depended on no network partitioning");
return true;
}
return shouldIsolate ? gHits == NUM_ISOLATION_LOADS : gHits == NUM_CACHED_LOADS;
}
This confirms the nuance I raised in the previous comment, and goes further than I expected. Network-state partitioning already invalidated the "two first parties share third-party state" premise for these two tests, and rather than redefining the mode, the assertions were stubbed out with todo() and the mode left in place. So a third of the mode's coverage is already not being checked, and the cookieBehavior=4 pin is no longer even sufficient to keep the premise true — it only defers the same reckoning for the storage-backed tests.
That reframes the bug slightly. It is not "switch the suite to the shipping config when convenient"; it is "the mode this suite is built around has been decaying for years, and each decay has been absorbed locally instead of fixed."
Per-test scope
| Category | Tests | Work involved |
|---|---|---|
| Default comparison; dFPI clearly partitions | browser_cookieIsolation.js, browser_localStorageIsolation.js, browser_cacheAPI.js, browser_broadcastChannel.js, browser_sharedworker.js |
Mechanical — the mode's expectation flips from shared to partitioned |
| Default comparison; needs a decision | browser_blobURLIsolation.js, browser_clientAuth.js |
See below |
| Custom compare with real logic | browser_cache.js |
Most intricate check in the suite |
Already stubbed with todo() |
browser_httpauth.js, browser_imageCacheIsolation.js |
Should be restored to real assertions, not left as-is |
Compare func is () => true, assertions internal |
browser_permissions.js, browser_sanitize.js |
Unaffected by the mode change |
Where the risk actually sits
browser_clientAuth.js. Whether remembered client-certificate decisions are keyed by partitionKey is not evident from the test, and guessing wrong means asserting the wrong security property rather than merely getting a red test. This one likely needs an answer from someone who owns that code rather than an inference from observed behaviour.
browser_cache.js. Its doCheck enumerates cache entries across firstPartyDomain × userContextId × private × anonymous. Extending that matrix to partitionKey is the fiddliest change in the set, and it is the test most likely to fail in a confusing way and get "fixed" incorrectly.
The failure mode of the whole exercise is more todo()s. The suite already demonstrates the temptation twice. If this lands with stubs for the awkward cases, the bug closes and the coverage ends up worse than today, because at least the pin documented the gap honestly.
Implication for the approach
One thing that keeps this tractable: runTests already accepts an aCompareResultFunc, so the per-test escape hatch exists and no framework redesign is needed. But that also argues strongly for landing this as a short series rather than a single patch flipping 13 tests across 3 modes at once — with everything changing together it is very hard to distinguish a real isolation bug from a stale test expectation when try comes back red.
Rough shape: five tests near-mechanical, two needing a product decision, one (cache) a real piece of work, two needing un-stubbing, plus the mode rename. Small series, not a one-liner.
As before, this is from reading the code — no try push yet, and step 1 of the previous comment could still change the picture.
Description
•