Open Bug 1246615 Opened 8 years ago Updated 2 years ago

Make Firefox explicitly aware when running a profile from future.

Categories

(Toolkit :: Startup and Profile System, enhancement)

Product:
Toolkit
Component:
Startup and Profile System
Type:
enhancement

Tracking

()

People

(Reporter: jujjyl, Unassigned)

References

(Depends on 2 open bugs, Blocks 2 open bugs)

Details

(Whiteboard: [tw-dom][DevAdvocacy]) 

Firefox user profiles are forward compatible and are migrated to new Firefox version when user upgrades Firefox. Developers routinely toggle between Firefox channels (stable->dev->nightly->beta->stable) when debugging issues, and currently when going back in time, there may be a change that subtly breaks the profile from working in older Firefoxes when it is briefly run in Firefox Nightly.

Examples of such issues are https://bugzilla.mozilla.org/show_bug.cgi?id=1244717 and likely also https://bugzilla.mozilla.org/show_bug.cgi?id=1128163 and https://bugzilla.mozilla.org/show_bug.cgi?id=1194170.

We should not allow Firefox to silently start failing if user runs a profile "from the future" in an older version of Firefox, but we should instead track such incompatible changes and tell the user that when a profile is no longer compatible with the version of Firefox that the user is running in, or if that is too much work or difficult to do reliably, then track which versions of Firefox a profile has been run with, and immediately if it is run with a profile from the future, offer a warning ribbon that the user is doing something unsupported.

This would save a lot of frustration, cryptic debugging and can't-reproduces. Just recently we were debugging an issue cross operating systems, cross teams, cross partner organizations with 10+ developers scratching their heads, including Firefox developers, with the result being that a db schema had changed, which caused Firefox to start silently failing.

Also it would better warrant the scenarios where a developer wants to mark a bug WONTFIX if it does not occur in a clean profile, since we can rule out if it was a result of running a profile from future.
Sounds like a good idea to me.
This also sounds good to me too.

I've received many such bug reports in the past, mostly related to IndexedDB throwing mysterious errors, when a profile tries to move back to an older version, such as:

https://bugzilla.mozilla.org/show_bug.cgi?id=1117129#c2
https://bugzilla.mozilla.org/show_bug.cgi?id=1118816
Blocks: 1118816
and bug 1174113 in my case, also related to IndexedDB.

(Maybe bug 1143003 was all about the same cause too?)
If anyone has cycles, displaying a browser-wide warning for future profiles would really help people working with service workers.
Whiteboard: [tw-dom]
Is this a DOM feature or a Firefox front-end work? Do we need a spec from Firefox UX?
No worries, I thought UI_VERSION in nsBrowserGlue._migrateUI is equal to major version. We probably need something that reads major version number from elsewhere in this bug.

https://dxr.mozilla.org/mozilla-central/rev/3e04659fdf6aef792f7cf9840189c6c38d08d1e8/browser/components/nsBrowserGlue.js#1801-1816
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #5)
> Is this a DOM feature or a Firefox front-end work? Do we need a spec from
> Firefox UX?

Maybe a bit of both?  I think we should do something initially and then let UX approve or prettify it later.
Moving to the correct component. However, we intend for Firefox profiles to be basically backwards and forwards compatible at least across the four release cycles nightly/aurora/beta/release, and in general we strive for longer than that. In general code that makes incompatible changes should be prepared to change filenames or otherwise fork the data so that older builds don't break things.
Component: Profile: Migration → Startup and Profile System
Product: Core → Toolkit
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #8)
> Moving to the correct component. However, we intend for Firefox profiles to
> be basically backwards and forwards compatible at least across the four
> release cycles nightly/aurora/beta/release, and in general we strive for
> longer than that. In general code that makes incompatible changes should be
> prepared to change filenames or otherwise fork the data so that older builds
> don't break things.

I'm sorry, I don't understand this.  Can you explain what you expect storage APIs like IDB and Cache to do when reading a profile from the future?

1) Wipe the users data and start over?
2) Start over in a new dir, but keep the future data in a different dir?  So effectively like a wipe, but if they move to the newer version again they get different data suddenly?  How does this ever get cleaned up or merged?

I don't understand how this can work in practice and I've never heard this requirement before.
Flags: needinfo?(benjamin)
I think the only sane thing to do here would be to treat it like running out of quota.  In those cases the QuotaManager deletes the entire origin storage atomically.  That ensures that sites don't see partial state.

So we could add some mechanism for individual storage providers like IDB and Cache to tell QM that they failed to open due to bad disk state.  Then QM could wipe the storage area so the site could start over.

Jan, what do you think?
Flags: needinfo?(jvarga)
(In reply to Ben Kelly [:bkelly] from comment #9)
> (In reply to Benjamin Smedberg  [:bsmedberg] from comment #8)
> > Moving to the correct component. However, we intend for Firefox profiles to
> > be basically backwards and forwards compatible at least across the four
> > release cycles nightly/aurora/beta/release, and in general we strive for
> > longer than that. In general code that makes incompatible changes should be
> > prepared to change filenames or otherwise fork the data so that older builds
> > don't break things.
> 
> I'm sorry, I don't understand this.  Can you explain what you expect storage
> APIs like IDB and Cache to do when reading a profile from the future?

Yeah, I can't imagine how to do this with storage APIs.
(In reply to Ben Kelly [:bkelly] from comment #10)
> I think the only sane thing to do here would be to treat it like running out
> of quota.  In those cases the QuotaManager deletes the entire origin storage
> atomically.  That ensures that sites don't see partial state.
> 
> So we could add some mechanism for individual storage providers like IDB and
> Cache to tell QM that they failed to open due to bad disk state.  Then QM
> could wipe the storage area so the site could start over.
> 
> Jan, what do you think?

Does this mean that if I go back e.g. from Nightly to an older firefox which uses incompatible database schema, all data is deleted and it won't be available even in Nightly ?
That doesn't sound good to me. The solution you described introduces a new problem when someone suddenly loses all his important data.

Two other solutions come to my mind:
1.Fork user data with each incompatible database schema (something like this bkelly already described in comment 9).
This is probably the best solution for users who like to switch between different Firefox versions, but it needs much more space on disk and it's also questionable how to clean it in future.

2. Launch an alert dialog when future incompatible profile is used (probably at firefox startup)

and maybe third would be to alert when a future database is used, but I'm not sure if it's worth it. We use IndexedDB internally.
Flags: needinfo?(jvarga)
To be clear, I don't think we should support going backwards from one channel to a previous one on the same profile.  I think we should warn users not to do that.

If the product managers insist on supporting that transparently, though, I think wiping storage when there is a schema change is the only sane way to deal with the problem.

I'm strongly opposed to trying to fork directories, merge entries, etc.  That's a lot of complexity for what is essentially a rare corner case.  Its mainly power users moving between channels.  They can manage separate profiles.

Or better yet, we should just default to different profiles for each channel unless the user specifies a specific profile.
It sounds like we're talking about some particular backwards-incompatible change, rather than a general policy. Can you describe what thing in particular is changing incompatibly?

In general for testing purposes we have specifically asked users who see problems on release or beta to try various nightlies instead, and it's common that we need the same profile data and addons to reproduce. So I continue to argue against the notion of having per-channel profiles by default. But ultimately that's Jeff's decision, not mine.

The most common solution that we have used in the past for a format change (e.g. when we switched from bookmarks.html to the places database) is we kept the old bookmarks file around and so if you ran an old version you got your older bookmarks and places was the "new hotness". It's hard without details for me to understand whether that would or wouldn't work in this case.
Flags: needinfo?(benjamin)
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #14)
> It sounds like we're talking about some particular backwards-incompatible
> change, rather than a general policy. Can you describe what thing in
> particular is changing incompatibly?

Database schema changes that enable new functionality/data storage that is visible to script.  This is not comparable to the (then-)local-only bookmark storage example.

This becomes very complicated because script state may be distributed amongst multiple Gecko subsystems, making "just use the old thing when using Firefox N-1 on a Firefox N profile" untenable.  For example the sw-toolbox project uses a combination of the DOM Cache and IndexedDB to manage its caches (https://github.com/GoogleChrome/sw-toolbox/pull/37).  In order for that to work out, the Quota Manager (or other mechanism) needs to checkpoint the entirety of local state for an origin whenever any piece upgrades.

It also assumes the local state has no impact on server state, which means assuming all script operations are idempotent from a server perspective.  For example, if using an email app using offline storage in IndexedDB with the following: sequence of events: "write angry email to boss, put in outbox with one hour timeout", "upgrade firefox", "delete message from outbox", "decide new version of Firefox is bad, go back to old version".  We don't want the angry email sent.  Similarly, for the bookmarks example you give, I would imagine many naive implementations of bookmark replication (like Firefox Sync) would not do well with the same client/GUID bouncing between two completely independent states.

The best thing we can do in these cases is say "sorry, this origin got upgraded, you need to go back to the more recent version of Firefox you use".  I agree it can be better for the user if we're saying that because something did get upgraded rather than us just noticing the version numbers don't match.

To that end, what about using the observer service to notify on the topic "profile-from-the-future-broke-this-origin" with the principal as the subject and the subsystem (ex: "indexeddb", "dom-cache") that broke as the someData?

This would enable an info-bar to be displayed on impacted tabs and possibly have a link/button that lets the user trash the entirety of storage for the origin as an explicit choice[1].

1: Assuming we can guarantee all subsystems can delete data despite schemas/etc. being from the future.  The observer notification might need to pack a boolean in there to convey whether nuking is an option.  For example, if the cookie database saw a schema from waaaaaay in the future (because the old build is REALLY old), it might just generate the notification for everything.
Another backward incompatible storage change just landed.  See bug 1246615.
See Also: → 1195930
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #8)
> Moving to the correct component. However, we intend for Firefox profiles to
> be basically backwards and forwards compatible at least across the four
> release cycles nightly/aurora/beta/release, and in general we strive for
> longer than that. In general code that makes incompatible changes should be
> prepared to change filenames or otherwise fork the data so that older builds
> don't break things.

Even if that's the intention, I don't think it's the case in practice. I've
lost count of the number of issues I've seen over the last few weeks caused by
people using the same profile with multiple versions of Firefox. Bug 1272446
would solve most of the issues I've seen recently, but I don't think it would
make a huge dent in the issues most users see.

So if we do intend to maintain forward and backward compatibility across all
channels, and continue to use the same profile for all channels by default, I
think that a user-visible warning still makes sense. We may not intend for the
to be issues, but I don't think we can reasonably expect there to be no issues.
When I was thinking about jumping from Beta to Aurora, my topmost concern was whether or not I could go back if (unfortunately) Aurora become too unstable. I decide to made the move because I realized can always use Sync to start a new profile in Beta again.

We need to answer the question here from product strategy point of view. We got new campaign spinning up to drive for more Aurora and Nightly users -- we would need to reveal the commitments needed to make that switch, and also make sure you don't need to take a day off on Internet if Aurora/Nightly broke for the day.

Pascal, you are promoting Nightly currently. Would you like to shed some light here?
Flags: needinfo?(pascalc)
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo; OOO and on leave until July) from comment #18)
> 
> Pascal, you are promoting Nightly currently. Would you like to shed some
> light here?

I think a better solution would be to have separate profiles, that is bug 1265322. That would avoid a lot of troubleshooting problems with profile corruptions and our testers could focus on actual bugs to report. That would make it also easier to promote the use of Nightly as a secondary browser alongside Release. That would also allow users that don't know how to create and open Firefoxes with separate profiles from the command line to be able to test if a bug or regression they are seeing on Release is already fixed on Nightly.
Flags: needinfo?(pascalc)
bug 1265322 was marked as a duplicated of bug 1255740. Also, CCing Marcia and Sylvestre on this bug because I think the result of this discussion could have a positive impact on the efforts to improve quality on our prerelease channels.
Separate profiles like dev-edition defaults to would be great!

It would still be nice to have some warning in the profile manager to help power users.  Ideally the profile manager would show you what version the profile has been upgraded to.
Another change that will prevent running upgrades profiles in older versions.
See Also: → 768074
Benjamin, even though I agree with the accuracy of comment 8 from the historical point of view, I think the current situation is unacceptable and we're not winning anything by not solving this problem soon, and we keep destroying the users' profile data, especially those users who engage with us to help and try running newer builds for testing purposes.

The situation is so bad that bkelly pointed out to me that I myself have screwed up and landed backwards incompatible changes, for example bug 1251872.  Without anything enforcing the backwards compatibility of our profiles, I'm thinking that we may want to bite the bullet and do something about this, for example show a prompt and refuse to start the browser.

Would you accept a patch that does something along those lines?
Flags: needinfo?(benjamin)
Personally I view this similar to other changes we've made to let us "go faster".  We have minimized our extension API surface area, we have deprecated plugins, etc.  These things reduce features for users, but improve our stability and let us iterate on the browser faster.

Disallowing downgrades is a loss of a feature for a small set of users.  But it will let us to "go faster" in storage related code.  Writing and *testing* all these downgrade paths is a huge amount of work.  (As Ehsan noted, we don't realistically test this stuff at all today...)
Yeah, what bkelly said.
Note, we also have a robust account sync mechanism today.  We could recommend that people use that feature instead of sharing profiles across channels.
Jeff and I are discussing, will come back next week with better discussion. We cannot do this without a larger plan about mozregression, profiling, and bug reproducibility.
Do we have telemetry about how frequently this happens (using a future profile with a previous version)?
Flags: needinfo?(jvarga)
(In reply to Andrew Overholt [:overholt] from comment #30)
> Do we have telemetry about how frequently this happens (using a future
> profile with a previous version)?

We don't have a conclusive way to detect it today.  Right now we only have a wide variety of places where we notice "stuff is not as expected!".  We need a "updated-to-version" value in the profile first, and then we can start tracking how often it goes backwards.
(In reply to Ben Kelly [:bkelly] from comment #31)
> (In reply to Andrew Overholt [:overholt] from comment #30)
> > Do we have telemetry about how frequently this happens (using a future
> > profile with a previous version)?
> 
> We don't have a conclusive way to detect it today.

We have the extensions.lastAppVersion and extensions.lastPlatformVersion preferences.
(In reply to Kris Maglione [:kmag] from comment #32)
> We have the extensions.lastAppVersion and extensions.lastPlatformVersion
> preferences.

These aren't quite what we need.  They only record the last version, but will then overwrite the newer version when opened in an older browser.

We need FF50 -> FF51 -> FF50 to always remember its "max version" was 51.
(In reply to Ben Kelly [:bkelly] from comment #31)
> We need
> a "updated-to-version" value in the profile first, and then we can start
> tracking how often it goes backwards.

Not sure you're explicitly suggesting this, but a single value seems tricky.  If we have a single manually updated strictly-increasing constant somewhere, it seems like we'll forget to update it reliably.  Using semver-style checking based on automated buildid/channel information to complain about going back to a prior release seems like it detects a different scenario than the problem we're dealing with here.  It's only a problem downgrading across schema changes.  (Although knowing when people use a prior day's build/etc. could be interesting on its own, I think it'd be an excessively harsh criteria for throwing up a warning or refusing to run, etc.)

I'd suggest rooting a preferences branch someplace like "storage.schemaVersions" (we already have storage.vacuum for the mozStorage autovacuum stuff) so we'd have "storage.schemaVersions.dom.indexedDB.{db,structuredclone}"[1], "storage.schemaVersions.dom.quota.{storage,metadata}", etc.  Either we augment the preferences API or create a small observer-notification-generating service helper around it that allows us to leverage whatever constant we already update in our code when we change the schemas.  That way there's no extra step once a given storage mechanism is hooked up.  (Although code could also maintain a separate downgrade-not-safe version if there are situations where they think downgrades are safe and are willing to support that.)

We could hook it up to telemetry to start with and a warning later.  And the telemetry might even report which storage schema was the source of the problem, so we can have informed discussions in the future about whether we're not future-proofing our storage enough and/or whether people are being impacted.

Also: We might also have the logic set a preference surfaced in about:support that indicates when a timewarp last occurred, wall-clock-wise, so if someone reports a problem and it turns out they just timewarped their profile, maybe that was the source of the problem.  Even if it's standard practice to suggest trying a different build, the fact that the user crossed a schema version change boundary is probably still relevant to any problem diagnosis.

1: Although IndexedDB has a static_assert on the structured clone version and appropriately bumps its SQLite db schema version, I think it would be useful to track the structured clone change separately if we're trying to figure out what is forcing our hand from telemetry.
I think we should use the firefox version number.  We always set a version for the release, so it seems unlikely we will forget to update it.  This is the number the user understands, so I think we should base our checking on it.  Otherwise users will be confused about when they can or cannot open a profile.

The only issue with using FF version would be not understanding schema breakage within a version number.  That should only affect nightly, though.

Just my vote, anyway.
(In reply to Ben Kelly [:bkelly] from comment #33)
> (In reply to Kris Maglione [:kmag] from comment #32)
> > We have the extensions.lastAppVersion and extensions.lastPlatformVersion
> > preferences.
> 
> These aren't quite what we need.  They only record the last version, but
> will then overwrite the newer version when opened in an older browser.
> 
> We need FF50 -> FF51 -> FF50 to always remember its "max version" was 51.

As far as collecting telemetry goes, they should be enough to at least give us a basic understanding of how often this happens.
Note that even if this doesn't happen "often", there's still an argument to be made about us not causing dataloss issues for users who are going out of their way to assist us, no matter how few their numbers are...
FYI, bug 1321550 is another disk format change that will affect trying to downgrade profiles.  In this case, however, we won't see a "corrupt" schema.  Instead that bug will collapse file:// URIs with fragments into a single origin in newer firefox, but have separate origins per fragment in older firefox.  So data will confusingly appear/disappear from the users perspective if they try to move a profile forward and backward.
See Also: → 1321550
What I'd recommend we do as a minimum first step is that whenever a profile is loaded up, the distribution channel (stable/beta/deved/nightly) + build date of the Firefox browser in question is recorded to that profile, and if Firefox is starting up on a profile that has a previous record of having been opened on a Firefox from a different channel, or a future build date, we show a banner at the top of the browser page telling 

"This profile was last opened in Firefox <edition> <build date>, but is now opened in an older Firefox <edition> <build date>. Downgrading profiles is currently not supported and may run into unexpected issues. Visit <mdn doc page on how to create, manage and launch a specific profile> for more information."

but still let the browser run on the given profile. This should be a very minimal amount of work, and will precisely capture the current unsupported scenario and visibly flag to the user that they might now be getting something bad.

The comments here read that doing more precise stuff like downgrading profiles back, or actively tracking backwards breaking changesets to be able to precisely tell when compatibility was lost, will be much more work, so doing a blind channel & build date comparison should be a safe minimum starting point that will be automatic and not add continuous maintenance burden.
Flags: needinfo?(jvarga)
See Also: → 1345120
See Also: 1345120 → 1339081
(In reply to Benjamin Smedberg [:bsmedberg] from comment #29)
> Jeff and I are discussing, will come back next week with better discussion.
> We cannot do this without a larger plan about mozregression, profiling, and
> bug reproducibility.

Benjamin, did you have a chance to come up with this plan yet?
See Also: → 1345760
I'm really sorry for the delays here. Jeff and I had alternating travel schedules and then I lost this.

Here is an analysis of the patterns of people downgrading: https://gist.github.com/bsmedberg/1af70857106bfe29128a0e8d0b656804 - this analysis was reviewed by chutten

TLDR:
users that switched channels at all: 0.48%
users that reverted to an older version: 2.55%
users that reverted to an older version, staying on the release channel: 2.19%

Downgrades are more prevalent than I thought, and channel-switching is not a primary factor in downgrades. So the ideas that we would mostly solve this problem by giving channels separate profiles appears to be false.

I'm going to set up a meeting about this tomorrow. I believe that a banner warning is essentially user-hostile in that there's no much a typical user can do about it.
Flags: needinfo?(benjamin)
(In reply to Benjamin Smedberg [:bsmedberg] from comment #42)
> Downgrades are more prevalent than I thought, and channel-switching is not a
> primary factor in downgrades. So the ideas that we would mostly solve this
> problem by giving channels separate profiles appears to be false.

But it will solve the problem like comment #37. We should not stop separate profiles just because it does not fix all problems.

> I'm going to set up a meeting about this tomorrow. I believe that a banner
> warning is essentially user-hostile in that there's no much a typical user
> can do about it.

Is it less user-hostile to leave user's profile broken silently? Doesn't "Refresh Firefox" fix the issue?
Given that the options seem to be:

- Break in undefined ways (the current situation)
- Do the work to silently throw away data ("hard downgrade") when opened in an older version
- Do the work to explicitly disallow opening profiles in older versions
- Show a warning to allow the user to make a decision

I feel that a warning is the least user-hostile option!

Perhaps I just opened the wrong Firefox version, or opened the wrong profile -- say I'm a developer who routinely uses Developer Edition, and I only keep release around for testing.

Or perhaps there's a minor bug in a new Firefox and so I'm downgrading until it's fixed (or maybe one of my add-ons isn't supported yet), but given the choice between some amount of data loss or _not downgrading after all_, I'll put up with some inconvenience for a week.
(In reply to Benjamin Smedberg [:bsmedberg] from comment #42)
> I'm going to set up a meeting about this tomorrow.

What was the meeting result?
Flags: needinfo?(benjamin)
The initial result is that RT is going to own next steps. That we all don't want to show a modal warning, but we don't yet understand enough about downgrade patterns to decide what the other options should be. pdol is talking with RT about it this week, and they will probably follow up with chutten to get more details out of the data.
Flags: needinfo?(benjamin) → needinfo?(rtestard)
See Also: → 1258596
From looking into what may cause users to downgrade it seems we can safely ignore users who downgrade through the stub installer or users who downgrade through app update because these systems are engineered to prevent this. We are left with 2 scenarios that need addressing in separate ways:
1 Users of multiple channels
This is a supported use case that has value, mostly for developers who suffer from profile corruption issues when using profiles from the future. Our goal is to make profiles backwards and forwards compatible although this is a hard goal to achieve and we need a way to at least inform users that they may be experiencing issues.
Using profiles per channel, deleting profile data or blocking users as they're about to open a profile from the future all seem inappropriate solutions that would likely create a worst user experience. I'll now engage with UX to come-up with a proposal to address informing users of a potential profile corruption.

2 Users of multiple release versions
It seems very likely (previous users research and common sense) that users who reverted to an older version, staying on the release channel (2.19% mentioned on Comment 42) are users with multiple release versions installed on their system. We don't have a good understanding of these users right now to understand whether it is a non issue or if we need a way to resolve this. The new install ping being specified (bug 1351393) looks to be a good place to collect install data that would help understand size and pattern of users with multiple release versions on their systems, I'll follow-up on bug 1351393 with requirements to address this.
Flags: needinfo?(rtestard)
The most recent casualty of this issue is Epic Games, who skipped shipping IndexedDB persistence support for Unreal Engine 4.16 release for WebAssembly exports, because they were bit by this behavior. Their QA had "done their diligence" and briefly used Firefox Nightly to test if another issue would have been fixed in upstream before reporting a new bug against Firefox. This broke IndexedDB from working for them in Firefox stable, resulting in their QA thinking that IndexedDB is broken in Firefox stable altogether and that they ran out of time to be able to ship IndexedDB support for Unreal Engine 4.16, but had to postpone to 4.17 release.

The troubles were caused in part by a related behavior of how IndexedDB currently acts when user profile is in such a bad state, marked down separately as bug 1355736. They had not been aware of the behavior in bug 1355736 nor by this bug 1246615.

Comment 44 approach, show a non-modal warning banner if running Firefox on a profile from the future, would have at least some > 0% probability of carrying the important information over to the user so that they will not run into mysterious breakages. At least in the above case, that might have saved time for Epic's QA so that they would have learned what is going on in time to ship. Or is the current plan to do something more involved?
(In reply to Jukka Jylänki from comment #48)
> The most recent casualty of this issue is Epic Games...

Thanks for the update. I think irrespective of what we're planning for developer edition, we should do this. Per-channel profiles seems to be much cheaper than making indexedDB future / past proof.
(In reply to Jeff Griffiths (:canuckistani) (:⚡︎) from comment #49)
> (In reply to Jukka Jylänki from comment #48)
> > The most recent casualty of this issue is Epic Games...
> 
> Thanks for the update. I think irrespective of what we're planning for
> developer edition, we should do this. Per-channel profiles seems to be much
> cheaper than making indexedDB future / past proof.

+1. This is also going to continue biting extension developers who test across multiple channels, especially when we migrate more extension storage APIs to be backed by indexedDB. Even if we have a way to notify them of the problem, this isn't the kind of thing developers should be regularly bitten by.
This just bit me. Ended up with weird errors from the service worker. Thread https://twitter.com/jaffathecake/status/854593874106482688
A relnote has been added on the fallout from this for Firefox 55beta, tracked on bug 1357428.

And in other examples of people getting burnt and new edge cases to be on the lookout for, the Devtools PM ran into this problem on a developer edition profile, as tracked in bug 1367539.

For those looking for data on how often we're rev'ing the various schemas, I've gathered some data here:
https://public.etherpad-mozilla.org/p/quota-manager-schema-change-log

The short story is that we have rev'ed a non-downgrade-capable schema in every release since 49 (and maybe before) except for Firefox 53.  However, in Firefox 53, we did rev the ServiceWorkerRegistrar twice, and it is downgrade-friendly only in the sense that it simply forgets about all the ServiceWorkers it knew about previously which temporarily breaks push notifications for those sites, etc.

(In reply to Romain Testard [:RT] from comment #47)
> 1 Users of multiple channels
...
> Using profiles per channel, deleting profile data or blocking users as
> they're about to open a profile from the future all seem inappropriate
> solutions that would likely create a worst user experience. I'll now engage
> with UX to come-up with a proposal to address informing users of a potential
> profile corruption.

Any progress update on these?  It seems like this wants to be 2 sub-bugs so we can track these better?
* Profile-per-channel
* UX for profile corruption / profile-from-the-future


> 2 Users of multiple release versions
...
> The new install ping being specified (bug 1351393) looks to be a good place
> to collect install data that would help understand size and pattern of users
> with multiple release versions on their systems, I'll follow-up on bug
> 1351393 with requirements to address this.

I see bug 1351393 is fixed with the bug that actually implemented the telemetry, bug 1120370, landing in Firefox 55.  When do we expect to have actionable data from the ping?  Specifically, during the beta cycle, release cycle, other?
Flags: needinfo?(rtestard)
See Also: → 1367539
Depends on: 1373244
Apologies for the delay, I was away on leave. Answers below.

> * Profile-per-channel
This seems like the right long term solution per Jeff's comment.
Benjamin also rightly mentionned on a separate e-mail that we'd need a way for testers to use nightly with their release profile from the command line. Some details have to be specified there and I now created bug 1373244 to scope and implement.

> * UX for profile corruption / profile-from-the-future
> 
I discussed with Bram and he proposed a UX for this although my understanding is now that using per channel profiles will be a better solution for the problem. Are there reasons we'd need this if bug 1373244 gets implemented?

> 
> > 2 Users of multiple release versions
> ...
> > The new install ping being specified (bug 1351393) looks to be a good place
> > to collect install data that would help understand size and pattern of users
> > with multiple release versions on their systems, I'll follow-up on bug
> > 1351393 with requirements to address this.
> 
> I see bug 1351393 is fixed with the bug that actually implemented the
> telemetry, bug 1120370, landing in Firefox 55.  When do we expect to have
> actionable data from the ping?  Specifically, during the beta cycle, release
> cycle, other?

Alessio clarified with me that the install ping actually is only sent when Firefox is started for new profiles. This means that, if some user installs Firefox again and uses an existing profile, no ping is send so it's not actually useful in our case.
I although ran an analysis of new installs through the stub installer using the stub ping data and found that 15% to 17% of new installs are done when there is already another Firefox install in another directory (https://sql.telemetry.mozilla.org/queries/3948#7852). This looks suspiciously high and I'll run this now by a data analyst to check validity.
Another thing worth mentionning is that in FF55 we ship a new stub installer that is streamlined and removes the "Options" page where users could change the installation folder (bug 1328445). Users willing to customize their installs will still be able to use full installers but this change of defaults may reduce the number of users ending-up with several installs of Firefox in several directories out of confusion.
Flags: needinfo?(rtestard)
> > * UX for profile corruption / profile-from-the-future
> > 
> I discussed with Bram and he proposed a UX for this although my
> understanding is now that using per channel profiles will be a better
> solution for the problem.

Profile-per-channel is *not* a solution to the problem. As noted in the data analysis, relatively few of the cases of people going release-backwards were caused by changing profiles, and so while it will help a small and valuable group of people it's not sufficient.
(In reply to Benjamin Smedberg [:bsmedberg] from comment #55)
> Profile-per-channel is *not* a solution to the problem. As noted in the data
> analysis, relatively few of the cases of people going release-backwards were
> caused by changing profiles, and so while it will help a small and valuable
> group of people it's not sufficient.

I'm not sure I understand the next-step implications of your comment.  Could you clarify?

My current parsing is that we still also need explicit UX for profile-from-the-future, possibly specialized to recognize that the user received an upgraded version of Firefox and explicitly rejected it by their actions in favor of an earlier version.  Something like opening up a tab with a picture of a Foxkeh in a leisure suit in a disco that lets the user know that they're using a "retro" version and offers the option to:
* Do the about:support profile refresh to combat breakage if they want to stick with this version for any amount of time.
* Provide feedback on why they chose to downgrade (via a specialized variant on the "Help", "Submit feedback..." qsurvey.mozilla.com path?).  Perhaps with a soft sell option to upgrade again after feedback has been provided.
* Hit snooze on the upgrade?  (As a preferred option to the user entirely disabling updates.)
* Soft sell on upgrading again immediately in the event they only accidentally downgraded.
Flags: needinfo?(benjamin)
I have no clue what the UX implications are, or next steps. That's why we have a product manager.
Flags: needinfo?(benjamin)
Depends on: 1374227
Per comment 47, the data analysis on comment 42 pointed us to 2 lead use cases:
- Users of multiple channels - this will be addressed in bug 1373244 
- Users of multiple versions of the same channel (likely installed in different directories) - this will be addressed in bug 1374227

Bug 1374227 probably requires UX. I pinged Bram on the bug since he had ideas when we were considering addressing bug 1373244 with UX (as opposed to the chosen solution which is now to have dedicated profiles per channel).

Alessio, my understanding is that the update system is engineered to prevent the use case you described ("the user received an upgraded version of Firefox and explicitly rejected it by their actions in favor of an earlier version") - this is why I am ignoring this for now but please let me know if you're aware of any way what you described can happen. We'd have to fix this in the update system if validated.
Blocks: 1376983
Whiteboard: [tw-dom] → [tw-dom][DevAdvocacy]
It looks like there is a new regression ongoing with current Firefox 60.0.2 vs Firefox Nightly from 2nd of July that if user once opens their Firefox 60.0.2 Stable channel user profile in Nightly, getting that profile migrated, then opening the profile again in Stable renders Firefox Stable channel unable to navigate to any web pages. Typing in an address to address bar and pressing Enter does nothing. Creating a new user profile remedies the issue, up until the next time that user profile is (accidentally) opened in Firefox Nightly.
Type: defect → enhancement
Blocks: 1355736
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.