1650479 - "Conflict Catcher"-like utility to help users diagnose problems without resorting to a Firefox Refresh (binary search troubleshooting)

Status: REOPENED

(Toolkit :: General, enhancement)

Description

[Asif Youssuff]

One of the downsides of Firefox's extensions capabilities and overall customization features is that when things go wrong, it can be very hard to track down what is causing the issue.

While problems are clearly not as bad as in the past with the permissive/legacy extensions model, and the continual removal of footguns in Firefox, it is still very easy to screw up Firefox in unexpected ways.

It would be nice for Firefox to have a built in troubleshooting UI (perhaps built into about:support) to help enterprising users track down issues and perhaps file bugs to fix underlying issues in add-ons, or in Firefox itself.

Classic Mac OS had a utility called "Conflict Catcher" that dealt with this kind of issue:

As in previous versions, Conflict Catcher handles the simple but extremely tedious task of testing all possible combinations of startup files until an offender (or an offending combination) is identified. It does this by restarting your Mac with half of the previous startup file set disabled, and then asking you if the problem still exists. This process is repeated until the offending startup files are isolated or startup file conflicts are ruled out.

https://tidbits.com/1995/05/08/conflict-catcher-3/

Essentially, this utility did a binary search to locate the combination of extensions that caused the issue the user is experiencing.

In the Firefox version, the process might look like:

1. Does the issue exist in safe mode?
2. If yes, do a binary search of preferences modified away from the default
3. If no, move on to extensions, doing a binary search of enabled extensions

When completed, tell the user what preference or extensions are causing the issue, allowing them to easily report issues to either bugzilla or the extension developers.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[Rob Wu [:robwu]]

We are probably not going to add a tool to automatically enable/disable add-ons in Firefox. Users can manually do this through about:addons .

We are interested in informing users about changes from add-ons, and bug 1461447 is about showing preference changes by add-ons.
Add-ons can cause conflicts in more ways, e.g. interfering each other with request handling or content changes. Such conflicts cannot easily be summarized in about:support, and in those cases the only way to find the culprit is to disable/enable add-ons.

If there is interest from third-party developers, we are willing to consider opening up the management.setEnabled API, to allow extensions to disable/enable other extensions (provided that there are enough safeguards to prevent abuse, e.g. requiring another user prompt to confirm the decision). See bug 1282982 .

Comment 3

[Asif Youssuff]

Rob, the request isn't necessarily for add-ons, although it gets significantly less powerful if add-ons are not in scope.

Is there a better home for this if we consider just the preferences modifiable by user.js and about:config? Think about the problems going live with a ghacks user.js (https://github.com/ghacksuserjs/ghacks-user.js/) can cause - it would be fantastic if Firefox had tools to let people diagnose why something isn't working within a single build (like mozregression does across many builds).

Comment 4

[Asif Youssuff]

Reopened and rehomed to toolkit like bug 594464 for now.

Comment 5

[Rob Wu [:robwu]]

(In reply to Asif Youssuff from comment #3)

Rob, the request isn't necessarily for add-ons, although it gets significantly less powerful if add-ons are not in scope.

I understood the bug report as a request to automatically enable/disable add-ons to find a defect, especially because of the comment "Essentially, this utility did a binary search to locate the combination of extensions that caused the issue the user is experiencing.".
If that's not the case, could you clarify your request?

Is there a better home for this if we consider just the preferences modifiable by user.js and about:config?

about:config and user.js is for advanced users. Modified default prefs are already shown at about:support (which could help with finding the culprit remotely), and users can already manually flip preferences at about:config. It's not automatic, but safer than automatically resetting pref values (which could have side effects). I don't think that we can offer a reliable way to toggle prefs to help with diagnosing issues, especially because the user needs to manually verify whether an issue has been resolved. I think that the power user is in the best position of identifying potentially relevant prefs and then trying out different combinations until the bug went away (after making a backup of the profile first).

Comment 6

[Asif Youssuff]

(In reply to Rob Wu [:robwu] from comment #5)

about:config and user.js is for advanced users. Modified default prefs are already shown at about:support (which could help with finding the culprit remotely), and users can already manually flip preferences at about:config. It's not automatic, but safer than automatically resetting pref values (which could have side effects). I don't think that we can offer a reliable way to toggle prefs to help with diagnosing issues, especially because the user needs to manually verify whether an issue has been resolved. I think that the power user is in the best position of identifying potentially relevant prefs and then trying out different combinations until the bug went away (after making a backup of the profile first).

All modified default prefs are not shown in about:support - which I totally understand, there are way too many preferences and it would end up looking like about:config.

I'm not proposing that we automatically reset prefs (which we do by the way with the Firefox Refresh feature - very dangerous and hard to undo!), but rather create temporary copies of profiles and launch them sequentially, like mozregression does.

I don't think that we can offer a reliable way to toggle prefs to help with diagnosing issues, especially because the user needs to manually verify whether an issue has been resolved.

Well, that would be the point - the user would identify what they think is "good" and "bad", and Firefox would report back which preference causes the deviation from the standard set of preferences in base Firefox. The "good" set would always be the default set (like after a Firefox Refresh), and if it were not "good", the feature would bail and say that there is a deeper issue.

If a user stops in the middle or the process is stopped because the user reports that a base configuration is not "good", their original profile is not modified and the user is free to continue with their own troubleshooting steps - this would be a completely nondestructive operation (unlike say Firefox Refresh) and users would have to take the information they receive and follow up on it (perhaps Firefox could offer to reset just the preferences identified by the user as causing the issue).

Comment 7

[Asif Youssuff]

(In reply to Rob Wu [:robwu] from comment #5)

I understood the bug report as a request to automatically enable/disable add-ons to find a defect, especially because of the comment "Essentially, this utility did a binary search to locate the combination of extensions that caused the issue the user is experiencing.".
If that's not the case, could you clarify your request?

I wasn't sure if my prior comment clarified my request enough - like I said in my initial comment:

(In reply to Asif Youssuff from comment #0)

In the Firefox version, the process might look like:

1. Does the issue exist in safe mode?
2. If yes, do a binary search of preferences modified away from the default
3. If no, move on to extensions, doing a binary search of enabled extensions

When completed, tell the user what preference or extensions are causing the issue, allowing them to easily report issues to either bugzilla or the extension developers.

The minimal version contemplates preferences only, while an expanded version would include extensions as part of the search.

The goal here is to get people back up and running to a configuration that works well for them, and to make it easy to disseminate that learning back to the community, and Firefox and extension developers - to either fix those issues, or to eliminate footguns.

Right now, I'm sure that many issues are worked around by Refreshing Firefox - because it is so tedious to play the guessing game of trying to figure out which preference or group of preferences or extension is newly causing an issue. This makes it so that people remain in a sub-optimal configuration simply because of the effort involved in diagnosing what is wrong (or they switch to a browser that works better for them).

With some documentation around this feature on SUMO, we could accelerate problem discovery and resolution, and increase user satisfaction across the board (by identifying long hidden issues) quicker than waiting for a single enterprising power user to encounter the issue in their configuration and do the work to identify what causes it.

I know I have often given up and performed a Firefox Refresh when experiencing issues - how many of those issues that I encountered are still being experienced by users today (because I was unable to easily discover the problem)?

Comment 8

[Rob Wu [:robwu]]

I see. The bug report was in the WebExtensions product because a bot moved it, rather than you filing it there. This request is not specific to add-ons, the only relevant part is that add-ons may be one of the things that you want to reset.

The goal here is to get people back up and running to a configuration that works well for them,

Firefox Refresh, as you know, does this. It's a big hammer though.
Firefox Safe Mode is a lighter variant to run without add-ons, but I don't think that there is anything to run without prefs, other than temporarily renaming prefs.js with a file manager. Renaming/removing prefs.js is not recommended at all because other data in the profile may rely on prefs (for example, add-on data will be inaccessible because addon -> uuid mappings are stored there).

A prerequisite for a tool that meaningfully flips prefs is that there is a reasonably complete list of preferences that may cause bugs and that relate to each other (e.g. there is no point in testing pref flips when the relevant feature is disabled). I can't imagine many users being thrilled to run their reproduction steps hundreds of times. Maintaining such a list is quite some work, and I would recommend to focus on establishing such a list before even considering the automation of pref flipping.

I think that it would be more realistic to ask for a filter to show modified prefs (starting with fixing bug 1502867), with the option to filter prefs that aren't interesting (e.g. auto-generated values from Firefox's internals) - based on that list.

Comment 9

[Asif Youssuff]

It seems like a subset of my request is present in Microsoft Edge as "extension bisect" - an overview can be found at https://code.visualstudio.com/blogs/2021/02/16/extension-bisect