1399937 - Add-on install problems with GNOME and twm on linux

Status: NEW

(Firefox :: Site Identity, defect, P3)

Description

[reikred]

Attachment: screenshot.2017-09-14.09-38-47.jpg

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0
Build ID: 20170824123605

Steps to reproduce:

1. install firefox 55.0.3 64b on linux (the bug does not exist on windows, apparently)
2. create a new and empty profile
3. attempt to install https://addons.mozilla.org/en-US/firefox/addon/ublock-origin



Actual results:

1. ublock-origin addon install fails silently
2. about:addons will show progress as a partial download (see attached screenshot)



Expected results:

1. the addon ublock-origin should install and work

NOTE: I am just using ublock origin as a prime example here, since it is an addon with a huge user base. There are many other addons that will not install properly , including

https://addons.mozilla.org/en-US/firefox/addon/languagetool/ (which is a webextension)
https://addons.mozilla.org/en-US/firefox/addon/simplify-awesome-bar/ (which is NOT a webextension)

No, it is not a download/disk/network problem. I ask other users please to reproduce the bug. It will take only 5 minutes to do. IMPORTANT: This is a linux-only bug, it does not occur on windows AFAICT.

Comment 1

[reikred]

Attachment: about:support.txt

Here is the about:support txt output for completeness.

Comment 2

[reikred]

Attachment: about:support.raw

Here is the about:support raw/json output for completeness.

Comment 3

[Henri Kemppainen]

> No, it is not a download/disk/network problem. I ask other users please to
> reproduce the bug. It will take only 5 minutes to do. IMPORTANT: This is a
> linux-only bug, it does not occur on windows AFAICT.

I cannot reproduce this problem on Arch with the same build that you used.  I installed all three addons successfully.

If you start firefox from a terminal, does it output any diagnostics when you try to install an addon?

Comment 4

[reikred]

(In reply to Henri Kemppainen from comment #3)
> I cannot reproduce this problem on Arch with the same build that you used. 
> I installed all three addons successfully.
> 
> If you start firefox from a terminal, does it output any diagnostics when
> you try to install an addon?

During FF startup from clean profile I get the message

1505410961345   addons.xpi      WARN    Error parsing extensions state: [Exception... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [amIAddonManagerStartup.readStartupData]"  nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)"  location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: loadExtensionState :: line 1596"  data: no] Stack trace: loadExtensionState()@resource://gre/modules/addons/XPIProvider.jsm:1596 < getInstallState()@resource://gre/modules/addons/XPIProvider.jsm:1631 < checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3152 < startup()@resource://gre/modules/addons/XPIProvider.jsm:2246 < callProvider()@resource://gre/modules/AddonManager.jsm:271 < _startProvider()@resource://gre/modules/AddonManager.jsm:741 < startup()@resource://gre/modules/AddonManager.jsm:908 < startup()@resource://gre/modules/AddonManager.jsm:3122 < observe()@jar:file:///usr/local/firefox-55.0.3/omni.ja!/components/addonManager.js:65

Right after pressing to the button to add ublock-origin addon I get the message

1505411072730   addons.xpi      WARN    Please specify whether you want browser_style or not in your options_ui options.

Does the above indicate anything to you, Henri?

Comment 5

[reikred]

After a few minutes of idling I also get these messages on the terminal"


1505411472218   addons.update-checker   WARN    Update manifest for [e-mail] did not contain an updates property
1505411472245   addons.update-checker   WARN    Update manifest for [e-mail] did not contain an updates property
1505411472250   addons.update-checker   WARN    Update manifest for [e-mail] did not contain an updates property
1505411472259   addons.update-checker   WARN    Update manifest for [e-mail] did not contain an updates property
1505411472264   addons.update-checker   WARN    Update manifest for [e-mail] did not contain an updates property
1505411472274   addons.update-checker   WARN    Update manifest for {972ce4c6-7e08-4474-a285-3208198ce6fd} did not contain an updates property
1505411472279   addons.update-checker   WARN    Update manifest for [e-mail] did not contain an updates property
1505411472429   addons.xpi      WARN    Add-on [e-mail] is not compatible with application version.
1505411472532   addons.xpi      WARN    Add-on [e-mail] is not compatible with application version.
console.log: install ADDON_INSTALL

Comment 6

[Henri Kemppainen]

(In reply to reikred from comment #4)
> Does the above indicate anything to you, Henri?

I see the same exact messages, even though addon installation works just fine.  I guess we need to find another clue.

What happens if you try to install the development channel version of uBlock Origin?  (Scroll down, it is listed at the bottom of the addon page)

Is there some other addon that does install properly for you?  If you could name some that work, maybe we can spot a pattern.

I'd also check the output of journalctl and dmesg just in case some system specific errors are logged there.

Comment 7

[reikred]

Development version of ublock-origin also fails to install So far I have also tried installing the following addons, and they fail as well

Ghostery
TaxMixPlus
SessionManager

Nothing suspect observed in dmesg or journalctl.

Comment 8

[reikred]

(In reply to Henri Kemppainen from comment #6)

Development version of ublock-origin also fails to install So far I have also tried installing the following addons, and they fail as well

Ghostery
TaxMixPlus
SessionManager

Nothing suspect observed in dmesg or journalctl.

Comment 9

[reikred]

(In reply to Henri Kemppainen from comment #6)

By the way, if I go back to FF 54.0 with a clean profile, it appears to be no problem installing these same addons (generally  speaking, I have not done a differential test of all of them yet). However, in 54.0, some of the newer Webextension adddons will not run properly it seems, but will then run if I run 55.0.3 afterwards on that same profile. I have to disappear for a few hours but will be back to check if you have any other ideas.

Comment 10

[Andy McKay]

What path does your user profile live in?

Comment 11

[Andrew Swan [:aswan]]

Can you go to about:config, flip extensions.logging.enabled to true, then attach the full browser output from during a failed install?

Comment 12

[Andrew Swan [:aswan]]

Can you also check the browser console for any errors?
https://developer.mozilla.org/en-US/docs/Tools/Browser_Console

Comment 13

[reikred]

(In reply to Andy McKay [:andym] from comment #10)
> What path does your user profile live in?

$HOME/.mozila/firefox/RANDOMSALT.profileZ2

Comment 14

[reikred]

(In reply to Andrew Swan [:aswan] from comment #11)
> Can you go to about:config, flip extensions.logging.enabled to true, then
> attach the full browser output from during a failed install?

(In reply to Andrew Swan [:aswan] from comment #12)
> Can you also check the browser console for any errors?
> https://developer.mozilla.org/en-US/docs/Tools/Browser_Console

I did both of the above, before attempting to install ublock-origin yet again into yet another pristine profile . The terminal output is a subset of the browser console output. The relevant section (after pressing the install button) is

1505430963064   addons.xpi      WARN    Error parsing extensions state: [Exception... "Component returned failure cod
e: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [amIAddonManagerStartup.readStartupData]"  nsresult: "0x80520012 (NS_ERROR_FI
LE_NOT_FOUND)"  location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: loadExtensionState :: line 15
96"  data: no] Stack trace: loadExtensionState()@resource://gre/modules/addons/XPIProvider.jsm:1596 < getInstallState
()@resource://gre/modules/addons/XPIProvider.jsm:1631 < checkForChanges()@resource://gre/modules/addons/XPIProvider.j
sm:3152 < startup()@resource://gre/modules/addons/XPIProvider.jsm:2246 < callProvider()@resource://gre/modules/AddonM
anager.jsm:271 < _startProvider()@resource://gre/modules/AddonManager.jsm:741 < startup()@resource://gre/modules/Addo
nManager.jsm:908 < startup()@resource://gre/modules/AddonManager.jsm:3122 < observe()@jar:file:///usr/local/firefox-5
5.0.3/omni.ja!/components/addonManager.js:65
1505431067291   addons.xpi      DEBUG   Download started for https://addons.mozilla.org/firefox/downloads/latest/ublo
ck-origin/addon-607454-latest.xpi?src=dp-btn-primary to file /tmp/tmp-iqj.xpi
1505431067517   addons.xpi      DEBUG   Download of https://addons.mozilla.org/firefox/downloads/latest/ublock-origin
/addon-607454-latest.xpi?src=dp-btn-primary completed.
1505431068108   addons.xpi      WARN    Please specify whether you want browser_style or not in your options_ui optio
ns.
1505431068167   addons.repository       DEBUG   cacheAddons: enabled true IDs ["[e-mail]"]
1505431068169   addons.repository       DEBUG   Requesting https://services.addons.mozilla.org/en-US/firefox/api/1.5/
search/guid:uBlock0%40raymondhill.net?src=firefox&appOS=Linux&appVersion=55.0.3
1505431068324   addons.repository       DEBUG   Got metadata search load event
1505431068327   DeferredSave.addons.json        DEBUG   Save changes
1505431068327   DeferredSave.addons.json        DEBUG   Starting timer
1505431068379   DeferredSave.addons.json        DEBUG   Starting write
1505431068381   DeferredSave.addons.json        DEBUG   Write succeeded

The whole thing from the console is bit more verbose, and more stuff got added over time. I will make an attchment out of it,
Name firefox.debug-55.0.3-browser-console.txt.

Comment 15

[reikred]

Attachment: firefox.debug-55.0.3-browser-console.txt

Comment 16

[reikred]

(In reply to Andrew Swan [:aswan] from comment #11)
> Can you go to about:config, flip extensions.logging.enabled to true, then
> attach the full browser output from during a failed install?

I completed the above needinfo request, but the flah needinfo did not get cleared. I guess I don't know how to signal that the request has been completed.

Comment 17

[reikred]

(In reply to Andrew Swan [:aswan] from comment #11)
> Can you go to about:config, flip extensions.logging.enabled to true, then
> attach the full browser output from during a failed install?

I completed the above request for needinfo, but the needinfo flag did not get cleared. Is it supposed to occur automatically? In any case, is there additional info that anyone needs to understand what is going with this bug?

Comment 18

[Andrew Swan [:aswan]]

The needinfo flag got cleared when you wrote comment 13.
Anyway, nothing looks unusual in the logs you attached, the download seemed to work just fine.  All the logging stops at the point when you should be prompted to accept the extension permissions.  It looks like you're installing from the discovery pane?  (ie, the "Get Add-ons" panel in about:addons).  Can you wait for the download to finish (the switch that you click to install the addon also doubles as a progress bar) then wait a moment, then take a screenshot of the browser at that point?

Comment 19

[reikred]

(In reply to Andrew Swan [:aswan] from comment #18)
> All the logging stops at the point when you should be
> prompted to accept the extension permissions.  It looks like you're
> installing from the discovery pane?  (ie, the "Get Add-ons" panel in
> about:addons).  Can you wait for the download to finish (the switch that you
> click to install the addon also doubles as a progress bar) then wait a
> moment, then take a screenshot of the browser at that point?

Right, I think you are onto something there. The permission pop-up dialog is a problem! When I click on the permission dialog "Add" button, I get a strange message in the browser console, namely

PopupNotifications._onButtonEvent: Button click happened before the window was focused (*) 

Also, the permission popup does not go away. Or rather, it goes away momentarily but then it pops back up again, possibly when I move the underlying window, not exactly sure what makes it appear again. Clicking multiple times does not get past the permission popup. You can "cancel" it, but of course that does not accomplish anything.

I also have another tidbit: I'm running with good old twm window manager. I started a gnome session to see if a different window manager would make a difference WRT. the "focus" issue. I found that in a Gnome session, I can SOMETIMES get the addon to install, but also in Gnome the attempt will fail most of the time, and I will also get the (*) error message when it fails.

I'll create an attachment for the screen shot. I don't think it will tell you anymore than what I just described in words, but for completeness I will do it.

Comment 20

[reikred]

Attachment: permissions popup screenshort

Comment 21

[Andrew Swan [:aswan]]

Florian, any thoughts on the PopupNotifications error mentioned in comment 19?

Comment 22

[reikred]

I hope that doesn't mean nobody is interested in this bug -- it is very real to me.

Comment 23

[Andy McKay]

It means we won't be fixing this in Firefox 57, but the bug is still open. I understand its a bug for you, but unfortunately users of twm window managers are an extremely small percentage of our population and as such we have to prioritize our scarce time and resources accordingly.

Comment 24

[reikred]

(In reply to Andy McKay [:andym] from comment #23)
> It means we won't be fixing this in Firefox 57, but the bug is still open. I
> understand its a bug for you, but unfortunately users of twm window managers
> are an extremely small percentage of our population and as such we have to
> prioritize our scarce time and resources accordingly.

But, but, the bug is there in GNOME too!!That can hardly count as being a small percentage.  Shall I go and test all the various windowmanager combos? And the bug is NOT there in FF 54.0. So what changed?

Comment 25

[reikred]

I have also checked that the bug exists in FF 56.0b8. If I was going to make a guess, I would say the file that needs looking into is 

https://dxr.mozilla.org/mozilla-beta/source/toolkit/modules/PopupNotifications.jsm

If I knew how to find out which version of this file is in 54.0 versus 55.0.3, a diff between those two versions would probably be illuminating.

Comment 26

[Henri Kemppainen]

I can reproduce this with twm.

Reverting https://hg.mozilla.org/mozilla-central/rev/34554df9f50c fixes it.

Comment 27

[Henri Kemppainen]

It appears that twm doesn't care about override redirect (except to the extent that it won't try to draw borders).  Thus it will allow you to focus the popup, like any other window.  So the main window is never in focus when you click a button in that popup.  I could argue that twm is at fault here.  But the fix to bug 1343571 feels a little hackish and will cause problems with any wm that lacks specific behavior; it fixes one minor corner case and introduces another.

Comment 28

[reikred]

(In reply to Henri Kemppainen from comment #27)
> But the fix to bug 1343571 feels a little hackish and will cause problems
> with any wm that lacks specific behavior; it fixes one minor corner case and
> introduces another.

And, in fact, my testing shows that GNOME with gnome-shell/mutter WM (a quite recent version 3.22.3, 3.24 is the very latest) is also one of those other affected Desktop/window manager combos. Were you running GNOME in your previous Arch linux test, Henri?

Comment 29

[reikred]

(In reply to Henri Kemppainen from comment #26)
> I can reproduce this with twm.
> 
> Reverting https://hg.mozilla.org/mozilla-central/rev/34554df9f50c fixes it.

Henri, would you mind telling me how to revert a .jsm file in an installed tarball on linux? I have deduced that toolkit/modules/PopupNotifications.jsm lives in omni.ja, as is shown by the command

unzip -l ./omni.ja | grep PopupNotifications

Is it safe/feasible/sensible to doctor/modify the omni.ja file to have a temporary workaround? And god forbid, does the omni.ja need to be signed and do I have to install several GB worth of some development tools to get a signature on it? As you see, I can hack a bit but I know next to nothing about the mechanics of mozilla releaes and javascript tool architecture.

Comment 30

[Henri Kemppainen]

(In reply to reikred from comment #29)
> Henri, would you mind telling me how to revert a .jsm file in an installed
> tarball on linux?

Sure, you can try this if you're feeling adventurous.  I'm hardly an expert on Firefox internals; no warranty, keep backups.  I don't know what'll break if/when you update :-)

unzip omni.ja.  It'll want to overwrite chrome.manifest, that's probably ok.  Delete omni.ja, since you won't be using it.

Delete the offending code in modules/PopupNotifications.jsm.  Search for it, or find it on lines 1438 to 1443, if we're looking at the same exact version:

>      if (Services.focus.activeWindow != this.window) {
>        Services.console.logStringMessage("PopupNotifications._onButtonEvent: " +
>                                          "Button click happened before the window was focused");
>        this.window.focus();
>        return;
>      }

That's all, you should be good to go.  Firefox will find these files outside of omni.ja (which isn't created anyway if you compile the browser from source).

As for your earlier question, no, I did not test under Gnome.  I use cwm.

Comment 31

[Henri Kemppainen]

If you want to get fancy, you could try unpack & edit & re-pack with the instructions given here:
https://developer.mozilla.org/en-US/docs/Mozilla/About_omni.ja_(formerly_omni.jar)

(BMO might break that URL so copy it or append the closing paren)

Comment 32

[reikred]

(In reply to Henri Kemppainen from comment #30)
> (In reply to reikred from comment #29)

YES!!!!!!!! That worked. It is indeed necessary to remove omni.ja or move to (say) omni.ja.ASIDE for the modification of modules/PopupNotifications.jsm to be used. I did not try to re-pack omni.ja, I will leave as is for now.

Truly awesome that you figured out the workaround, Henri. It is a lifesaver for me. I also hope that the powers that be will realize that the cause of the problem should be backed out of the code (as you said, one corner case fix creating another corner case bug) and will release a new 55.0.X with the change in it. I can only hope.

Comment 33

[reikred]

(In reply to Henri Kemppainen from comment #30)

Henri, I have one remaining problem: One of my firefox profiles does not get the modded/rolledback/fixed  modules/PopupNotifications.jsm file. How do I know? Well, when running I get the same old "Button click happened before the window was focused" that I had commented out!

Many other profiles (I tried something like 7 so far) get the intended jsm file. I run exactly the same hacked /usr/local/firefox-55.0.3/firefox on ALL the profiles.

Do you have any thoughts on how the old code may be used in just one profile case? I have cleared all http and  caches, restarted, etc etc etc but keep getting the old buggy code. Makes me wonder if there is a some hidden code cache somewhere that I have not found yet. Any ideas? Thx.

Comment 34

[Henri Kemppainen]

(In reply to reikred from comment #33)
> Makes me wonder if there is a some hidden code
> cache somewhere that I have not found yet. Any ideas? Thx.

Look for startupCache somewhere under the profile directory.  You can delete the whole directory and firefox will recreate if as needed.

Comment 35

[reikred]

Should this bug not now be considered CONFIRMED? I have personally confirmed that it exists and that the reverting https://hg.mozilla.org/mozilla-central/rev/34554df9f50c (as suggested by Henri) will fix it. The question now is when will the reversal be applied to the codebase?

Comment 36

[Florian Quèze [:florian]]

(In reply to Andrew Swan [:aswan] from comment #21)
> Florian, any thoughts on the PopupNotifications error mentioned in comment
> 19?

It's likely that either window.focus() or Services.focus.activeWindow is not behaving correctly on that system.

Comment 37

[Johann Hofmann [:johannh]]

(In reply to reikred from comment #32)
> (In reply to Henri Kemppainen from comment #30)
> Truly awesome that you figured out the workaround, Henri. It is a lifesaver
> for me. I also hope that the powers that be will realize that the cause of
> the problem should be backed out of the code (as you said, one corner case
> fix creating another corner case bug) and will release a new 55.0.X with the
> change in it. I can only hope.

and

(In reply to reikred from comment #35)
> Should this bug not now be considered CONFIRMED? I have personally confirmed
> that it exists and that the reverting
> https://hg.mozilla.org/mozilla-central/rev/34554df9f50c (as suggested by
> Henri) will fix it. The question now is when will the reversal be applied to
> the codebase?

Just for the record, the "corner case fix" is an important security patch that prevents attackers from obtaining device permissions through phishy popup-windows. It should (and will) not be backed out under any circumstances.

I'm happy to take a patch that fixes your platform issues while maintaining the same protections that the offending patch offers.

I'm not sure where to put this patch, so I'm leaving it in the front-end component for now.

Comment 39

[Andy McKay]

Suggesting we bump up the priority on this, it sounds like its affecting a good chunk of Linux users.

Comment 41

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage/Projects/Bug_Handling/Bug_Husbandry#Move_fix-optionals

Comment 42

[aeb]

A me too: add-ons fail to install.
Came here via Googling of "PopupNotifications._onButtonEvent: Button click happened before the window was focused"
This is on Linux with twm.
In Bug 1427108 the workaround 'push Alt-A without moving the mouse after clicking "Add to Firefox"' is suggested.
That works for me.

Comment 43

[a463b]

Bug confirmed with FF 74.0, 72.0, 70.0

It can be reliably reproduced like so:

• Install vanilla Kubuntu 19.10
• System Settings -> Window Management -> Window Behavior -> Focus policy: change from "Click" to "Hover"
• Launch FF and try to install an add-on or try to click on any pop-up like request for access to camera/microphone, etc

Thus for Kubuntu there are two workarounds:

• with the mouse-pointer being outside of the pop-up: Left Alt + a (or whatever the underlined character is for the button) as has been mentioned here
• switch the Focus policy back to "Click"

As this page here is quite hard to find (tried many things including apparmor before discovering the "PopupNotifications._onButtonEvent: Button click happened before the window was focused") a fix would be fantastic.

Comment 45

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug is a regression, but please revert this change in case of error.

Comment 46

[BugBot [:suhaib / :marco/ :calixte]]

The severity field for this bug is relatively low, S3. However, the bug has 3 duplicates.
:pbz, could you consider increasing the bug severity?

For more information, please visit auto_nag documentation.

Comment 47

[BugBot (nomail) [:suhaib / :marco/ :calixte]]

The last needinfo from me was triggered in error by recent activity on the bug. I'm clearing the needinfo since this is a very old bug and I don't know if it's still relevant.