1234658 - foreignInstall flag is true for add-ons installed from AMO

Status: RESOLVED INCOMPLETE

(Toolkit :: Add-ons Manager, defect)

Description

[Andreas Wagner [:TheOne] [use NI]]

We haven't found a reliable way to reproduce this yet, but sometimes somehow foreignInstall is being set to true for add-ons installed from AMO which causes preliminary reviewed add-ons to get disabled.

STR are still pending.

Comment 1

[Andreas Wagner [:TheOne] [use NI]]

I saw this happening with "Tree Style Tabs" after upgrading to 43 in a profile that was about 1.5 years old.

I was also able to reproduce this exactly once with a fresh profile in 42 and then upgrading to 43.

Comment 2

[Andy McKay]

Just a note that 12,646 add-ons are preliminary reviewed on AMO and could get into this state potentially.

Comment 3

[Dan Stillman]

Note that this isn't specific to AMO — we saw this with two different unlisted, front-loaded extensions of Zotero's, one of which has never been on AMO and another that hasn't been on AMO for years and was likely never installed from there by the affected users. Looking at the extensions.json files of people who reported this, various extensions had the flag set to true seemingly in error, but I couldn't find any pattern as to which ones.

Comment 4

[Kris Maglione [:kmag]]

Attachment: Turn off signing requirements on release to work around foreign install flag problems

Comment 5

[Madalin Cotetiu]

I have installed some preliminary reviewed addons in AMO-dev using FF42(Win7), this is the screencast after installing them: http://screencast.com/t/aQkjQqmduwii
I updated the browser to 43 and then restarted. 
The result was that all the add-ons installed were removed:  http://screencast.com/t/XdZCqcki

Comment 6

[Madalin Cotetiu]

Attachment: logsAfterAddonInstall.txt

Attaching the logs obtained after following TheOne steps.
Here are the logs after the addons were installed.

Comment 7

[Madalin Cotetiu]

Attachment: logsAfterRestart.txt

The logs quiting firefox and update.

Comment 8

[Madalin Cotetiu]

Attachment: extensions.json

Comment 9

[Madalin Cotetiu]

:TheOne
Please let me know if I can do anything else.

Comment 10

[Tony Mechelynck [:tonymec]]

(In reply to Madalin Cotetiu from comment #5)
> I have installed some preliminary reviewed addons in AMO-dev using
> FF42(Win7), this is the screencast after installing them:
> http://screencast.com/t/aQkjQqmduwii
> I updated the browser to 43 and then restarted. 
> The result was that all the add-ons installed were removed: 
> http://screencast.com/t/XdZCqcki

In the first screenshot, the add-ons are "pending for install" ("<add-on-name> will be installed after you restart Firefox"), they are not yet fully installed. Is the behaviour different if at that point you restart the same browser version (e.g. by clicking "Restart now" on any of these add-ons) and only upgrade the browser afterwards? (with a second restart, or a closedown and startup)

Comment 11

[Andreas Wagner [:TheOne] [use NI]]

(In reply to Madalin Cotetiu from comment #6)
> Created attachment 8701824 [details]
> logsAfterAddonInstall.txt
> 
> Attaching the logs obtained after following TheOne steps.
> Here are the logs after the addons were installed.

Those were the steps I provided:

1) Install 42 and use a fresh profile
2) set "extensions.logging.enabled" to true
3) restart Firefox with -console
4) install the add-ons
5) Copy the output in the console to a safe place
6) Look for updates, let 43 install
7) Quit Firefox normally (no restart)
8) Copy extensions.json to a safe place
9) Start Firefox with -console again
10) Copy the console output to a safe place
11) Quit Firefox
12) Copy the extension.json to a safe place

Comment 12

[Dave Townsend [:mossop]]

(In reply to Madalin Cotetiu from comment #5)
> I have installed some preliminary reviewed addons in AMO-dev using
> FF42(Win7), this is the screencast after installing them:
> http://screencast.com/t/aQkjQqmduwii
> I updated the browser to 43 and then restarted. 
> The result was that all the add-ons installed were removed: 
> http://screencast.com/t/XdZCqcki

These add-ons are from the dev site and so don't use the production signing root and so Firefox considers them unsigned. Because you don't restart Firefox 42 after staging the add-on installs you hit bug 1196797 and Firefox 43 removes the extensions entirely.

Setting xpinstall.signatures.dev-root to true in the profile would make Firefox consider these add-ons to be properly signed.

Comment 13

[Dave Townsend [:mossop]]

I can't find any way to reproduce this and without that I can't understand what is going on here.

Comment 14

[Andreas Wagner [:TheOne] [use NI]]

Kris and I were both inspecting the code that could lead to this. The only possible way we could identify is when the extensions metadata (extensions.json?) gets thrown away, possibly due to a file locking issue.
We need to investigate how we could simulate that on a Windows machine.

Comment 15

[Dave Townsend [:mossop]]

(In reply to Andreas Wagner [:TheOne] from comment #14)
> Kris and I were both inspecting the code that could lead to this. The only
> possible way we could identify is when the extensions metadata
> (extensions.json?) gets thrown away, possibly due to a file locking issue.
> We need to investigate how we could simulate that on a Windows machine.

Is this windows only? We have had many issues in the past with tools like Anti-virus and backup locking files that then breaks things.

Comment 16

[Andreas Wagner [:TheOne] [use NI]]

(In reply to Dave Townsend [:mossop] from comment #15)
> (In reply to Andreas Wagner [:TheOne] from comment #14)
> > Kris and I were both inspecting the code that could lead to this. The only
> > possible way we could identify is when the extensions metadata
> > (extensions.json?) gets thrown away, possibly due to a file locking issue.
> > We need to investigate how we could simulate that on a Windows machine.
> 
> Is this windows only? We have had many issues in the past with tools like
> Anti-virus and backup locking files that then breaks things.

I don't know. I have seen it several times on a Windows machine. I don't know of any occurrences on other platforms, but that doesn't mean anything. I don't think we gathered that data from people reporting this issue.

Comment 17

[Dan Stillman]

We had a report from an Ubuntu user, who had an extensions.json file with a whole bunch of extensions, including many with the AMO CDN as the sourceURI, marked as foreignInstall: true|1.

Comment 18

[Kris Maglione [:kmag]]

(In reply to Dan Stillman from comment #17)
> We had a report from an Ubuntu user, who had an extensions.json file with a
> whole bunch of extensions, including many with the AMO CDN as the sourceURI,
> marked as foreignInstall: true|1.

Do you have any more details on this? It's not uncommon for add-ons to be installed in strange ways via system package managers on Linux, or for package managers/users to do strange things with profiles, so I wouldn't be surprised if this was unrelated.

There are definitely cases of this happening to Windows users with more ordinary usage patterns, though, and I haven't heard of anyone reproducing it on OS-X.

Comment 19

[Kris Maglione [:kmag]]

(In reply to Andreas Wagner [:TheOne] from comment #14)
> Kris and I were both inspecting the code that could lead to this. The only
> possible way we could identify is when the extensions metadata
> (extensions.json?) gets thrown away, possibly due to a file locking issue.
> We need to investigate how we could simulate that on a Windows machine.

I don't think that was my conclusion. If extensions.json gets thrown away, we should recreate the database in recovery mode, and detected extensions should not have the foreign install flag. If we had a valid database at startup, though, and it didn't include the add-on, that would cause it to be marked as a foreign install.

You said this happened to you after installing the add-on, and then immediately restarting a couple of times, which led me to suspect that there may have been an issue writing the updated database after the first restart, and the add-on being detected as a foreign install after the second. This is basically a stab in the dark until we have a reliable way of reproducing this, though.

Comment 20

[Dave Townsend [:mossop]]

(In reply to Kris Maglione [:kmag] from comment #19)
> (In reply to Andreas Wagner [:TheOne] from comment #14)
> > Kris and I were both inspecting the code that could lead to this. The only
> > possible way we could identify is when the extensions metadata
> > (extensions.json?) gets thrown away, possibly due to a file locking issue.
> > We need to investigate how we could simulate that on a Windows machine.
> 
> I don't think that was my conclusion. If extensions.json gets thrown away,
> we should recreate the database in recovery mode, and detected extensions
> should not have the foreign install flag. If we had a valid database at
> startup, though, and it didn't include the add-on, that would cause it to be
> marked as a foreign install.

Yeah I just verified this with a local profile. Deleting extensions.json (same effect as if it is corrupt or otherwise unreadable) means it gets recreated and foreignInstall set to false for already installed add-ons. It probably isn't perfect but we're erring on the right side there I think.

> You said this happened to you after installing the add-on, and then
> immediately restarting a couple of times, which led me to suspect that there
> may have been an issue writing the updated database after the first restart,
> and the add-on being detected as a foreign install after the second. This is
> basically a stab in the dark until we have a reliable way of reproducing
> this, though.

Yeah it would have to be either blocking extensions.json writes during a restartless install (and until app exit) or blocking on the next startup while installing a restartful add-on but then not blocking later. Do we have any idea about if this only affects restartless or restartful?

Comment 21

[Kris Maglione [:kmag]]

(In reply to Dave Townsend [:mossop] from comment #20)
> Do we have any idea about if this only affects restartless or restartful?

Every report I've seen so far has been for a restartful add-on.

Comment 23

[Madalin Cotetiu]

(In reply to Tony Mechelynck [:tonymec] from comment #10)
> (In reply to Madalin Cotetiu from comment #5)
> > I have installed some preliminary reviewed addons in AMO-dev using
> > FF42(Win7), this is the screencast after installing them:
> > http://screencast.com/t/aQkjQqmduwii
> > I updated the browser to 43 and then restarted. 
> > The result was that all the add-ons installed were removed: 
> > http://screencast.com/t/XdZCqcki
> 
> In the first screenshot, the add-ons are "pending for install"
> ("<add-on-name> will be installed after you restart Firefox"), they are not
> yet fully installed. Is the behaviour different if at that point you restart
> the same browser version (e.g. by clicking "Restart now" on any of these
> add-ons) and only upgrade the browser afterwards? (with a second restart, or
> a closedown and startup)

Sorry, I missed this comment.
The results where the same even when using your scenario. All the installed add-ons were removed.

Comment 24

[Dave Townsend [:mossop]]

(In reply to Madalin Cotetiu from comment #23)
> (In reply to Tony Mechelynck [:tonymec] from comment #10)
> > (In reply to Madalin Cotetiu from comment #5)
> > > I have installed some preliminary reviewed addons in AMO-dev using
> > > FF42(Win7), this is the screencast after installing them:
> > > http://screencast.com/t/aQkjQqmduwii
> > > I updated the browser to 43 and then restarted. 
> > > The result was that all the add-ons installed were removed: 
> > > http://screencast.com/t/XdZCqcki
> > 
> > In the first screenshot, the add-ons are "pending for install"
> > ("<add-on-name> will be installed after you restart Firefox"), they are not
> > yet fully installed. Is the behaviour different if at that point you restart
> > the same browser version (e.g. by clicking "Restart now" on any of these
> > add-ons) and only upgrade the browser afterwards? (with a second restart, or
> > a closedown and startup)
> 
> Sorry, I missed this comment.
> The results where the same even when using your scenario. All the installed
> add-ons were removed.

The add-ons get completely removed from the add-ons manager? That would be very unexpected but if so please file a new bug to track it, it isn't the issue we're trying to find here.

Comment 25

[Madalin Cotetiu]

I can't no longer reproduce the issue I have mentioned in comment 5. Restarting the browser after or before updating have the same result, the add-ons are still displayed as enabled: http://screencast.com/t/JmUSaXUz

Comment 26

[Andy McKay]

If bug 1245956 happens, then this will be irrelevant.

Comment 27

[Andy McKay]

Taken off the signing blocker. Mossop, is this something you think we should keep open to get accurate foreignInstall data? Without reliable STR, not sure if there's much we can do here.

Comment 28

[Dave Townsend [:mossop]]

Yeah ultimately with no way to reproduce this I don't think we're ever going to see this fixed.