Last Comment Bug 730040 - Error: "could not be installed because Firefox cannot modify the needed file" when installing add-on
: Error: "could not be installed because Firefox cannot modify the needed file"...
Status: UNCONFIRMED
:
Product: Toolkit
Classification: Components
Component: Add-ons Manager (show other bugs)
: 10 Branch
: x86_64 Windows 7
: -- normal with 3 votes (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
:
Mentors:
Depends on: 714168
Blocks:
  Show dependency treegraph
 
Reported: 2012-02-23 11:18 PST by John Nagle
Modified: 2016-01-26 09:33 PST (History)
8 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
adlimiter.xpi - built with SDK 1.5 (BAD) (220.36 KB, application/octet-stream)
2012-02-23 11:18 PST, John Nagle
no flags Details
adlimiter.xpi - built with SDK 1.4.3 (GOOD) (248.00 KB, application/octet-stream)
2012-02-23 11:20 PST, John Nagle
no flags Details
Screen shot of error console (260.04 KB, image/png)
2012-02-23 12:18 PST, John Nagle
no flags Details

Description John Nagle 2012-02-23 11:18:31 PST
Created attachment 600101 [details]
adlimiter.xpi - built with SDK 1.5 (BAD)

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0) Gecko/20100101 Firefox/10.0
Build ID: 20120129021758

Steps to reproduce:

Downloaded add-on SDK 1.5.
Rebuilt working add-on previously built with SDK 1.4.3 using "cfx xpi" command.


Actual results:

Firefox displayed the message: "Ad Limiter could not be installed because Firefox cannot modify the needed file".


Expected results:

"Add on installed"
Comment 1 John Nagle 2012-02-23 11:20:00 PST
Created attachment 600102 [details]
adlimiter.xpi - built with SDK 1.4.3 (GOOD)

This is the version that works, using the older SDK.
Comment 2 Nils Maier [:nmaier] 2012-02-23 11:38:02 PST
I cannot reproduce.

What I tried:
- Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2) Gecko/20100101 Firefox/10.0.2
- Fresh profile

1) Install GOOD first, then upgrade to BAD -> worked
2) Install only BAD -> worked
3) Install Version 1.0. then upgrade to BAD -> worked

Can you set extensions.logging.enabled to true (in about:config), try to install BAD and check the Error Console to see if there are any error messages logged?

Any chance you have any files in <profile>/extensions open, or any command line prompts with a working directory within that location?
Comment 3 John Nagle 2012-02-23 12:18:01 PST
Created attachment 600134 [details]
Screen shot of error console

1. Deleted add-on.
2. Enabled extension logging.
3. Closed and reopened Firefox.
4. Attempted to load failing .XPI file.
5. Resulting error log attached.

The logging indicates a file permissions problem, but doesnt't give the file name. I'm running as a non-privileged user, but Firefox itself is installed on Windows 7 for all users.  Did something change in SDK 1.5 which causes installation to try to modify a protected file without permission to do so?

Error messages like this should give the file name. With that, this would be much easier to diagnose.
Comment 4 Nils Maier [:nmaier] 2012-02-23 12:33:50 PST
This translates here:
https://mxr.mozilla.org/mozilla-release/source/toolkit/mozapps/extensions/XPIProvider.jsm#7910
which belongs to DirInstallLocation_installAddon:
https://mxr.mozilla.org/mozilla-release/source/toolkit/mozapps/extensions/XPIProvider.jsm#7857

So this seems this bug belongs indeed into the Add-ons Manager realm.

em:unpack changes from true to bad (SDK 1.4.3 -> 1.5), so that might be a culprit here?
Makes me wonder why DirInstallLocation_installAddon is called in the first place now that the XPI defines em:unpack=false...
Dave might know more about it?
Comment 5 Dave Townsend [:mossop] 2012-02-23 12:59:34 PST
(In reply to Nils Maier [:nmaier] from comment #4)
> This translates here:
> https://mxr.mozilla.org/mozilla-release/source/toolkit/mozapps/extensions/
> XPIProvider.jsm#7910
> which belongs to DirInstallLocation_installAddon:
> https://mxr.mozilla.org/mozilla-release/source/toolkit/mozapps/extensions/
> XPIProvider.jsm#7857
> 
> So this seems this bug belongs indeed into the Add-ons Manager realm.
> 
> em:unpack changes from true to bad (SDK 1.4.3 -> 1.5), so that might be a
> culprit here?
> Makes me wonder why DirInstallLocation_installAddon is called in the first
> place now that the XPI defines em:unpack=false...
> Dave might know more about it?

Yep, definitely something in the add-ons manager code, highly unlikely there is an SDK issue here. Looks like immediately after we copy the XPI into the final place we attempt to update its last modified time and that faile. This could be caused by AV tools on your system scanning the XPI or something. It can also happen if your profile folder is on a strange filesystem, that seems unlikely for Windows though. Could also be an issue in Firefox itself.

One way to figure this out would be to use process monitor to track what accesses files in the profile during the install but that can be quite noisy. Ultimately this problem is probably already fixed in Firefox 13 by bug 714168 which ignores failures from setting the modification time. Can you test that?
Comment 6 Nils Maier [:nmaier] 2012-02-23 13:21:05 PST
(In reply to Dave Townsend (:Mossop) from comment #5)
> Yep, definitely something in the add-ons manager code, highly unlikely there
> is an SDK issue here. Looks like immediately after we copy the XPI into the
> final place we attempt to update its last modified time and that faile. This
> could be caused by AV tools on your system scanning the XPI or something.

After diving into the XPIProvider code a bit, I'd agree that some external culprit seems most likely.

John, could you tell us what Antivirus software you have installed? Could you try to temporarily turn it off and try to install again?
Does it happen when you install other XPIs in the same profile? E.g. https://addons.mozilla.org/addon/adblock-plus/ which is reasonably large and does not use em:unpack as well.

> One way to figure this out would be to use process monitor to track what
> accesses files in the profile during the install but that can be quite
> noisy.

Process Monitor (Microsoft Freeware)
http://technet.microsoft.com/en-us/sysinternals/bb896645
John, if you'd like to try that, I suggest you add an Inclusion filter "Path ends with .xpi" (not Image Path).
If you aren't familiar with such a log, then saving it to .pml and attaching it here would be appreciated.

> Ultimately this problem is probably already fixed in Firefox 13 by
> bug 714168 which ignores failures from setting the modification time. Can
> you test that?

If this problem was caused by some popular AV product, then the problem could be far more widespread. Waiting for 13 would be problematic then...
Comment 7 Dave Townsend [:mossop] 2012-02-23 13:25:00 PST
(In reply to Nils Maier [:nmaier] from comment #6)
> (In reply to Dave Townsend (:Mossop) from comment #5)
> > Yep, definitely something in the add-ons manager code, highly unlikely there
> > is an SDK issue here. Looks like immediately after we copy the XPI into the
> > final place we attempt to update its last modified time and that faile. This
> > could be caused by AV tools on your system scanning the XPI or something.
> 
> After diving into the XPIProvider code a bit, I'd agree that some external
> culprit seems most likely.
> 
> John, could you tell us what Antivirus software you have installed? Could
> you try to temporarily turn it off and try to install again?
> Does it happen when you install other XPIs in the same profile? E.g.
> https://addons.mozilla.org/addon/adblock-plus/ which is reasonably large and
> does not use em:unpack as well.

Also backup software can do this, and things like dropbox if you have your profile in that somehow (though I bet that is a bad idea for many reasons!)

> > Ultimately this problem is probably already fixed in Firefox 13 by
> > bug 714168 which ignores failures from setting the modification time. Can
> > you test that?
> 
> If this problem was caused by some popular AV product, then the problem
> could be far more widespread. Waiting for 13 would be problematic then...

Possibly yeah, though we haven't really had widespread reports of this and this would have been a problem since Firefox 4, so if it is caused by an AV tool it's likely a recent update to it. We could always consider backporting the fix to aurora or beta.
Comment 8 John Nagle 2012-02-23 14:05:27 PST
1.  The machine this is on is not running any "anti-virus software".
2.  Dropbox is not installed. 
3.  I just ran the latest version of the Microsoft Malicious Software Removal Tool, and it reports no problems.

I've discovered that if the adrater.xpi file is read-only, it will not install.
If made writeable, it will install.  
This happens only with SDK 1.5, not with SDK 1.4.3.  

You should now be able to reproduce the problem. Take the "adlimiter.xpi" file above, make it read-only, and try to install it. 

The file content doesn't change. The date modified doesn't change. 
But apparently the installer is trying to change some attribute of the file being installed, which it shouldn't be doing.  Or it copied the file, made the copy read-only, and then couldn't update its own copy. 

Bad installer. No donut.
Comment 9 Nils Maier [:nmaier] 2012-02-23 14:14:07 PST
Might be an SDK/cfx issue after all then.
John, are you sure the XPI by "cfx xpi" does have the read-only flag set?
In my copy it did not set that flag.

Anyway, distributing the XPI via the AMO or the internet in general isn't an issue, as neither the upload nor the subsequent downloads will preserve the read-only attribute, so feel free to submit the new version.

Dave, would you consider this a bug in the XPIProvider? Shouldn't it try to strip the read-only attribute?
Comment 10 Dave Townsend [:mossop] 2012-02-23 14:25:54 PST
(In reply to John Nagle from comment #8)
> 1.  The machine this is on is not running any "anti-virus software".
> 2.  Dropbox is not installed. 
> 3.  I just ran the latest version of the Microsoft Malicious Software
> Removal Tool, and it reports no problems.
> 
> I've discovered that if the adrater.xpi file is read-only, it will not
> install.
> If made writeable, it will install.  
> This happens only with SDK 1.5, not with SDK 1.4.3.  
> 
> You should now be able to reproduce the problem. Take the "adlimiter.xpi"
> file above, make it read-only, and try to install it. 

What steps are you following to install this file?
Comment 11 John Nagle 2012-02-23 14:59:29 PST
I didn't say that cfx made the file read-only. I said that it wouldn't install when read-only.  I made it read-only by putting the .xpi file under source control. And yes, I opened the properties box on Windows and looked at the read-only check mark.

So who's tried making the .xpi file read only and loading it?  Anybody?
Comment 12 Jeff Griffiths (:canuckistani) (:⚡︎) 2012-02-23 15:03:44 PST
So the difference is that 1.5 no longer un-packs the xpi, whereas 1.4.x still did. It seems like this is a bug in the xpi installer, and it also seems like there is a fix available in nightly. 

* Should we nominate this fix for aurora and beta? The use case of the xpi ending up read-only feels like an edge case to me.
Comment 13 John Nagle 2012-02-24 10:58:07 PST
It's a form of security issue. Requiring write permission for something that's only being read encourages users to open up security holes.  In-house add-ons can't be distributed through a read-only shared directory on a server, for example.

(Somewhere, an in-house developer is probably cursing Mozilla for this right now.)
Comment 14 microrffr 2012-05-13 15:18:52 PDT
(In reply to John Nagle from comment #13)
> (Somewhere, an in-house developer is probably cursing Mozilla for this right
> now.)

I'm writing an extension for Fennec Native.
To make testing easier, I wrote a script to push my addon to my phone's SD card and open it in Nightly.

All the files on the SD card are set to:
    ----rwxr-x system   sdcard_rw     xxxx xxxx-xx-xx xx:xx test.xpi

Code in XPIProvider.jsm copies it into the profile, keeping the same permissions, but the owner changes:
    ----rwxr-x app_57   app_57        xxxx xxxx-xx-xx xx:xx {extension-id}.xpi
(on Android, each application has its own userid; Nightly is app_57 for me)

Then nothing works because it can't access the xpi.

Anyway, for the old XUL overlay based extensions, I worked around this by chmodding the staged XPI to 644 (the permissions used for extensions downloaded from the Internet) between installation and restart. I can't do this for the new bootstrapped restartless extensions.

So yeah, curse you Mozilla.

I'm glad I found this thread though. I'm going to try using unpacking to work around this.
Comment 15 microrffr 2012-05-13 15:20:47 PDT
I think bug 401719 is the same problem.
Comment 16 Wes Kocher (:KWierso) 2012-05-13 16:41:05 PDT
(In reply to microrffr from comment #14)
Can you see if it works correctly in a current Firefox Beta?
Comment 17 microrffr 2012-05-13 17:00:00 PDT
(In reply to Wes Kocher (:KWierso) from comment #16)
I think Fennec Native won't be in the Beta channel until the 15th.
Comment 18 microrffr 2012-05-17 17:30:32 PDT
I tried it with Android Firefox Beta.

1. put an xpi on the sdcard that has bootstrap=true and unpack=false
   (the file properties should be similar to the description in comment 14.)
2. open the xpi in Firefox Beta (this shows a blank page and opens an addon installation dialog)
3. accept the installation

actual results:

the following appears in the logs:
E/GeckoConsole( 7334): [JavaScript Warning: "WARN addons.xpi: Error loading bootstrap.js for dl@w: Error opening input stream (invalid filename?)"]
E/GeckoConsole( 7334): [JavaScript Warning: "WARN addons.xpi: Add-on dl@w is missing bootstrap method install" {file: "resource://gre/modules/XPIProvider.jsm" line: 3637}]
E/GeckoConsole( 7334): [JavaScript Warning: "WARN addons.xpi: Add-on dl@w is missing bootstrap method startup" {file: "resource://gre/modules/XPIProvider.jsm" line: 3637}]

The extension shows up in the addons manager, but it doesn't work.

expected results:
no errors, extension works
Comment 19 IDEVFH 2014-02-06 21:53:32 PST
Using the Addon SDK 1.5 with binary an ( .exe ) file included in the MyAddon.xpi file installs and works on Aurora 28.0a2 2014-02-03, but not for Firefox 27.0. I had MOOK on IRC to check the .xpi for problems but he could see any problems concerning the structure. Must be something with the addon manager between the two browsers.
Comment 20 IDEVFH 2014-02-06 22:03:39 PST
Failed to mention that although ( cfx xpi ) would not include the .exe file upon completing the archiving. I had to manually build unzip the results from ( cfx xpi ) then added to a sub-directory within the structure and rebuilt a zip then renamed to myaddon.xpi. As I stated NO problems with Aurora just FF 27.
Comment 21 IDEVFH 2014-02-07 08:25:54 PST
I checked my directory: C:\Users\IDEVFH\AppData\Roaming\Mozilla\Firefox\Profiles\b2gp6xtu.default-1353508588839\extensions

This morning I found that I needed to remove Read-Only on the files from the above directory. I'm not sure if my UAC setting possible flip to Read-Only or if Firefox applied it with the Addon Manager but once I did apply the change, Firefox 27 was successful loading the Addon SDK MyAddon.xpi with the inclusion of a sub-directory with an .exe in it to be called by the add-on. The .exe was located in the sub-directory and launched correctly as well.

I hope this info my help.

IDEVFH

Note You need to log in before you can comment on or make changes to this bug.