Last Comment Bug 733436 - Add-ons won't install when the profile is located on a FAT32 drive
: Add-ons won't install when the profile is located on a FAT32 drive
Status: RESOLVED FIXED
:
Product: Toolkit
Classification: Components
Component: Add-ons Manager (show other bugs)
: Trunk
: x86_64 Mac OS X
: -- normal (vote)
: mozilla15
Assigned To: Dave Townsend [:mossop]
:
Mentors:
Depends on: 795873
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-06 10:18 PST by Mike Kaply [:mkaply] (Out June 27-July 5)
Modified: 2012-10-01 06:11 PDT (History)
3 users (show)
ryanvm: in‑testsuite-
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
patch rev 1 (5.79 KB, patch)
2012-05-11 09:01 PDT, Dave Townsend [:mossop]
bmcbride: review+
Details | Diff | Review

Description Mike Kaply [:mkaply] (Out June 27-July 5) 2012-03-06 10:18:26 PST
We're hosting our profile on a USB stick formatted with FAT32.

We've found that add-ons will not install on this profile.

The errors we got were:

Error: ERROR addons.xpi: Failure moving /Volumes/Data 1/.se/browser/profile/extensions/staged/crashme@ted.mielczarek.org/components/._crashme.manifest to /Volumes/Data 1/.se/browser/profile/extensions/crashme@ted.mielczarek.org/components
Source File: resource://gre/modules/XPIProvider.jsm
Line: 250

Error: ERROR addons.xpi: Failed to move entry /Volumes/Data 1/.se/browser/profile/extensions/staged/crashme@ted.mielczarek.org/components/._crashme.manifest: [Exception... "Component returned failure code: 0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST) [nsIFile.isDirectory]"  nsresult: "0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST)"  location: "JS frame :: resource://gre/modules/XPIProvider.jsm :: <TOP_LEVEL> :: line 243"  data: no]
Source File: resource://gre/modules/XPIProvider.jsm
Line: 243

Error: ERROR addons.xpi: Failure moving /Volumes/Data 1/.se/browser/profile/extensions/staged/crashme@ted.mielczarek.org/components to /Volumes/Data 1/.se/browser/profile/extensions/crashme@ted.mielczarek.org
Source File: resource://gre/modules/XPIProvider.jsm
Line: 250

Error: ERROR addons.xpi: Failed to move entry /Volumes/Data 1/.se/browser/profile/extensions/staged/crashme@ted.mielczarek.org/components: [Exception... "Component returned failure code: 0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST) [nsIFile.isDirectory]"  nsresult: "0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST)"  location: "JS frame :: resource://gre/modules/XPIProvider.jsm :: <TOP_LEVEL> :: line 243"  data: no]
Source File: resource://gre/modules/XPIProvider.jsm
Line: 243

Error: ERROR addons.xpi: Failure moving /Volumes/Data 1/.se/browser/profile/extensions/staged/crashme@ted.mielczarek.org to /Volumes/Data 1/.se/browser/profile/extensions
Source File: resource://gre/modules/XPIProvider.jsm
Line: 250

Error: ERROR addons.xpi: Failed to install staged add-on crashme@ted.mielczarek.org in app-profile: [Exception... "Component returned failure code: 0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST) [nsIFile.isDirectory]"  nsresult: "0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST)"  location: "JS frame :: resource://gre/modules/XPIProvider.jsm :: <TOP_LEVEL> :: line 243"  data: no]
Source File: resource://gre/modules/XPIProvider.jsm
Line: 243

After some debugging, it was determined that on Mac, resource forks are created automatically for all the files in the zip when it is unzipped. These files begin with ._

If all of these files are removed before you restart Firefox, the add-on installs correctly.

Per Mossop:

Well XPIProvider is seeing them in the directory and attempting to do something with them. Specifically nsIFile.isDirectory is claiming they don't exist (even though it found it throughnsIFile.directoryEntries) . I'd have to debug in to find out exactly why
Comment 1 Dave Townsend [:mossop] 2012-03-06 10:24:42 PST
We could potentially workaround this by ignoring anything starting "._", but it'd be interesting to debug an isDirectory call for one of these files to see specifically what is going on.
Comment 2 Mike Kaply [:mkaply] (Out June 27-July 5) 2012-03-13 11:00:39 PDT
I discovered that even if I manage to get the add-on installed by deleting some files at install, the add-on then can't be deleted because some of the ._ files get recreated.
Comment 3 Dave Townsend [:mossop] 2012-05-10 12:47:53 PDT
Having finally got a chance to test this myself I'm not seeing the same behaviour here. On my mac (10.7) with a profile on a FAT USB stick, no additional files are created when extracting from an extension I tested (I used Context Search as an example). The extension installed and uninstalled just fine.
Comment 4 Mike Kaply [:mkaply] (Out June 27-July 5) 2012-05-10 12:50:47 PDT
I'm using 10.6.8 - not sure if that makes a difference.

Context Search doesn't have unpack:true set.

To see this, you need an add-on that is unpacked.
Comment 5 Dave Townsend [:mossop] 2012-05-10 12:53:08 PDT
I enabled extensions.alwaysUnpack first
Comment 6 Mozilla RelEng Bot 2012-05-10 23:01:47 PDT
Try run for e3462c4100e1 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=e3462c4100e1
Results (out of 57 total builds):
    success: 36
    warnings: 21
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/dtownsend@mozilla.com-e3462c4100e1
Comment 7 Dave Townsend [:mossop] 2012-05-11 09:01:41 PDT
Created attachment 623177 [details] [diff] [review]
patch rev 1

To summarise (in a long-winded fashion), OSX sometimes stores resource forks for files. On FAT which doesn't support forks as such it stores the data in what is called the AppleDouble format, which means the data file remains as-is and the resource data goes in a file prefixed with "._". OSX automatically handles keeping the resource files stuck with their data counterparts so when we go through the list of files and move the data file, OSX automatically moves the resource file, then when we come to move that (because we work from a cached list of files in the directory) it is gone and things fall over.

In this patch I sort the list of files to ensure that we encounter the main file first (meaning that if there is a resource file it gets correctly moved with its data) and then if a file has already vanished when we get to it then we just continue on ignoring that error.

Unfortunately I don't think we can build an automated test for this case. I've tested it manually though and it works for me.
Comment 8 Blair McBride [:Unfocused] (mostly unavailable, needinfo open, reviews not) 2012-05-13 22:16:48 PDT
Comment on attachment 623177 [details] [diff] [review]
patch rev 1

Review of attachment 623177 [details] [diff] [review]:
-----------------------------------------------------------------

(In reply to Dave Townsend (:Mossop) from comment #7)
> Unfortunately I don't think we can build an automated test for this case.
> I've tested it manually though and it works for me.

Heh, yea, I wondered about that.

Should we be doing this for the staging directory/processFileChanges too?

::: toolkit/mozapps/extensions/XPIProvider.jsm
@@ +1341,3 @@
>    try {
> +    let entry;
> +    while (entry = entries.nextFile)

Can this really throw? Because there are places where we don't have this in a try/finally block.
Comment 9 Dave Townsend [:mossop] 2012-05-16 11:38:21 PDT
(In reply to Blair McBride (:Unfocused) from comment #8)
> Comment on attachment 623177 [details] [diff] [review]
> patch rev 1
> 
> Review of attachment 623177 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> (In reply to Dave Townsend (:Mossop) from comment #7)
> > Unfortunately I don't think we can build an automated test for this case.
> > I've tested it manually though and it works for me.
> 
> Heh, yea, I wondered about that.
> 
> Should we be doing this for the staging directory/processFileChanges too?

I looked over those carefully this morning and I don't believe there is any problem there. The code will see the resource files as normal XPI files, attempt to load a manifest from them and fail and then just skip over them.

> ::: toolkit/mozapps/extensions/XPIProvider.jsm
> @@ +1341,3 @@
> >    try {
> > +    let entry;
> > +    while (entry = entries.nextFile)
> 
> Can this really throw? Because there are places where we don't have this in
> a try/finally block.

Don't think so, removed the try...catch.

https://hg.mozilla.org/integration/mozilla-inbound/rev/95def0230d0c
Comment 10 Ryan VanderMeulen [:RyanVM] 2012-05-16 19:57:39 PDT
https://hg.mozilla.org/mozilla-central/rev/95def0230d0c
Comment 11 Mike Kaply [:mkaply] (Out June 27-July 5) 2012-09-30 05:29:35 PDT
I just got around to retesting this (sorry) and it works much better, but still fails.

In the case of the crashme extension (http://code.google.com/p/crashme/) all the files are copied properly including forks except one file - crashme@ted.mielczark.org.json (that's not a file in the XPI, I guess that's related to staging?)

The error is slightly different though

Error: ERROR addons.manager: Exception calling provider startup: [Exception... "Component returned failure code: 0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST) [nsILocalFile.isDirectory]"  nsresult: "0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST)"  location: "JS frame :: resource:///modules/XPIProvider.jsm :: <TOP_LEVEL> :: line 1970"  data: no]
Source File: resource:///modules/XPIProvider.jsm
Line: 1970

Do you want me to open a new bug?
Comment 12 Blair McBride [:Unfocused] (mostly unavailable, needinfo open, reviews not) 2012-09-30 17:15:15 PDT
(In reply to Michael Kaply (mkaply) from comment #11)
> Do you want me to open a new bug?

Yes, please.
Comment 13 Mike Kaply [:mkaply] (Out June 27-July 5) 2012-10-01 06:11:24 PDT
Opened bug 795873

Note You need to log in before you can comment on or make changes to this bug.