504397 - Plug-ins registry not updated when software update updates the JEP

Status: RESOLVED WORKSFORME

(Core Graveyard :: Plug-ins, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.1pre) Gecko/20090715 Shiretoko/3.5.1pre ID:20090715031842

Today I noticed that the plug-ins registry has not been updated for the embedded java plugin. Now that Java Embedding Plugin 0.9.7.2 has been released and should be present in current nightly builds about:plugins and the plugin tab of the add-ons manager still show me the older version Java Embedding Plugin 0.9.7.1.

Here a directory listing for the plug-ins folder:

$ ls -l /Applications/Shiretoko.app/Contents/MacOS/plugins/
total 0
drwxr-xr-x@ 3 henrik  admin  102  8 Feb 12:28 DefaultPlugin.plugin
drwxr-xr-x@ 3 henrik  admin  102  8 Feb 12:28 JavaEmbeddingPlugin.bundle
drwxr-xr-x@ 3 henrik  admin  102  8 Feb 12:28 MRJPlugin.plugin

$ ls -l /Applications/Shiretoko.app/Contents/MacOS/plugins/MRJPlugin.plugin/
total 0
drwxr-xr-x@ 6 henrik  admin  204 15 Jul 14:26 Contents

$ ls -l /Applications/Shiretoko.app/Contents/MacOS/plugins/MRJPlugin.plugin/Contents/
total 24
-rw-r--r--@ 1 henrik  admin  5078 15 Jul 14:26 Info.plist
drwxr-xr-x@ 6 henrik  admin   204 15 Jul 14:26 MacOS
-rw-r--r--@ 1 henrik  admin     8 15 Jul 14:26 PkgInfo
drwxr-xr-x@ 4 henrik  admin   136 15 Jul 14:26 Resources

This problem only occurs in my daily profile. Other profiles or new ones don't suffer from it. The pluginreg.dat has the following content for the Java plugin:

$ ls -l pluginreg.dat 
-rwx------  1 henrik  staff  6412 15 Jul 20:32 pluginreg.dat

/Applications/Shiretoko.app/Contents/MacOS/plugins/MRJPlugin.plugin:$
MRJ Plugin version 1.0-JEP-0.9.7.1:$
1234092526000:0:5:$
Runs Java applets using the latest installed versions of Java. For more information: <A HREF=http://javaplugin.sourceforge.net/>Java Embedding Plugin</A>. Run version test: <A HREF=http://gemal.dk/browserspy/java.html>Java Information</A>.:$
Java Embedding Plugin 0.9.7.1:$
16
0:application/x-java-applet;version=1.3:Embedded Java Applet:xja13:$
1:application/x-java-applet;version=1.5:Embedded Java Applet:xja15:$
2:application/x-java-applet;version=1.4.1:Embedded Java Applet:xja141:$
3:application/x-java-applet;version=1.1.3:Embedded Java Applet:xja113:$
4:application/x-java-applet;version=1.2:Embedded Java Applet:xja12:$
5:application/x-java-applet;version=1.2.1:Embedded Java Applet:xja121:$
6:application/x-java-applet;version=1.1:Embedded Java Applet:xja11:$
7:application/x-java-applet;version=1.4.2:Embedded Java Applet:xja142:$
8:application/x-java-applet;version=1.1.1:Embedded Java Applet:xja111:$
9:application/x-java-applet;version=1.3.1:Embedded Java Applet:xja131:$
10:application/x-java-applet;version=1.6:Embedded Java Applet:xja16:$
11:application/x-java-applet:Embedded Java Applet:xja:$
12:application/x-java-vm:Embedded JVM:xjv:$
13:application/x-java-applet;version=1.4:Embedded Java Applet:xja14:$
14:application/x-java-applet;version=1.1.2:Embedded Java Applet:xja112:$
15:application/x-java-applet;version=1.2.2:Embedded Java Applet:xja122:$

I have installed Shiretoko in February and I believe it was the 8th. Since then I have only run AUS to update my version of Shiretoko. So I wonder if something is wrong with the automatic update.

Comment 1

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

This is almost certainly a dupe of one of the many existing bugs about pluginreg.dat not being properly regenerated when needed.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Probably. Steven seems to have found the problem specific to the embedded Java plugin. So we will try to fix this bug which will hopefully also fix all the other ones.

Comment 3

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

For instance, see bug 313700 (which is where I'd dupe this); the JEP+Fx software update was mentioned there several times.

Comment 4

[Robert Strong (they/them - no direct email)]

Note: software update will only update the installed files and it is the responsibility of the individual components to perform any actions they may need after the app has been updated. This is especially true when files in the profile need to be updated since the user may have multiple profiles all of which will need updating or one user may have updated the app and a second user launch the app after the software update.

Comment 5

[Steven Michaud [:smichaud] (Retired)]

(In reply to comment #4)

The issue here is that software update, though it updates the JEP's component files correctly, doesn't change any of those files' dates.

This means that the plugin loading code (which also updates pluginreg.dat) doesn't realize that the JEP has been updated -- since it sees a bundle with the same name and path and date.

Comment 6

[Steven Michaud [:smichaud] (Retired)]

> The issue here is that software update, though it updates the JEP's
> component files correctly, doesn't change any of those files' dates.

Oops, this is wrong.

Software update changes the *files'* dates, but not the *directories'*
dates.  And the OS (apparently) gets the date of a bundle from the
date of its top-level directory.

Comment 7

[Steven Michaud [:smichaud] (Retired)]

Let's not dup this bug to another.

Instead let's narrow its focus to the plugins registry not getting updated when software update is run.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Just an information which could also be helpful. There are also a couple of other folders/bundles which date don't get updated:

$ ls -l /Applications/Shiretoko.app/Contents/Plug-Ins/
total 0
drwxr-xr-x@ 3 henrik  admin  102  8 Feb 12:28 PrintPDE.plugin

$ ls -l /Applications/Shiretoko.app/Contents/MacOS/
total 77408
drwxr-xr-x@   3 henrik  admin       102  8 Feb 12:28 crashreporter.app
drwxr-xr-x@   5 henrik  admin       170  8 Feb 12:28 defaults
drwxr-xr-x@   5 henrik  admin       170  8 Feb 12:28 plugins
drwxr-xr-x@   3 henrik  admin       102  8 Feb 12:28 updater.app

Robert, if this could raise other problems and it should be filed as a new bug I can do that.

Comment 9

[Robert Strong (they/them - no direct email)]

I'm not fluent with Mac OS X but this is the first time I know of where there has been a problem like this since the updater was created way back when. I suspect there are OS specific behaviors involved here and that the best solution would be to not rely on app bundle times or directory times for whether files have been updated / added.

Comment 10

[Steven Michaud [:smichaud] (Retired)]

I don't think FF has ever bundled any other plugins than the JEP.  So this problem, though old, is unique.

I'm doubt there's any good alternative to relying on app bundle timestamps.

How difficult would it be to change directory timestamps?  Do you foresee it causing trouble in other contexts?

Comment 11

[Steven Michaud [:smichaud] (Retired)]

> I don't think FF has ever bundled any other plugins than the JEP.  So this
> problem, though old, is unique.

Though Henrik (in comment #8) points out other cases of including app bundles.

I don't know if these are checked for updated information (like plugins).

Comment 12

[Robert Strong (they/them - no direct email)]

Actually, I ran into this same problem with the extension manager on Windows with directories on FAT filesystems for the appManaged add-ons a long time ago. To solve this I reloaded the metadata for appManaged add-ons on app upgrade to fix this.

Adding code to the updater to solve problems for individual components was not considered a good path to take back then when I wasn't the owner of app update and I still agree with that decision since breaking app update could strand users.

This problem also has a chicken / egg component in that any changes to the updater would not be available until the next update after the user had the updater with the new code.

I suspect that the Mac OS X filesystem doesn't update the directory / app bundle timestamps the same as directories on FAT filesystems don't when the files it contains are updated.

Another route that could be taken is to force a complete update for the specific app bundle in question which would first delete the app bundle and then add the new one which would then have new time stamps.

Comment 13

[Steven Michaud [:smichaud] (Retired)]

> I suspect that the Mac OS X filesystem doesn't update the directory
> / app bundle timestamps the same as directories on FAT filesystems
> don't when the files it contains are updated.

Actually, they *do* (generally) get updated -- whether you change a
file in the graphical UI (by opening and saving it in the Finder) or
at a Terminal prompt (e.g. using emacs).  I don't know why Software
Update's methods for updating files don't cause the containing
directory to be updated.  Might there be code that specifically
prevents this?

> Another route that could be taken is to force a complete update for
> the specific app bundle in question which would first delete the app
> bundle and then add the new one which would then have new time
> stamps.

This would work fine for JEP updates -- which generally change every
file in the plugin.  I'm not sure how efficient it would be in
general, though.

Comment 14

[Robert Strong (they/them - no direct email)]

(In reply to comment #13)
>...
> Actually, they *do* (generally) get updated -- whether you change a
> file in the graphical UI (by opening and saving it in the Finder) or
> at a Terminal prompt (e.g. using emacs).  I don't know why Software
> Update's methods for updating files don't cause the containing
> directory to be updated.  Might there be code that specifically
> prevents this?
The methods used to apply diffs may very well use methods that circumvent this. Perhaps Josh would know so cc'ing him.

Comment 15

[Steven Michaud [:smichaud] (Retired)]

I just checked on Linux (on Kubuntu 9.04 with the ext3 filesystem) and Software Update does update the distro's directory timestamps there.  (I downloaded a Shiretoko nightly from last week and chose Help : Check for Updates.)

Odd the same thing doesn't happen on OS X.

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to comment #12)
> Another route that could be taken is to force a complete update for the
> specific app bundle in question which would first delete the app bundle and
> then add the new one which would then have new time stamps.

I already had some complete updates since I have installed Shiretoko on this box in February. But those timestamps never got updated.

Comment 17

[Robert Strong (they/them - no direct email)]

That is entirely possible on Mac with bundles for complete updates. If the mar is creating by diff against a build that doesn't have the bundle and the bundle is marked as a forced update it should delete the original and create the new with the new time stamps.

Comment 18

[Robert Strong (they/them - no direct email)]

After thinking on this a bit more it is quite possible that my comment #17 is incorrect for app bundles / directories and forcing a complete update won't work.

Comment 19

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

It occurred to me that there's a possibility that bug 493503 is related, if the main app bundle's timestamp is related to how LaunchServices and the Finder decide when to rescan an app.

Note that Xcode/xcodebuild is very careful *always* to 'touch -c' any bundle when you rebuild it.  I haven't paid attention to see what Mac OS X Software Update does, but I would be surprised if it didn't do something similar or at least related.

It seems like you could probably solve this "stale bundle date problem" by making updater.app 'touch' (perhaps even with the proper timestamp for the given bundle) any bundles whose files it changes after applying the diffs, if fixing an entire class of update-related problems on a platform is in scope for app update code.

Comment 20

[Robert Strong (they/them - no direct email)]

I am fine with updater doing this for bundles / directories especially if Apple's app update does something similar.

Comment 21

[Robert Strong (they/them - no direct email)]

The bundle's timestamp is updated on each successful update now. Could someone check to see if this bug is now fixed?

Comment 22

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Robert, the bundle we were concerned about getting updated in this bug was the JEP bundle, not the main app bundle (although that's not completely clear all the time in the comments).  Unless I missed something, bug 600098 only handles touching the app bundle, so it wouldn't fix this (see also bug 600098 comment 16).

Comment 23

[Robert Strong (they/them - no direct email)]

I wasn't positive whether it was the app bundle or a different bundle. It sounds like it would be best if App update could enumerate all of the dirs / files after an update and touch all bundles / dirs after a successful update to avoid this issue for other bundles, etc.

Comment 24

[Robert Strong (they/them - no direct email)]

Attachment: patch rev1

Comment 25

[Robert Strong (they/them - no direct email)]

Attachment: patch rev1

forgot to fix the editor on my Mac to insert spaces instead of tabs... other than that this is the same as the first patch

Comment 26

[Robert Strong (they/them - no direct email)]

Attachment: patch with tests

I wonder if this should also touch any Info.plist files it finds?

Comment 27

[Robert Strong (they/them - no direct email)]

The tests also cover bug 600098

Comment 28

[Justin Dolske [:Dolske]]

Comment on attachment 480563 [details] [diff] [review]
patch with tests

>+++ b/toolkit/mozapps/update/updater/updater.cpp
...
>+# include <ftw.h>

Hmm, the OS X manpage says ftw() is deprecated, but it also says it's in POSIX so I don't think it's going to be removed any time soon.

But fts() -- the replacement -- does look like it has some nice features, such as avoiding directory cycles. Fairly unlikely we'd find this in our app bundle, but there are some crazy people out there... :) Worth looking into?


>+      ftw(cwd, ftw_touch_dir, 1);

It's a little sadmaking that this makes up walk the whole appbundle, but I suppose this brute-force approach isn't as bad as it could be, because the updater is likely to be touching a lot of it anyway, so it should be cached.

Still, is it worth considering alternatives? Maybe hardcoded list of bundle dirs to touch? Or add support for a TOUCH action in the manifest, and have the MAR/manifest generation include "TOUCH foo/bar/baz.app/" iff a file in a bundle is updated?

Comment 29

[Robert Strong (they/them - no direct email)]

(In reply to comment #28)
> Comment on attachment 480563 [details] [diff] [review]
> patch with tests
> 
> >+++ b/toolkit/mozapps/update/updater/updater.cpp
> ...
> >+# include <ftw.h>
> 
> Hmm, the OS X manpage says ftw() is deprecated, but it also says it's in POSIX
> so I don't think it's going to be removed any time soon.
> 
> But fts() -- the replacement -- does look like it has some nice features, such
> as avoiding directory cycles. Fairly unlikely we'd find this in our app bundle,
> but there are some crazy people out there... :) Worth looking into?
Will do.

> 
> >+      ftw(cwd, ftw_touch_dir, 1);
> 
> It's a little sadmaking that this makes up walk the whole appbundle, but I
> suppose this brute-force approach isn't as bad as it could be, because the
> updater is likely to be touching a lot of it anyway, so it should be cached.
> 
> Still, is it worth considering alternatives? Maybe hardcoded list of bundle
> dirs to touch? Or add support for a TOUCH action in the manifest, and have the
> MAR/manifest generation include "TOUCH foo/bar/baz.app/" iff a file in a bundle
> is updated?
I very much want to avoid having another list to maintain / overlook as we have with removed-files.in and package-manifest.in. I considered adding another action but chose not to since keeping track of which directories had changes would be more complicated with little savings in execution time. If we were to just do bundles I would very much prefer enumerating all files / dirs vs. maintaining another list.

I also noticed one time that the dock icon was not updating on icon modification and after performing a touch on the parent directory it did update. I need to investigate exactly will cause the dock icon to refresh.

Comment 31

[Steven Michaud [:smichaud] (Retired)]

Bug 613840 is a dup of this bug.  But there Java applets stopped
working until the user deleted his old (inaccurate) pluginreg.dat
file.

I didn't know this was possible.  A new development?

Comment 32

[Robert Strong (they/them - no direct email)]

I don't think there is anything else that needs to be done here anymore but I'm not sure so I'm reassigning to default.

Comment 33

[Georg Fritzsche [:gfritzsche]]

The Java Embedding Plugin is not used anymore and i haven't seen any other bugs on pluginreg.dat not being updated recently.
Closing.