14923 - Local components tracking bug

Status: RESOLVED WONTFIX

(Core :: XPCOM, defect, P3)

Description

[Daniel Veditz [:dveditz]]

We eventually want to allow users--particularly on unix--to be able to install
local components since they may not have rights to install them to the global
installation. To some extent this overlaps the requirements to be able to
install local skins or chrome.

dp says "after 5.0", shaver wants it in 5.0, but I think has an optimistic view
of what would be required. This bug will attempt to track all the issues. If we
decide to take this on for 5.0 then bugs should be generated for the
sub-issues.

We need:
- a local component registry
- installer knowledge when to register in that one, as opposed to the global
  (or always scan the local components dir, as we do with plugins)
- discovery of local .xpt files
- discovery of local default/prefs
- discovery of local chrome registry fragments (when we have them at all!)
- some way for chrome URLs to specify local chrome
- a way to install local pref-pane add-ons (probably through XUL overlays
  specified in chrome registry fragments; once we solve this problem for
  global components and also use local fragments this is a no-op)

Is there anything else that goes with components that would have to be
duplicated at a local level?

Since I wouldn't want to have to write two installs for each component, one
that installs to a global place and one to a local place we have to have some
mechanism for the installer to switch it's idea of the "default place" for
these various items. How do we do that -- would it be automatic (check dir
permissions globally first)? User setting?

Comment 1

[Mike Shaver (:shaver emeritus)]

I accept your challenge!

Comment 2

[Mike Shaver (:shaver emeritus)]

More thoughts on this (I'll file bugs to track them, too)...

General beliefs:
- we need to do _something_ to fix 16600, because people _will_ have clock skew
or something else that varies the timestamps on the component registry, and we
can't just crash
- local should always trump system, which means that we should check in the
user's chrome world first, and load prefs from there last, etc.
- we _desperately_ need to reduce the number of places in the system that look
up files by hand, and funnel them all through something like the file locator
onto which we will graft global/local lookup smarts.  This might require moving
the file locator into XPCOM, but I don't want to do that, so I'll try and find
another way
- ``system'' vs. ``user'' may not be all we're asked to do, and whatever designs
we come up with should try to not _exclude_ such behaviour as
system->user->profile or allowing an embedding app to add an additional
component repository somewhere in the chain.  I doubt anyone on this list will
actually do system->user->profile for 5.0, but maybe someone else will?

Specific issues:
- [16599] the local-component registry problem -- other than the ``into which do
I register my component'' part, which is properly an installer issue -- isn't a
hard one: after you load the system registry, you load the per-user registry and
it gets to add to and/or override the system settings.  We might want to add a
profile: prefix analogous to rel:/abs:/lib: for specifying paths relative to the
currently active profile, or something.
- [16602] discovery of local files is a problem that I hope to be mostly solving
in pursuit of faster startup: enumerate the components directory but once,
handing files (spec/date/size) to the appropriate handler based on suffix.  Once
that's in place, it's not much harder to walk other component areas and hand
those files off as well.
- determination of where to install should be left with the user, probably with
some sane default.  Policies like ``install to system registry if you have
permissions, else per-user or per-profile'' should be expressible.  In my head,
I picture a world where the system components dir and the per-user components
dir have identical structure, so the installer would ``only'' have to vary the
prefix.  (Can you smell the drugs from there?)

I might start hacking on this tomorrow, in hopes of fixing 16600, though 16602
might have priority.  Or, I might not. =)

Comment 3

[Christopher Blizzard (:blizzard)]

I'm bored.  I need something to do.

Comment 4

[David Krause]

Please see bug 45229 (Just root can register all Mozilla modules (Breaks
mail/news and themes)).  Several distributions (SuSE,Debian,TurboLinux) that I
know of for sure are having problems distributing mozilla because of this type
of thing.  What is the status of this bug?  It would be nice if you never had to
register the components in the dist directory as root and all the writing could
be in ~/mozilla/ as the individual user.

Comment 5

[Daniel Veditz [:dveditz]]

David, this is not related to bug 45299 in any way. This is about the ability 
for users on a multi-user system to have a component in their local directory 
that is not installed or visible for other users, somewhat the way Netscape 
Navigator allows users to install local plugins in addition to the ones the 
sysadmin has installed globally.

Anyone who ships a fixed set of components can certainly pre-generate the 
troublesome files and ship them too, obviating the problem in 45299.

Comment 6

[Christopher Blizzard (:blizzard)]

As part of my RPM generation I regenerate the component.reg.  Why are other
distributions having this problem?

Comment 7

[Daniel Veditz [:dveditz]]

I think it's the chrome files causing the problems mainly, but maybe those 
other distros don't know about any of the generated files.

In any case it's irrelevant to this bug.

Comment 8

[R. Saravanan]

I have managed to run NS PR3 as non-privileged user, after installing and
running once as root. However, to install any optional component, such as
chatzilla or xmlterm, I need to run as root again to get the component installed
in the bin/components directory and then registered in component.reg. This means
that for a shared install of mozilla, a non-privileged user will only be able to
try out browser add-ons that the sys admin chooses to install. This sort of
defeats the purpose of XPCOM in a multi-user system. Hopefully local components
will make it to mozilla 1.0

As a stopgap measure, couldn't one (optionally) move the global registry
location from bin/component.reg to $HOME/.mozilla/user/component.reg, making it
user writable? (see also bug 41057) This wouldn't solve the problem of
autoregistration of local components, of course.

voting for this bug, which seems to be more appropriate than bug 41057, and
cc'ing myself

Comment 9

[R. Saravanan]

Here is *a* proposed solution for the local components problem on Unix
platforms, using symlinks. This may not be the ideal solution, but I think it
can be done without introducing any new interfaces at all.

- root installs mozilla as usual, creating $MOZILLA_FIVE_HOME/component.reg,
  $MOZILLA_FIVE_HOME/components etc. as usual
- user creates new registry $HOME/.mozilla/component.reg
- user creates new components directory $HOME/.mozilla/components
- user creates the symlink ./mozilla/components/system ->
                     $MOZILLA_FIVE_HOME/components
- user runs mozilla, which autoregisters components by recursively descending
symlinked directories
- user installs additional components in .mozilla/components as needed
- unnecessary autoregistration can be avoided by comparing the modification
dates on the system registry and the user's registry
- to ensure that user's components trump system components, one may want to
impose a particular autoregistration order for symlinked directories

I have checked that component autoregistration handles symlinked directories
recursively. The XPT stuff doesn't, but I know how to fix it.

Any comments?

Comment 10

[Dawn Endico]

dp is no longer @netscape.com. changing qa contact to default for this product

Comment 11

[Daniel Veditz [:dveditz]]

*** Bug 78465 has been marked as a duplicate of this bug. ***

Comment 12

[Doug Turner (:dougt)]

*** Bug 100368 has been marked as a duplicate of this bug. ***

Comment 13

[Doug Turner (:dougt)]

*** Bug 57134 has been marked as a duplicate of this bug. ***

Comment 14

[Doug Turner (:dougt)]

*** Bug 45703 has been marked as a duplicate of this bug. ***

Comment 15

[Judson Valeski]

Adding topembed keyword.

This is not just a problem for OS's that restrict dir access, it's an issue w/
plugin vendors as well.

Plugin vendors push plugins to various dirs, and we need to pickup those plugins
from those dirs.

I agree that until we have a real runtime versioning story, an "override"
(local/other) dir makes sense. If we just have an "override" dir, then the extra
dir scan could be static, but I'm not sure that's right. Hardcoding an
"override" dir is too confining, but I'm not sure how other apps can tell us
where to look w/ out having precidence/ordering problems themselves.

Comment 16

[John Bandhauer]

As I pointed out in bug 100368 we also need to worry about loading typelibs
(.xpt and also zip/jar files containing .xpt files). The typelib loader (xpti)
does not currently support multiple directories. My comment in bug 100368
pointed at newsgroup discussions on the topic.

xpti builds a manifest of which files contain which interface infos, but does
not track the directories of the files - just their leaf names. The way to fix
this is to add another level of factoring in xpti to talk about directories.
xpti also needs access to an ordered list of search directories it will use at
autoreg time. Note that we need to be able to store unambigous stringified
directory names in xpti's manifest (did this problem get solved for Mac?)

Comment 17

[Brendan Eich [:brendan]]

jband: I just use nsIFile's GetPath method and don't worry about the Mac.  Isn't
the Mac issue relevant only where volume names are overloaded somehow?  I've
been ignoring that when it comes to files in the user profile.

/be

Comment 18

[Judson Valeski]

is this really a tracking bug, or is it *the* bug? pulling "topembed" from this
as I think the major issue blocking some embedding folks is a plugins/xpti dir
scanning issue.

Comment 19

[Conrad Carlen (not reading bugmail)]

Brendan, using nsIFile::GetPath() is not good. On the Mac, the user can change
the name of their drive at any time, have more than one drive with the same
name, as move things around. Any of which makes full paths invalid.
nsIFile::GetPersistentDescriptor() is better. On the Mac, it returns a file
alias (encoded as text) which can be resolved by SetPersistentDescriptor() no
matter how the user moves or renames things. On other platforms, these calls
just use a path.

Comment 20

[John Bandhauer]

Yuk! It looks like you can't give a string from GetPath to 
SetPersistentDescriptor. Nor a string from GetPersistentDescriptor to 
InitWithPath. And there is no PersistentDescriptor version of NS_NewLocalFile. 
Right?

It would be nice if InitWithPath could detect up front whether it was getting an 
encoded or raw string and just do the right thing.

Comment 21

[Conrad Carlen (not reading bugmail)]

It is possible to use NS_NewLocalFile, passing nsnull for the path param and
then use SetPersistentDescriptor.
It would be better just have an NS_NewLocalFile with no params and then
InitWithPath, InitWithPersistentDescriptor, etc.

Comment 22

[Daniel Veditz [:dveditz]]

We really should figure out whether we're doing this or not. While it may seem
this kind of thing could be silently added later it affects the installer API
which we want to freeze.

Comment 23

[Winnie Lam]

edt considering minusing this bug since this would not be needed for the next
release.

Comment 24

[Judson Valeski]

http://bugzilla.mozilla.org/show_bug.cgi?id=45699 related?

Comment 25

[Daniel Veditz [:dveditz]]

Conceptually related. A user on a multi-user system might want personal
components for the same reasons they'd want their own plugins, either to add
features the sysadmin didn't install, or to override older plugins/components
with newer ones.

Internally the mechanisms are not related.

Comment 26

[Jaime Rodriguez, Jr.]

Confirming topembed- per EDT triage [Note from dougt -- "...outside the scope of
the GRE as I understand it."].

Comment 27

[Boris 'pi' Piwinger]

If this is mainly Unix/Linux, why is OS=Win NT?

pi

Comment 28

[timeless]

offhand i'd imagine that dveditz filed this bug from windows nt.

ok, so value added.  Suppose that I'm a mortal ('timeless') on a roaming system
where various network paths have: Mozilla1.0, Mozilla1.4, Mozilla1.5,
Mozilla1.6, Mozilla1.7, Mozilla1.8, Netscape7.2, Komodo2, Komodo3, Compuserve10
and a few other things.  The point of the long futuristic list is that my
components which i might want to install as timeless might not be compatible
with all of the products available to me.  Historically it has been unsafe to
leave things like a spellchecker lying around in the components directory
between upgrades...

So how do we handle this problem? Should the per user place have GRE major-minor
based folders for components? should it have Product release based folders?
should it have some way of marking whether a specific set of files should be
used with a specific product-release?

Personally I wouldn't want to have to install a component 10 times just so that
I can use it with whichever version of a product is available on my network in a
given day (the computer in question is a laptop, and it isn't impossible for it
to be configured so that it roams among networks with different versions of
mozilla).

I think a package based system might be best.

it would work something like this:

<usermagicdir>\components\<packagename>\<version>\<files>
<usermagicdir>\components\<application>\<version>\components.cdb
An xpcom app checks to see if it has a component database (cdb) and enumerates
the component packages.  If it finds new component packages it lists them for
the user along with some information about them.  The user decides which
packages should be made available to the xpcom app.  the xpcom app writes them
to its cdb with an annotation indicating they were being test flighted
(usecount:0).  when the app quits successfully it marks the components as
somewhat safe (usecount:1), it might actually handle count based on whether any
xpcom components from a package were actually used. i suppose we could actually
have the database know which components were alive when a process was running...

if the xpcom app crashes then the next time it starts it reads through its cdb
and lists components which might have been unstable. it prompts the user about
those components suggesting that the user consider disabling some of them.

this allows the user to tell an xpcom app not to use a component (perhaps they
read a release note for the component indicating it's incompatible with a
specific xpcom app), or to decide to try a component. it also allows for
components to be shared without creating 10 copies of each component.  the only
cost is a component manager interface. imo we need that interface anyway (not
necessarily for normal users, i'd settle for it as an addon/standalone app).

Comment 29

[Benjamin Smedberg]

This is WFM by addition of the components dirlist key.

Comment 30

[Pulsebot]

Pushed by ffxbld-merge:
https://hg.mozilla.org/releases/mozilla-beta/rev/11cef007eda7
[fenix] For https://github.com/mozilla-mobile/fenix/issues/14923 - Removes auto size preference toggle