557460 - Third party libraries should be centralized

Status: RESOLVED INVALID

(Firefox Build System :: General, defect)

Description

[Mike Hommey [:glandium]]

(Not sure the component is the right one)

This is a somehow general bug, and I think it should be used as a tracker bug for individual cases, if this is deemed an worthwhile goal (I do think so, but you may not agree).

Anyways, I think third party libraries should be put in a single place where they can easily be found. /third-party/ would come to mind as such a central place.

There are just too many third party libraries scattered all over the tree, that it is really hard to track when a new one is added (which happened with e.g. libffi in ctypes in 1.9.2). There are some at various depths in modules, some in media, some in gfx/cairo, some in extensions/spellcheck, some in db/sqlite3, some in dbm, some in jpeg, some in js/ctypes, and some in toolkit/crashreporter/google-breakpad.

This is a real mess...

Comment 1

[Benjamin Smedberg]

We have in general made the exact opposite decision: we want the imported code to live in a named directory which is related to its function. I'm open to change if there's a compelling reason, but I don't see one yet.

Comment 2

[(not currently active) Ted Mielczarek]

I agree with bsmedberg here. For contributors to Mozilla, it makes it a lot easier to work with Breakpad knowing it's underneath the code that uses it in toolkit/crashreporter, or the libffi code knowing it's under the js/ctypes directory that uses it.

What benefit is there to moving them elsewhere? Many of these we don't even support building with a system version anyway (like Breakpad and libffi).

Comment 3

[Mike Hommey [:glandium]]

Linux distributors want to use system libraries when it is possible. And when it is not, they also want to track what libraries are embedded in the software they distribute, for, at the very least security tracking.

For example, in Debian we (try to) keep a list of embedded copies in http://svn.debian.org/wsvn/secure-testing/data/embedded-code-copies?op=file

The layout of the mozilla code base makes such tracking very difficult. For example, I only found out about libffi by mere luck. So if you don't think having a central point for third-party libraries to be good for you, maybe can you mandate that including third-party libraries MUST be accompanied by an entry in a file at the top level directory or something like that. But somehow, I have a hard time believing such a rule would be followed strictly.

Comment 4

[Mike Shaver (:shaver emeritus)]

System libraries are an entirely different ball of support pain and "oh god no, you have to keep supporting this version we currently have, because we don't want to upgrade AND we don't want to use the version you guys work and test against and ship, WHY DO YOU HATE FREEDOM!?!?!" noise that we would do well to leave out of this conversation.

There is work underway to audit and track the 3rd-party libraries for security purposes, Dan Veditz knows more about that.

From Debian's perspective I'm not sure why "Mozilla embeds a copy of gzip, so when there's a gzip bug we should grep and see if it affects that code" is a lot different from "Mozilla implements gzip, so when we learn about a gzip bug we should look at the Mozilla code and see if it affects their implementation".  That we use sqlite instead of rolling our own transactional storage and query system, or libjpeg instead of doing our own decoding, seems like purely an implementation detail.

To be frank, I also don't think that "a distributor likes to do X, and doing X requires them to actually understand Mozilla to do it well, so we should reorganize our lives so that they don't have to understand it" is a solid basis for planning our work.  Yes, distributing an operating system is a lot of work, and it would be nice if all the pieces magically came together and all that distributors had to do what decide how to name packages and when to ship releases, but I don't think it's our responsibility to make your lives easier.  We have a lot of work to do too, and we don't go telling you that you "should" change your package model to make our lives easier.

This would have been much better off as a thread in .platform or .build describing your desire and asking for ideas on how you might fulfill it, rather than a bug whose summary tells us what "should" be done to our source tree.

Comment 5

[Mike Hommey [:glandium]]

Forget it

Comment 6

[Gervase Markham [:gerv]]

Mike H: it should be a help to know that 3rd party libraries tend to come with a BSD-ish licence, and so have to be added to about:licence. So in practice, that is something like the list you seek. It would certainly make good source material.

Gerv

Comment 7

[Justin Wood (:Callek)]

In this particular case, I think a DEVMO/MDC page on "third party libraries used in Gecko" or some such would make sense, with a table...

We could list the dir they are in, in our tree. their use (succinctly), possibly the in-tree version number?, and a link to the distro homepage for that library.

But this would be an entirely different can of worms than this bug was asking for.

Comment 8

[Mike Shaver (:shaver emeritus)]

Then why ask about it here?  Dan Veditz, as I said above, is working on an audit list -- contact him if you want to participate.

Comment 9

[Justin Wood (:Callek)]

(In reply to comment #8)
> Then why ask about it here? 

Was more commenting on why I feel INVALID is the right choice. Not asking about it, at least in my comment.