385304 - (bz-par) Make bugzilla easier to install using PAR

Status: RESOLVED WONTFIX

(Bugzilla :: Installation & Upgrading, enhancement)

Description

[Steffen Mueller]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.1.2pre) Gecko/20061023 SUSE/2.0.0.1-0.1 Firefox/2.0.0.2pre
Build Identifier: 3.0

I've had some conversation with Max Kanat-Alexander about how one might go about improving the installation procedure for bugzilla. In particular about how one could ease the pain of installing dependencies from CPAN. I'll copy&paste the email I sent him.

If you're unsure what PAR or a .par file is, think about what a .jar is to Java and look at par.perl.org or search.cpan.org/dist/PAR.
-----------
Hi Max,

I sat down for a little bit and cooked up a little scheme. It's not perfect and you may decide it wouldn't benefit the users after all, but lets see:

- bugzilla gets a new subdirectory 'external'.
- All .par files which the user drops into this directory will be loaded as libraries using PAR. This works akin to "use lib 'foo';".
- Any modules in these .par files will be available to bugzilla without separate installation.
--> So far so good. This is a five-line change to Bugzilla.pm.
--> This is entirely unsuitable for CGI users (i.e. non-persistent) because the loading of PAR.pm and the scanning of those .par's takes a significant amount of time at startup.
- I wrote a script to automatically create .par's for every dependency you list in Bugzilla::Install::Requirements and every dependency thereof.
--> That script may fail for some of the weirder optional modules, I guess.
- The script also sorts the resulting .par files into four folders:
--> required
--> required_binary
--> optional
--> optional_binary
- It's rather simple to merge all non-binary required dependencies into a single .par file which the user can download and drop into 'external' instead of using CPAN.pm to install tons of modules.
- The binary stuff would have to be distributed for every platform, of course. That would add another .par per supported platform.

Well, there is a downside to this, of course. In order for this scheme to work, the user needs the PAR module which in turn needs a recent AutoLoader (5.62 which is available from CPAN or within Perl 5.10.), PAR::Dist (CPAN), Archive::Zip (CPAN) and Compress::Zlib (CPAN or within 5.10). Now, this might seem like dependency hell all over again, but not quite as badly: Except for Compress::Zlib which requires a C compiler, these modules are all pure-perl and can be bundled. AutoLoader, PAR, and PAR::Dist are just a single .pm file. Archive::Zip has a couple but isn't huge. Compress::Zlib might be a problem, but that will go away when 5.10 arrives.

The patch to Bugzilla.pm which is attached implements the change in a way that will fall back to the old behaviour when PAR is not present. The patch assumes that the code is executed within the bugzilla directory. Perhaps that needs to be changed, but I am sure bugzilla has some sort of awareness of its own location.

Furthermore, I'm attaching mentioned script for building the .par packages. It has some dependencies, but I'm sure you'll figure them out except for new versions of PAR::Dist and PAR::Dist::FromCPAN which I could not yet upload to CPAN for lack of my gpg key at work. I'm also attaching tarballs of those new versions but you can also get them from http://svn.openfoundry.org/par. You can ignore any warnings about missing signature files.

The resulting bunch of .par files can be merged into one with the following command:
perl -MFile::Copy=cp -MPAR::Dist -e 'merge_par(@ARGV); cp($ARGV[0], "required.par"); unlink($_) for @ARGV' required/*.par

perldoc PAR::Dist tells you about "merge_par" which does all the magic. Notice that it actually modifies the old pars.

I am attaching one such required.par containing the platform-independent depenencies. Sending along a copy of DBI and TT for linux/i386 seems a little excessive for testing.

Best regards,
Steffen
----------------
The preliminary patch to Bugzilla.pm would be:
----------------

--- Bugzilla.pm.orig	2007-03-16 21:17:49.000000000 +0100
+++ Bugzilla.pm	2007-06-20 14:05:32.000000000 +0200
@@ -26,6 +26,12 @@
 
 use strict;
 
+BEGIN {
+  if (eval "require PAR;") {
+    PAR->import('external/*.par');
+  }
+}
+
 use Bugzilla::Config;
 use Bugzilla::Constants;
 use Bugzilla::Auth;



Reproducible: Always

Steps to Reproduce:
1.
2.
3.

Comment 1

[Max Kanat-Alexander]

By the way, you might also be interested in bug 262269--although I think PAR is probably an easier way to go, the script in the other bug has some code in it that uses CPAN to install things directly into the Bugzilla directory, and it works. (Without having the long usr/bin/ paths in there.)

Comment 2

[Steffen Mueller]

Attachment: Script to generate .pars from bugzilla dependencies

Note that this script requires PAR::Dist 0.23 and PAR::Dist::FromCPAN 0.07 which haven't been uploaded to cpan yet. You can get an anonymous checkout from http://svn.openfoundry.org/par/PAR-Dist and http://svn.openfoundry.org/par/PAR-Dist-FromCPAN respectively.

Comment 3

[Steffen Mueller]

Attachment: Example .par of all portable dependencies for bugzilla.

For testing the patch.

Comment 4

[Steffen Mueller]

Hmm, I tried adding this comment before. I hope it doesn't show up twice now.
Here's some more thought on how to improve things for the slow CGI issue.
----

Hi Max,

Max Kanat-Alexander schrieb:
>> >> --> This is entirely unsuitable for CGI users (i.e. non-persistent) 
>> >> because the loading of PAR.pm and the scanning of those .par's takes
>> >> a significant amount of time at startup.
> > 
> > 	We have a lot of CGI users. Is there anything that can be done
> > for them? Like, can a .par be extracted into normal modules?

Well, it *does* work for CGI users. But Each CGI request will get the
full compilation+.par reading cost, naturally. Reducing the number
(!=size) of .par files that are read helps a lot.

Now, it is possible to use PAR::Dist to install the contents of a .par
into some place, but it's not perfect. I can only give you some pointers
for now - I don't know when I'll have time to work out a good,
user-friendly solution.

PAR::Dist::install_par() uses ExtUtils::Install::install(). There are
default targets taken from %Config, but of course, you have to be root
to install into the system. Once you start thinking about installing to
a different location such as into the bugzilla installation directory,
things get more involved. You can just specify a prefix a la
MakeMaker/Module::Build, but that will end up with a full /usr/blah/blah
directory structure *under the prefix path*. That's ****, of course, but
the same issue the aforementioned default installation modules have.
It's possible to give specific paths for all parts of a module
installation (lib, arch, bin, etc), so it's only a matter of a couple of
minutes of directory structure layout to make this work for bugzilla.

Best regards,
Steffen

Comment 5

[Steffen Mueller]

Okay, I thought about the CGI-user, installation issue some more.

After writing this comment, I will attach a script which, given the installation directory of bugzilla and the path to a PAR file, can install the contents of the PAR file to the "external" directory of the bugzilla installation.

It's not exactly polished, but I tested that it works for both pure-Perl as well as "binary", XS distributions.

I'll inline the patch needed to Bugzilla.pm to make this work. (Including the patch from above.) Basically, it's just a glorified "use lib". I'm not too familiar with Bugzilla's location/path handling, so this might be too simplistic to go into production.

------
--- Bugzilla.pm.orig 2007-03-16 21:17:49.000000000 +0100
+++ Bugzilla.pm 2007-06-21 12:27:42.000000000 +0200
@@ -25,6 +25,16 @@
 package Bugzilla;

 use strict;
+use File::Spec;
+BEGIN {
+  if (-d 'external') {
+    use lib File::Spec->catdir(qw/external lib/),
+            File::Spec->catdir(qw/external arch/);
+  }
+  if (eval "require PAR;") {
+    PAR->import('external/*.par');
+  }
+}

 use Bugzilla::Config;
 use Bugzilla::Constants;

-------

Cheers,
Steffen

Comment 6

[Steffen Mueller]

Attachment: Script which installs a PAR distribution into a bugzilla

Note that dropping the .par into the "external" subdirectory of bugzilla still works. This script just installs it *properly* which should improve the startup time as compared to dynamic PAR auto-extraction.

Comment 7

[Steffen Mueller]

One additional observation: The problem of introducing the dependency on Compress::Zlib (which is an XS module) can be semi-overcome in most scenarios:

- If the version of Perl is >= 5.10 (which will be released soonish), it's in the core.
- If using ActiveState's ActivePerl, which is widely used on Windows, Archive::Zip and Compress::Zlib are included in the distribution.
- If the OS is Unix/Linux, it is possible to use PAR::Dist and the installation script I attached to this enhancement request to install a .par of Compress::Zlib without actually using Compress::Zlib at all. This is possible because PAR::Dist can fall back to using the "unzip" tool which is usually present on *nix.

Basically, if it can be relied on that PAR::Dist works, and there is a pre-built .par of the dependencies for the platform, the user no longer needs to use CPAN.pm to get bugzilla running. PAR::Dist itself can be textually embedded into the installation script for bootstrapping purposes.

This means that with this mechanism, the installation instructions could become something like the following:

1) download bugzilla.tar.gz
2) unpack to destination
3) download bugzilla-yourplatform.par
4) run the installation script from the bugzilla directory to install the .par into bugzilla/external
5) optional: repeat with the platform-agnostic .par. This can be included in the other package, though.
6) optional: download bugzilla-optional-dependencies-yourplatform.par and install that the same way.
7) do the mysql setup as usual.

Additionally, it's possible to create a self-extracting binary of the whole bugzilla package using pp. Or rather, it would be a binary of an installation script which installs bugzilla plus its dependencies from a piggybacked zip(or .par) file. This would mean the user doesn't even need to download any additional files nor know how to use tar -xzf ;) It would, however, mean you need a complete binary bugzilla package for all supported platforms. In that case, you could as well use some kind of installer-generator.

Steffen

Comment 8

[Steffen Mueller]

New versions of PAR::Dist (0.23) and PAR::Dist::FromCPAN (0.07) are now available from CPAN. I just uploaded them. It can take up to a couple of hours or even days to propagate files to all CPAN mirrors.

Cheers,
Steffen

Comment 9

[Max Kanat-Alexander]

Hey Steffen! All of this is great. I'm going to have to take the chance to review it all at some point.

Is it possible to have a PAR that actually compiles the XS parts, on a *nix box? We could ship separate PARs for Windows--that's the only platform we support that wouldn't have a compiler, really. (Well, maybe Mac OS X too--it's a little trickier to get a compiler there.)

Comment 10

[Steffen Mueller]

Hi Max,

the list of suggestions has grown quite long, I fear. Sorry for the unstructured braindump.

Do I understand you correctly that you would like to install from sources when on *nix but use .par binaries for Windows and possibly MacOS? Well, a .par is a binary archive. It has the modules as they would be installed on a system. The C sources of XS modules aren't even included. It wouldn't be a problem to actually include the sources, but building from that is an entirely different beast.

However, it should be reasonably straightforward to fall back to using CPAN.pm instead of PAR if there is no .par file of a given module for the current architecture. That does, of course, introduce the CPAN.pm configuration hassle back into the game. If we were to reinvent the build process which CPAN.pm steers, we'd end up requiring quite a bit of configuration, too.

Summary:
- .par == package of blib/ subdirectory after a module build process
- building from source => needs different source than a .par file.
- building from source => Probably requires CPAN.pm or lots of work to automate
- Easiest solution: check whether a .par is available and fall back to checkinstall.pl's behaviour "Please run perl -MCPAN ..." otherwise.

I understand that you'd prefer not to supply binaries for every conceivable *nix+arch+perlversion combination since that's not possible. But having binaries for just some of the major configurations should help in >80% of cases. Everyone administrating some kind of big Unix box should be able to configure CPAN.pm anyway.

What kind of configurations would need to be supported?
- MSWin32-multi-thread-i386 + 5.8.8
- MacOS-X + whatever-default-perl-they-have (I think there are two common perls.)
And optionally, to get more coverage:
- i386-linux + 5.8.8
- i386-linux + 5.8.7
- x64-linux + 5.8.8
- x64-linux + 5.8.7
plus possibly the 5.8.4 for very old debians.

I wouldn't even try to support any 5.6ish installations. Those should be dying out on mainstream systems anyway.

That's, worst case, 9 configurations but I might have omitted some. Personally, I could build six of those (the linux ones). If my hard-drive hadn't crashed, I'd have a vmware image of w2k to build the Windows one. That only leaves the macs.

There is one more piece to the PAR puzzle which I haven't mentioned yet at all since I thought it might be overkill. If I have you confused as things are at this point, you can stop reading here.

PAR::Repository (and PAR::Repository::Client).
===============

A PAR::Repository is basically a certain directory structure containing .par's put into an ftp or http accessible place. The module comes with a handy administration script for this.

On a client side, you'd instantiate a PAR::Repository::Client object and tell it to find a suitable implementation of Module::X in one or more repositories. The major feature here is "suitable". It will look for the highest-versioned .par for your arch and perl and fall back to more portable implementations if necessary. Then, it can both load the module dynamically as well as install or update modules.

This doesn't fix the "fallback to sources if no .par found" issue, but it may help with the administration of the binary packages.

PAR::Repository::Client and its dependencies (except for Compress::Zlib, again) are pure Perl and portable, by the way.
The server side needs to support symlinks, however.

Comment 11

[Steffen Mueller]

One more note: This whole scheme covers the *Perl* dependencies only. If you have something like the libgd library, which isn't related to Perl at all, you'll have to install that separately, of course.

It *is* possible to package external libraries into a .par, but that's entirely non-trivial as the way one dll/so loads another can't be captured by an automatic packager.

This seemed obvious enough to me, but thinking about it, I realized it might not be obvious to everyone.

Comment 12

[Max Kanat-Alexander]

Hey Steffen--I just wanted to let you know that I haven't forgotten about this at all--it's one of the things I most want to see in Bugzilla 3.2.

I think what we'll do for 3.2 is just ship PARs of all the modules that don't require binary compilation (and all their dependencies), and that should greatly reduce the difficulty of installing Bugzilla. Most people have a Template-Toolkit and DBD package from their OS, anyhow.

Comment 13

[Steffen Mueller]

Hi Max,

great to hear from you!

> I think what we'll do for 3.2 is just ship PARs of all the modules that don't
> require binary compilation (and all their dependencies), and that should
> greatly reduce the difficulty of installing Bugzilla. Most people have a
> Template-Toolkit and DBD package from their OS, anyhow.

That definitely sounds like the lowest hanging fruit, I agree. Just remember to give CGI users the opportunity to install the .par's into the bugzilla directory so they don't have to pay the price of loading the .par's for every hit.

Let me know if and how I can help.

Best regards,
Steffen

Comment 14

[Max Kanat-Alexander]

  Okay, so here's my plan:

  1. Create a PAR::Repository, par.bugzilla.org.
  2. Have checksetup download the needed PARs from that repository.
  3. If checksetup is unable to download the PARs, it can give URLs to download
     the PARs, and place them in a special directory where checksetup will find
     them locally.
  4. Install the PARs as modules locally into a lib/ or external/
     directory in the Bugzilla directory itself. That is, Bugzilla itself,
     except the installation script, will never have to "use PAR".

Comment 15

[Max Kanat-Alexander]

Oh, and in addition, Bugzilla itself will always ship with an included PAR::Repository::Client in the lib/ or external/ directory, including all of its deps except Compress::Zlib, which checksetup will check for the existence of and recommend that you install it if it's not installed.

If a module is not available as a PAR (for example, because it contains XS code and we don't have a compiled version on par.bugzilla.org) then we will recommend the user install it via CPAN or PPM.

Comment 16

[Steffen Mueller]

That sounds like a good plan to me. Please let me know if you need any changes on the PAR::Repository(::Client) side of things.

Comment 17

[Max Kanat-Alexander]

(In reply to comment #16)
> That sounds like a good plan to me. Please let me know if you need any changes
> on the PAR::Repository(::Client) side of things.

  I might, I'm not sure yet, though. Does the install_module subroutine allow you to specify an installation directory, or do I have to use PAR::Dist's install_par? 

  Basically, what I'll need is logic that allows me to do the follwing:

  1) Is version X or higher of "Some::Module" already installed? (I have logic
     in Bugzilla already that tells me this, so if it's not in PAR, that's
     fine.)
  2) If not, install a PAR from the Repository into a certain directory
     (probably ./external/).

Comment 18

[Steffen Mueller]

(In reply to comment #17)
>   I might, I'm not sure yet, though. Does the install_module subroutine allow
> you to specify an installation directory, or do I have to use PAR::Dist's
> install_par? 

I'll quote the PAR::Repository::Client docs here:

  installation_targets
  
  Sets the installation targets for modules and scripts if any
  arguments are passed. Returns the current setting otherwise.
  
  Arguments should be key/value pairs of installation targets as
  recognized by the install_par() routine in PAR::Dist. The
  contents of this hash are passed verbatim to every call to
  install_par() made by this package.

[...]

So the answer is no, you don't need to download and locate the .par and then install it with PAR::Dist::install_par. But you *do* need to learn the somewhat convoluted parameter passing to install_par(). Or, you can copy the arguments from the installation script I posted to the other bug report.

>   Basically, what I'll need is logic that allows me to do the follwing:
> 
>   1) Is version X or higher of "Some::Module" already installed? (I have logic
>      in Bugzilla already that tells me this, so if it's not in PAR, that's
>      fine.)

PAR::Repository(::Client) is intended to be used somewhat "live". I.e. if you try to load a module in an application which isn't available locally, it instead loads it from the repository. This logic can be warped to be an installation mechanism:
- The install script calls ->install_module("Some::NameSpace") on a client object.
- The client tries to locate the newest version in the repository.
- The client installs it locally.

The "upgrade_module" method works similarly, but it tries to require the module locally first. If that fails or if the version is undefined, the corresponding .par is sought in the repository and locally installed.

Best regards,
Steffen

Comment 19

[Max Kanat-Alexander]

  A few weeks ago I was researching the popularity of various bug-tracking tools. I discovered that there are nearly as many hits for "about trac" in Google as there are for the simple word "Bugzilla".

  Every time I've talked to somebody about Trac, they've said the single reason that they used it was that it was *easier to install than Bugzilla*.

  Thinking about this more today, I've decided to make this bug my number one priority. I don't want this to wait for 4.0, where Bugzilla might fade from its #1 position in the open-source bug-tracking world before we get that release out.

  I think this is our #1 barrier to adoption, and so it's my #1 priority.

Comment 20

[Max Kanat-Alexander]

(In reply to comment #19)
>   Every time I've talked to somebody about Trac, they've said the single reason
> that they used it was that it was *easier to install than Bugzilla*.

  People also cite that Trac has SVN-browsing support and an integrated Wiki, but the main draw to *sysadmins* seems to be ease of installation. Setting up a separate Wiki isn't terribly hard, and bug 390964 (or somebody re-writing Bonsai) would handle the VCS-browsing thing.

Comment 21

[Max Kanat-Alexander]

  Okay, so I'm running into a problem with this:

  PAR and PAR::Repository::Client have a long prerequisite chain of non-core modules.

  This means that we must do one of the following:

  1) Check all of PAR, PAR::Repository::Client, and all of the prerequisite
     chain (except certain XS modules) into our CVS.

     Pros: Easy for users who download Bugzilla from CVS.
     Cons: Users still have to use CPAN to install XS modules. We have to
           track a *lot* of CPAN modules and constantly decide if we want
           to import the latest versions into CVS. This won't work, because
           nobody will do it.

     In other words, we're not doing that.

  2) Require that users install PAR::Repository::Client themselves if they want
     to have checksetup.pl auto-install modules.

     Pros: Don't have to add very much code to our CVS.
     Cons: All users still have to use CPAN, with all of the trouble they have
           with it. So this isn't really improving our ease of installation.
           PAR & its prereqs aren't packaged for most distros, so people who
           use packages will be in trouble.

  3) Create a script that can do CPAN installs for people automatically.

     Pros: Good for CVS users, and we can bundle the modules in our tarballs,
           so tarball users never have to worry about it.
     Cons: The API of CPAN/CPANPLUS changes constantly, and it would be a 
           huge pain to maintain this script. 
           But if we're going to do this, why are we using PAR at all?

  Perhaps we could combine #2 and #3--CVS users have to install PAR::Repository::Client, but in our tarballs we ship them. That way we don't have to check our cpan-install script into CVS, we only have to use it as part of the release process. We could maybe check it into contrib/.

  Still, at that point I might as well just bundle *all* of the prereqs, right? I suppose we could use PAR to get the XS modules. But as long as PAR itself has XS modules in its dependency chain, users will have to use CPAN to install XS modules. We only have one or two XS modules in Bugzilla's entire dependency chain, I think, and PAR::Repository::Client all by itself has more than that, I think.

  So I don't see the advantage of using PAR, at this point. If PAR::Repository::Client stops depending on any XS modules anywhere in its dependency chain, it would be helpful for installing XS modules. But right now I think it would actually *add* complexity.

Comment 22

[Steffen Mueller]

Hi Max,

(In reply to comment #21)
>   Okay, so I'm running into a problem with this:
> 
>   PAR and PAR::Repository::Client have a long prerequisite chain of non-core
> modules.

That's true. PAR needs:
- Compress::Zlib (XS)
- Archive::Zip (perl)
- newer version of AutoLoader than shipped with perl 5.8.8 (perl)
- PAR::Dist (perl)

PAR::Repository::Client is somewhat heavier. It additionally requires version.pm which is also an XS modules. All other dependencies should be pure-perl. The bulk of its dependencies come from requiring DBM::Deep, which, at the time of its adoption had less dependencies than it has now. In fact I had chosen it because I considered it light-weight at the time.

Both of the XS modules here will become core in 5.10. I realize that this will be too late. Until the adoption rate of 5.10 reaches a reasonable 80%, too much time will have passed, so this is still a problem.

>   This means that we must do one of the following:
> 
>   1) Check all of PAR, PAR::Repository::Client, and all of the prerequisite
>      chain (except certain XS modules) into our CVS.
[...]
>   2) Require that users install PAR::Repository::Client themselves if they want
>      to have checksetup.pl auto-install modules.
[...]
>   3) Create a script that can do CPAN installs for people automatically.
[...]
>   Still, at that point I might as well just bundle *all* of the prereqs, right?
> I suppose we could use PAR to get the XS modules. But as long as PAR itself has
> XS modules in its dependency chain, users will have to use CPAN to install XS
> modules. We only have one or two XS modules in Bugzilla's entire dependency
> chain, I think, and PAR::Repository::Client all by itself has more than that, I
> think.

I agree with your, but that last bit is something of an overstatement, if you ask me. Nonetheless, I can see that you'd rather not get yourself even more dependencies.

>   So I don't see the advantage of using PAR, at this point. If
> PAR::Repository::Client stops depending on any XS modules anywhere in its
> dependency chain, it would be helpful for installing XS modules. But right now
> I think it would actually *add* complexity.

That is exactly the reason I've been so keen on having 5.10 see the light of the day. No more XS dependencies...

So, too bad this didn't work out. Perhaps we can reconsider it once 5.10 is widespread.

Best regards,
Steffen

Comment 23

[Max Kanat-Alexander]

(In reply to comment #22)
> So, too bad this didn't work out. Perhaps we can reconsider it once 5.10 is
> widespread.

  Yeah, that's definitely an option! I love PAR; it's a great module and really handy. It just unfortunately won't work for us at this time.

Comment 24

[Max Kanat-Alexander]

I really think install-module makes things easy enough, and we don't need to use PAR now.