207039 - add description how to install bugzilla under regular user privileges

Status: RESOLVED FIXED

(Bugzilla :: Documentation, enhancement, P2)

Description

[Ralf Hauser]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030519
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030519

mysql can be run as user.
cgi-bin as well.

Therefore, it would be great to be able have a bugzilla instance set up without
requiring root privileges on the server.

Reproducible: Always

Steps to Reproduce:
1.
2.
3.



1) in http://www.bugzilla.org/docs216/html/stepbystep.html#AEN605, it would be
great to provide a full example of the "stanza" (or is the sample in
http://www.bugzilla.org/docs216/html/extraconfig.html#htaccess sufficient?)
2) I guess, I would have to run the perl script in
http://www.bugzilla.org/docs216/html/stepbystep.html#AEN624

Any further precautions I have to take?

3) On a side note, is there any precautions one would have to take to run it
under https or is that transparent?

Comment 1

[Zach Lipton [:zach]]

docs issue

Comment 2

[Ralf Hauser]

a first report on what all is problematic when trying to install it under
non-root privileges:

i) Recommendation: get ncftpget for 
<<perl -MCPAN -e 'install "Bundle::Bugzilla"'>> it can found in
http://www.ncftp.com/download/ or 
ftp://ftp.ncftp.com/ncftp/binaries/ncftp-3.1.5-*.tar.gz 

ii) ./checksetup.pl results in
   perl -MCPAN -e 'install "DBD::mysql"'
   Minimum version required: 1.2209
   perl -MCPAN -e 'install "AppConfig"'
   Minimum version required: 1.55
   perl -MCPAN -e 'install "Template"'
   Minimum version required: 2.07
   perl -MCPAN -e 'install "Date::Parse"'

ii) perl -MCPAN -e 'install "AppConfig"'
tries to install where I don't have rights:
--> http://rt.cpan.org/NoAuth/Bug.html?id=2663

iii) perl -MCPAN -e 'install "Template"'
<<...
Warning: prerequisite AppConfig 1.52 not found.
Checking if your kit is complete...
Looks good
...>>
and then, the same problem that it tries to install where I don't permissions to
do so and don't ask for an alternate directory.
--> http://rt.cpan.org/NoAuth/Bug.html?id=2664 (hope this will get fixed soon!)
   
iv) same with Date::Parse
--> http://rt.cpan.org/NoAuth/Bug.html?id=2665

v) The problem of "DBD::mysql" is described in
http://rt.cpan.org/NoAuth/Bug.html?id=121 - I symlinked to mysql_config from a
directory in my path and got one step further, but not successful 
--> http://rt.cpan.org/NoAuth/Bug.html?id=2666

As I am writing this, somebody claims that MakeMaker is responsible for this.
Unfortunately, I don't relevant documentation about this.
--> http://rt.cpan.org/NoAuth/Bug.html?id=2668

Comment 3

[Ralf Hauser]

see also http://bugzilla.mozilla.org/show_bug.cgi?id=207455 for the solution
suggested by the MakeMaker owner

a related bug possibly is also http://bugzilla.mozilla.org/show_bug.cgi?id=186866

Comment 4

[Jacob Steenhagen]

*** Bug 211986 has been marked as a duplicate of this bug. ***

Comment 5

[SineSwiper]

Attachment: New non-root section in Installation notes

See attached.  Patches the installation.xml file.  However, I don't have a good
environment for Jade, so I can't compile it to an HTML file.  Can somebody
patch this, compile it, and see if it works and/or attach the HTML file?

Comment 6

[Ralf Hauser]

o.k. found the file as
http://lxr.mozilla.org/mozilla/source/webtools/bugzilla/docs/xml/installation.xml

Does it really make sense to attach the html here - shouldn't the bugzilla folks
just integrate your patch? 
Anyway, I had troubles with my version of dbjade
(https://p4u.ch/public/cygwin_usr_local_bin/dbjade) too:
   a) I had to uncomment the DOCTYPE in installation.xml
   b) I can only handle "DocBook V4.1//EN" and not "DocBook XML V4.1.2//EN" - I
guess I first would have to upgrade
   c) got various errors starting with <<openjade:installation.xml:51:12:E:
general entity "min-mysql-ver" not defined and no default entity>>
==> I guess, one should only try to recreate this with
http://lxr.mozilla.org/mozilla/source/webtools/bugzilla/docs/makedocs.pl anyway...

Comment 7

[SineSwiper]

Attachment: New non-root section in Installation notes

Finally got jade to work.  It only took me three hours.

Anyway, here's the fixed version.

Comment 8

[SineSwiper]

Attachment: Non-root section HTML page

And here's the HTML file.

Comment 9

[SineSwiper]

What's the status on this?  Can this be added into the docs or not?  I don't 
want this to go to waste, as this is fairly useful information.

Comment 10

[SineSwiper]

Comment on attachment 127737 [details] [diff] [review]
New non-root section in Installation notes

Marking as "review?" on attachment...

Comment 11

[Tobias Burnus]

Small remark:
> Since you probably can't set up a symbolic link to /usr/bonsaitools/bin/perl as 
> a non-root user, you will need to hack the scripts to point to the right perl:
The current developer version (2.17.4) uses now /usr/bin/perl per default.

Comment 12

[Jacob Steenhagen]

My initial comments are:
* This is obviously written against 2.16.3, which is no big deal; it should
  require only minor updates to also put it in the tip docs. In fact, it's
  probably better that way :)
* I'm not sure if I like the quoted section from the CPAN docs in a 
  <programlisting/> container... though I'm also not sure what to use instead.
* The CPAN commands, however, should be contained in such a container (or more
  appropriately, the <screen/> container) instead of <computeroutput/>.
* Even though it doesn't seem to make any difference in the generated HTML, I'm
  still trying to make a habbit of adding |class="directory"| to <filename/>
  tags when they enclose a directory instead of a file.

Comment 13

[Joel Peshkin]

Actually, this can be done.. I have done it in multiple installations... (Linux
and Solaris)

1) Build perl from source, specifying /home/me/perl/ as the prefix. This is time
consuming, but simple.  It would be interesting to find out if someone has an
rpm that permits an end user to install.

2) Since you own the entire perl installation, you can install the modules from
CPAN with no permissions issues.  Just use /home/me/perl/bin/perl -MCPAN -e shell

3) Set up your /home/me/http/conf/httpd.conf to use a high-number port (like
8080).  Start httpd using the -d /home/me/http option.

4) mysql does not need to be started as root. Build it with PREFIX set to
/home/me/mysql or use pre-installed executables, specifying that you want to put
all of the data files in /home/me/mysql/data.  If there is another mysql server
running on the system and you do not own it, then use the -P command to specify
a TCP port that is not in use.
   When you start mysqld the first time, you then connect to it as "root" and
grant privs to other users.  The mysql root account has nothing to do with the
*nix root account.

5) Install the bugzilla files to /home/me/bugzilla/test/

6) Change the perl paths in the bugzilla files in the manner you noted 

7) configure and go

If you use this approach, there is no need to be root for anything. All you need
is a compiler.  Naturally, this needs to be made into a bit more of a cookbook
for the docs.  If someone would follow this recipe and take good notes, then
this could go into the docs.

Comment 14

[SineSwiper]

> The current developer version (2.17.4) uses now /usr/bin/perl per default.

True, though the user may have Perl in a non-standard location, so the same
instructions still apply, though I'd need to change the wording.

Besides, this was mainly designed for the stable version, anyway.

> This is obviously written against 2.16.3, which is no big deal; it should
> require only minor updates to also put it in the tip docs. In fact, it's
> probably better that way :)

Yeah, I figured that this could be written for the stable version, inserted 
in the docs, and then tweaked for the dev docs.  I'm not sure when the dev 
version will become official "stable", so we can help out non-root kids now,
in the meantime, with this :)

> I'm not sure if I like the quoted section from the CPAN docs in a 
> <programlisting/> container... though I'm also not sure what to use instead.

I was thinking the same thing.  I'm new to Docbook XML, so I basically just
followed by example.

> The CPAN commands, however, should be contained in such a container (or more
> appropriately, the <screen/> container) instead of <computeroutput/>.

How is this different from the bash# prompt?  I modelled this off of the 4.1 
install docs, so that's the tags they used there.

> Even though it doesn't seem to make any difference in the generated HTML, I'm
> still trying to make a habbit of adding |class="directory"| to <filename/>
> tags when they enclose a directory instead of a file.

How does that work?  Just add that verbatim, or put something in place 
of "directory"?

Comment 15

[SineSwiper]

> Actually, this can be done.. I have done it in multiple installations...
> (Linux and Solaris)

Errr...what can be done?  Installing Apache?  Did I catch you mid-thought? :)

> 1) Build perl from source, specifying /home/me/perl/ as the prefix. This is
> time consuming, but simple.  It would be interesting to find out if someone
> has an rpm that permits an end user to install.

Heh...I don't want to even mess with end-user RPMs.  I don't even think that's
possible anyway, as it needs access to the RPM database, which is usually
read-only.

> 2) Since you own the entire perl installation, you can install the modules
> from CPAN with no permissions issues.  Just use /home/me/perl/bin/perl
> -MCPAN -e shell

I guess that would be another option, though the install of perl would take
up a lot of space, which might be a problem for users with quotas (read: most
virtual host users).

> 3) Set up your /home/me/http/conf/httpd.conf to use a high-number port (like
> 8080).  Start httpd using the -d /home/me/http option.

Where's step #2.5 that says: Install Apache?  I dunno about you, but I don't
have a httpd.conf that I can edit.

> 4) mysql does not need to be started as root. Build it with PREFIX set to
> /home/me/mysql or use pre-installed executables, specifying that you want to
> put all of the data files in /home/me/mysql/data.  If there is another mysql
> server running on the system and you do not own it, then use the -P command
> to specify a TCP port that is not in use.
>    When you start mysqld the first time, you then connect to it as "root"
> and grant privs to other users.  The mysql root account has nothing to do
> with the *nix root account.

Wait, wait, wait...how does one keep a daemon running when you're not root?
Even if you can, the system admin is going to beat you when s/he finds it.
I could add some instructions to the current docs, but I'd put in a note that
one should ask for permission before you start up other daemons.

You want to help me out on the specifics?

Comment 16

[Jacob Steenhagen]

> How is this different from the bash# prompt?  I modelled this off of the 4.1 
> install docs, so that's the tags they used there.

It's used in addition to the bash# prompt. Example:
<screen>
<prompt>bash#</prompt> <command>something</command>
</screen>

It works basically the same as <programlisting/>.

> How does that work?  Just add that verbatim, or put something in place 
> of "directory"?

Yes. Example:

Perl is installed by default in the <filename
class="directory">/usr/bin</filename> directory.

Comment 17

[SineSwiper]

> It's used in addition to the bash# prompt. Example:
> <screen>
> <prompt>bash#</prompt> <command>something</command>
> </screen>
> 
> It works basically the same as <programlisting/>.

4.1 has this:

          <computeroutput>
            <prompt>bash#</prompt>

            <command>perl -MCPAN -e 'install "Bundle::Bugzilla"'</command>
          </computeroutput>

So, would computeroutput be alright for that?

> Perl is installed by default in the <filename
> class="directory">/usr/bin</filename> directory.

I take it this only applies to directory paths, as opposed to paths pointing to 
actual files?

Comment 18

[Joel Peshkin]

Of course a user must not be violating terms of service for any machine on which
he/she is a user.

You must be authorized to do this, but you need not have root privileges.  You
also need a fair amount of disk space.

In principle, it is possible to have your own local CPAN modules added to a
system  installation of perl that you do not control. However, I have seen this
break too much to be useful.  Many CPAN installs seem to forget about that
configuration.

If you type httpd  -V, you will get a list of the variables that your system
copy of httpd uses.  One of those 
 -D HTTPD_ROOT="/usr/local/pkg/apache/current"
tells you where that installation looks for its config information.
  cp -r /usr/local/pkg/apache/current /home/me/http
now you have a copy of the conf files to start editing.  When you edit those and
then use the -d option to override the HTTPD_ROOT compiled into the webserver,
you get control.

The same approach applies to mysql

In all these cases, you will need to start the daemons yourself.  You can either
ask your sysadmin to add them to system startup files or else add a crontab
entry that runs a script to check on these daemons and restart them if needed.

Comment 19

[Joel Peshkin]

On the disk space issue,

/home/me/perl - 34MB
/home/me/mysql/data - 2500MB (database has 5500 bugs)
/home/me/http - a few MB

If space for perl is an issue, then there is no space for the data

Comment 20

[SineSwiper]

> In principle, it is possible to have your own local CPAN modules added to a
> system installation of perl that you do not control. However, I have seen this
> break too much to be useful.  Many CPAN installs seem to forget about that
> configuration.

Well, I've got it to work.  That's why I'm writing this doc :)  If you try those
changes in CPAN, it'll work.  The only major issue I had was with Template.

> The same approach applies to mysql

Any specifics?  Is it -V for mysql, as well?  BTW, thanks for the info.  I'll
include this in the docs.

> /home/me/perl - 34MB
> /home/me/mysql/data - 2500MB (database has 5500 bugs)
> /home/me/http - a few MB

Wow...how many attachments are on that thing?  I'm only using 1.16MB, but then
again, I only have 24 bugs.  Even if you divided my MB usage by 24 and then
multiplied it by 5500, you'll only end up with 265.8MB.  Besides, at that point,
you're too complex, and probably using too much bandwidth to be running on a
mere web hosting service.

In any case, this is some useful info, which I'll include in the doc.  Does
anybody else have any other useful hints?

Comment 21

[Joel Peshkin]

Almosty all of it is attachments.
After that, the next biggest is longdescs (the text of comments) at just 13MB.

If you try ....
/var/libexec/mysqld -?
It'll confess its compiled-in values

Create a file .my.cnf in your home directory as follows....

[mysqld]
datadir=/home/joel/mymysql
socket=/home/joel/mymysql/thesock
port=8081

[mysql]
socket=/home/joel/mymysql/thesock
port=8081

[mysql.server]
user=mysql
basedir=/var/lib

[safe_mysqld]
err-log=/home/joel/mymysql/the.log
pid-file=/home/joel/mymysql/the.pid


Then,

mysql_install_db
safe_mysql &
mysql -uroot mysql
show tables;

Comment 22

[Jacob Steenhagen]

> 4.1 has this:

It's wrong :)

I majorly overhauled it on the tip, but didn't change much on 2.16. There should
be other examples where I used <screen/>

> I take it this only applies to directory paths, as opposed to paths pointing
> to actual files?

That's correct. The DocBook documentation has some more classes for <filename/>
but I think that's the only one currently used by the Bugzilla Guide.

Comment 23

[Jacob Steenhagen]

Comment on attachment 127737 [details] [diff] [review]
New non-root section in Installation notes

My previous comments weren't addressed. I'd like to see this get added in time
for the next point release of 2.16, but I don't know if I'm gonna have time to
make the mods myself.

Comment 24

[SineSwiper]

Oh, sorry...I need to get back on this.  Hopefully I'll have time sometime this 
week for the additions.

Comment 25

[Joel Peshkin]

The other key item is perl....
[Disclaimer:  I'm testing this right now -- may tweak later]

wget http://perl.com/CPAN/src/stable.tar.gz
tar xzf stable.tar.gz
cd perl-5.8.1
sh Configure -de -Dprefix=/home/yourownaccount/perl
make
make test
make install

Then, put that perl bin ahead of the /usr/local/bin/ in your $PATH and start
installing CPAN modules

Followed by changing the shebangs in the bugzilla scripts to use your own perl
instead of the system one.

Comment 26

[SineSwiper]

> Small remark:
> > Since you probably can't set up a symbolic link 
> > to /usr/bonsaitools/bin/perl 
> > as  a non-root user, you will need to hack the scripts to point to the 
> > right perl:
> The current developer version (2.17.4) uses now /usr/bin/perl per default.

I'm currently making the mods.  Question is that if I need to just take this 
out, or put a notice about people using pre-2.17 versions of Bugzilla.

Comment 27

[SineSwiper]

Hey Joel: Can you clarify comment #21?  I'm trying to add that part into the 
docs.

Comment 28

[SineSwiper]

Attachment: New non-root section with additions

Here is the new patch.	I could not test it because my OpenJade enivorniment is
currently hosed, so if somebody could plug that in and give me the resulting
HTML (or errors), that would be great.

Comment 29

[SineSwiper]

Comment on attachment 134354 [details] [diff] [review]
New non-root section with additions

Requesting review...

Comment 30

[Joel Peshkin]

Response to comment 27 clarifying comment 21.

Normally, it would be impossible to run mysqld as an ordinary user because it
would want to create a socket that a user may not be able to create, it may want
to create datafiles where a user cannot write, it may use a port someone else is
already using, and it may use PIDfiles where the user may not write.

Fortunately, the my.cnf file can override all of this.   This technique permits
a user to use any compiled mysqld and override all the values that make it
impossible to run as a user.

Comment 31

[Joel Peshkin]

Comment on attachment 134354 [details] [diff] [review]
New non-root section with additions

The xml file needs a little cleanup and there are some improvements that would
be worthwhile, but I think the best approach would be to 
a) Clean it up and add a disclaimer that the instructions are preliminary
b) go ahead and land it 
c) make improvments over time

Comment 32

[Joel Peshkin]

Attachment: Revised patch - joel

OK, here's the cleanup

Comment 33

[Joel Peshkin]

Attachment: And the html version for the sake of convenience

Comment 34

[Vlad Dascalu]

Comment on attachment 143805 [details] [diff] [review]
Revised patch - joel

Great docs! :-)

+      <para>However, it is HIGHLY recommended that the server runs
+      <ulink url="http://httpd.apache.org/docs/suexec.html">suEXEC</ulink>
+      or a similar system for added security. (This ensures that all web
+      scripts are ran as the user who maintains the site, instead of
+      nobody.)</para>

Like I said in IRC, this conflicts with justdave's point of view about it
expressed in bug 105366 comment #24.


+	 seperate database which is already locked down (or one big

"seperate" --> "separate"


+	     <para>After your mysqld program is built an any .my.cnf file is 

"an" --> "and" I guess.


+      you will need to hack the scripts to point to the right perl:</para>

"perl" --> "Perl"


Looks good otherwise to me during a fast read. You could have fixed that upon
checkin but I haven't made a full review (especially regarding the scripts that
are provided as example in order to do path substition and stuff), and since
the suEXEC change is pretty important, I guess another diff would make things
easier to review on the next run.



Nits (that you can ignore):

+	     <para>When you start mysqld the first time, you then connect to

How about:

+	     <para>After you start mysqld for the first time, you should
connect to


+      the machine (ugh!), you will have to build the sources

We can probably skip the "ugh!" here :-P.


+	 <para>You will also want to install the other optional modules:</para>

"will" --> "may" (because they are optional)


> You should have MySQL installed as root.

This could confuse users in thinking that you should have installed MySQL as
root, even if after one page we specify:

"You can install MySQL as a not-root".


+	       <para>Create a file .my.cnf in your 
+	       home directory as follows....</para>

Are our documentation readers smart enough to realise that /home/foo should be
replaced with the actual home directory or should we specify that as well? :)


+	 <para>With this method, the module install will usually go a lot

Is here "install" --> "installation" suitable?


Also, I don't know if it's suitable or not, but removing the bash prompt
accross the examples seems like a good idea. "bash" is a prompt and by
definition it will vary across different distributions. In addition, a newbie
doc reader might actually try to type "bash" by itself in an attempt to
reproduce the commands. Also, if there are several commands, the lack of the
"bash" prompt in front of each one of them makes copy-paste operations easier.
However "$bash" and "cpan>" help make a difference between the different types
of prompts, so this has downsides as well.

Comment 35

[Joel Peshkin]

Attachment: patch v2

Dropped the SuEXEC section and perfomed other suggested edits.

Comment 36

[Vlad Dascalu]

Comment on attachment 144453 [details] [diff] [review]
patch v2

+      <para>If you are running an *NIX OS as non-root, either due

"an" --> "a"

+	 successful browse to the <filename>index.cgi</filename> (like
+	 a Forbidden error), you may have to relax your permissions,

"(like a Forbidden error)" --> "(due to a forbidden error, for example)"

"successful" --> "successfully"

+      <para>Ideally, this also needs to be installed as root and
+      running under a special webserver account. As long as

I think "running" should be "ran" or something like that to be in the same form
with "installed".

Sorry for not catching those the first time. r=vlad with that. (fixing those
upon checkin and doing incremental improvement after the checkin sounds good,
*nods*)

Comment 37

[Dave Miller [:justdave]]

I'm assuming this is applicable to 2.16 as well, it should go in both docs. 
(might need a little reformating for 2.16).

Comment 38

[Vlad Dascalu]

Joel: don't forget comment #36 when you check in (<wink>)

Comment 39

[Joel Peshkin]

Too late :-) But I'll fix it

Comment 40

[Joel Peshkin]

Checked in on both branches

Comment 41

[SineSwiper]

> Like I said in IRC, this conflicts with justdave's point of view about it
> expressed in bug 105366 comment #24.

I respectfully disagree.  My experience with suEXEC is that it corrects
permission errors that would normally go unnoticed, and forces the admin to set
up usernames and user permissions correctly.  The alternative is to have Apache
run as user "nobody", which is pretty insecure.  I see no evidence that suEXEC
is ever more insecure than without it, except with the unthinkable prospect of
editing the source code for suEXEC.

> We can probably skip the "ugh!" here :-P.

:P  Not having Perl is also unthinkable :)

> This could confuse users in thinking that you should have installed MySQL as
> root, even if after one page we specify:
> "You can install MySQL as a not-root".

Difference between what you SHOULD have, and what you CAN have.  You can have
MySQL installed as non-root, but then you run into issues with running a server
on a box that you shouldn't (ie: against sysadmin policy), etc.

> Are our documentation readers smart enough to realise that /home/foo should be
> replaced with the actual home directory or should we specify that as well? :)

You never know :)  A safer answer would be that our documentation readers may
realize the reference, but may not catch it as they are cutting and pasting a
command in.  Thus, a reference might help (or if it is already in there, keep it
in there).

> Also, I don't know if it's suitable or not, but removing the bash prompt
> accross the examples seems like a good idea. "bash" is a prompt and by
> definition it will vary across different distributions. In addition, a newbie
> doc reader might actually try to type "bash" by itself in an attempt to
> reproduce the commands. Also, if there are several commands, the lack of the
> "bash" prompt in front of each one of them makes copy-paste operations easier.
> However "$bash" and "cpan>" help make a difference between the different types
> of prompts, so this has downsides as well.

Especially because of the latter comment, I would give our readers a benefit of
a doubt (since this is sort of an advanced thing to do; installing as non-root,
I mean).  One thing I noticed, at least in the HTML copy I last saw, the prompts
were spaces with a newline and then the command, like:

bash$
command /home/foo/blah

Dunno if that was the rendering with his version of Jade, or if that was the
actual effect...

> +      <para>If you are running an *NIX OS as non-root, either due
> "an" --> "a"

Depends if the reader is going to invisibly put in the U in UNIX or not.  (I
do.)  Even then, I have a pet peeve about using "an" in a place that is
technically correct English, but destroys the point of using "an" by not making
it flow correctly.  ("An UNIX" sounds awkward, but "A UNIX" doesn't.  Go
figure...)  In this respect, I agree with your change.

BTW, thanks for putting this in.  Though I no longer need the use of this set of
documentation (have my own VPS), I'd be glad to welcome/write any additions.