1067416 - Reorganize and update Bugzilla docs

Status: RESOLVED FIXED

(Bugzilla :: Documentation, defect)

Description

[Gervase Markham [:gerv]]

As discussed and approved as a plan in the forum and in Bugzilla meetings, I am reorganizing the Bugzilla docs. This bug tracks that work, and feedback/reviews of it.

The code is here:
https://github.com/gerv/bugzilla-docs
(docs-relicensing branch) and the built version of the docs is here:
http://bugzilla-new-docs.readthedocs.org/

This has been a lot of work and I don't want it to be bitrotting for months. So I clearly want to establish my goals here:

Commit Targets

* Must not be factually incorrect or actively misleading
* Must not be worse than existing documentation

Commit Non-Targets

* Must have complete consistency of styling
* Must be comprehensive

There is, in fact, quite a lot of new documentation. But the only _requirement_ is that the new docs be overall better than the old ones. We are reordering here to make a much better basis for enhancement later (e.g. by triaging and fixing all the old docs bugs) and so we know where the gaps are. _Filling in_ all those gaps is a non-goal for this phase.

Gerv

Comment 1

[Gervase Markham [:gerv]]

Attachment: Placeholder for feedback round 1

OK so, courtesy of a 9-hour plane ride to Beijing, here it is :-) All sections of the docs are ready for feedback except section 2, Installing, which still needs quite a bit of work for Windows and Mac OS X. (If anyone's around who's actually installed Bugzilla on either of those OSes, I could do with some help!) 

I suspect that the best way to review them would be to just read them from ReadTheDocs, but if you want to read them as raw rST from the Github repo, that might be OK - although you'll find it harder to grasp the structure, and there's some fun stuff with includes in the upgrading docs. Just don't try reviewing a diff - that won't work.

Here's what I'm looking for in this feedback request:

* Lots of places are marked "XXX" with a question. If you know the answer to the question, put it in your feedback.

* I circulated the organizational structure a few weeks ago, to no objection:
  https://wiki.mozilla.org/Bugzilla:Documentation_Plan
  but now's another chance to comment on it.

* Do you think we are on the right track here? What, for you, are the blockers to committing this branch to the trunk?

* Some stuff has been left out from the old docs as irrelevant for today. If I've made a wrong call here, tell me.

* If you see bits of documentation that need to be written, feel free to note them, but you'd need to make a good case for them being checkin blockers.

Thanks for your time!

Gerv

Comment 2

[:glob ✱]

Comment on attachment 8489407 [details]
Placeholder for feedback round 1

i've skimmed the docs, looking for XXX questions that i can quickly answer ...


> XXX Given this is a quick setup on a dedicated box, is it OK to use the MySQL root user?

no, we shouldn't be recommending insecure practices.

> XXX Why can't checksetup determine webservergroup automatically?

there's no standard cross-platform/distro/server way to do it.

> yum install .. perl-DateTime perl-Template-Toolkit perl-Email-Send perl

yum also supports: yum install 'perl(Email::Send)'

this may be preferred as it guards us against changing of perl package names between releases (we've been hit by this before).

> You can run testserver.pl http://bugzilla-url/ from the command line to check if your web server
> is correctly configured.
> XXX Does this work before doing any localconfig stuff?

good catch, it doesn't.  also the command should be |./testserver.pl http://bugzilla-url/|

> To configure your Apache web server to work with Bugzilla while using mod_cgi, do the following:
> XXX Shouldn't we be using sites-available/sites-enabled here?

no; that's a debian/ubuntu specific dance.  each distro has their own method for splitting httpd's conf file.

> We recommend that you allow at least 16MB packets by adding the max_allowed_packet parameter to
> your MySQL configuration in the [mysqld] section, like this:
> XXX is "adding" right? Surely it's normally present?

that's also distro-dependant.  eg. it isn't present on fedora 

> Once you have SQLite installed, no additional configuration is required to run Bugzilla.
> Simply set $db_driver to Sqlite (case-sensitive) in localconfig, when you get to that point in
> the installation.
> XXX This doesn't work - gives a timezone-related error on my box.

works for me (fedora 20).  a timezone error doesn't sound like a sqlite issue.

> 4.4 XXX BMO's merge accounts script?

that's an upstream contrib script, not BMO's.
the entire 4.4 section is weird -- it has odd phrasing and no answers.

> 7.1
> XXX So this requires wildcard DNS? We should explain a bit about what is needed here.

yes

> XXX Does the attachment itself still exist, it's just empty?

the attachment metadata still exists; only the attachment's contents is deleted.

> XXX Do we need to document it, then? Or even still support it? (shadow db)

yes, bugzilla uses it heavily, but not for the documented reason.

a standard large database setup involves a single db master and a pool of read-only db slaves.  queries which are not updating data can be directed to the slave pool, removing the load/locking from the master, freeing it up to handle writes.  bugzilla will switch to the shadow-db (ie. read-only db pool) when it knows it doesn't need to update the database (eg. searching, anonymous show_bug).

> 7.4
> XXX is this the "default version" in any sense? (product -> version)

yes.

Comment 3

[Gervase Markham [:gerv]]

Hi glob,

Thanks for the review :-) Other than the below, I've fixed the issues you raised and repushed.

(In reply to Byron Jones ‹:glob› from comment #2)
> > XXX Given this is a quick setup on a dedicated box, is it OK to use the MySQL root user?
> 
> no, we shouldn't be recommending insecure practices.

If we assume that Bugzilla is the only user of the database (this is the Quick Start, after all), why is it more insecure to have Bugzilla use the MySQL root user than to have Bugzilla use a user with all the privs we normally assign to the bugs user (which seems like a lot to me)?

> > XXX Why can't checksetup determine webservergroup automatically?
> 
> there's no standard cross-platform/distro/server way to do it.

Well, we could detect common distros and set it for those.

> > yum install .. perl-DateTime perl-Template-Toolkit perl-Email-Send perl
> 
> yum also supports: yum install 'perl(Email::Send)'
> 
> this may be preferred as it guards us against changing of perl package names
> between releases (we've been hit by this before).

OK - you want me to convert all the Perl package names to that form?

> > You can run testserver.pl http://bugzilla-url/ from the command line to check if your web server
> > is correctly configured.
> > XXX Does this work before doing any localconfig stuff?
> 
> good catch, it doesn't.  also the command should be |./testserver.pl
> http://bugzilla-url/|

I can't work out the best order for the configuration steps. I don't want to get into a situation where you have to configure thing X (e.g. the webserver) a bit, go away and do other stuff and then come back and configure it some more. But there seem to be a lot of dependencies (e.g. configuring Oracle involves webserver configuration).

> > To configure your Apache web server to work with Bugzilla while using mod_cgi, do the following:
> > XXX Shouldn't we be using sites-available/sites-enabled here?
> 
> no; that's a debian/ubuntu specific dance.  each distro has their own method
> for splitting httpd's conf file.

Well, shouldn't we recommend it for Debian/Ubuntu?

Does Red Hat require you to add all configuration to the distro-supplied httpd conf file, then?

> > We recommend that you allow at least 16MB packets by adding the max_allowed_packet parameter to
> > your MySQL configuration in the [mysqld] section, like this:
> > XXX is "adding" right? Surely it's normally present?
> 
> that's also distro-dependant.  eg. it isn't present on fedora 

OK. Seems the MySQL default is 1MB so we still need to ask people to do this.

> > Once you have SQLite installed, no additional configuration is required to run Bugzilla.
> > Simply set $db_driver to Sqlite (case-sensitive) in localconfig, when you get to that point in
> > the installation.
> > XXX This doesn't work - gives a timezone-related error on my box.
> 
> works for me (fedora 20).  a timezone error doesn't sound like a sqlite
> issue.

Well, it's a bit offtopic, but it is related to the Perl sqlite driver:

Precompiling templates...done.
Fixing file permissions...
DBD::SQLite::st execute failed: Cannot determine local time zone
 [for Statement "INSERT INTO audit_log (user_id, class, object_id, field,
                                removed, added, at_time) 
              VALUES (?,?,?,?,?,?,LOCALTIMESTAMP(0))"] at Bugzilla/Object.pm line 594.
	Bugzilla::Object::audit_log('Bugzilla::Field=HASH(0xc98adc8)', '__create__') called at Bugzilla/Object.pm line 732
	Bugzilla::Object::insert_create_data('Bugzilla::Field', 'HASH(0xbb1f8bc)') called at Bugzilla/Field.pm line 1062
	Bugzilla::Field::create('Bugzilla::Field', 'HASH(0xbb5360c)') called at Bugzilla/Field.pm line 1196
	Bugzilla::Field::populate_field_definitions() called at ./checksetup.pl line 165

> > 7.1
> > XXX So this requires wildcard DNS? We should explain a bit about what is needed here.
> 
> yes

Do we just say "This requires you to set up wildcard DNS", or can we be more helpful?

Gerv

Comment 4

[Frédéric Buclin]

(In reply to Gervase Markham [:gerv] from comment #3)
> > > XXX Given this is a quick setup on a dedicated box, is it OK to use the MySQL root user?
> > 
> > no, we shouldn't be recommending insecure practices.
> 
> If we assume that Bugzilla is the only user of the database (this is the
> Quick Start, after all), why is it more insecure to have Bugzilla use the
> MySQL root user than to have Bugzilla use a user with all the privs we
> normally assign to the bugs user (which seems like a lot to me)?

I agree with glob that we should NOT use the root MySQL account. We give privs to one given DB only, not to the other ones. Bugzilla being the only user or not is not a reason to give bad practice. So definitely use a separate account.

Comment 5

[Gervase Markham [:gerv]]

(In reply to Frédéric Buclin from comment #4) 
> I agree with glob that we should NOT use the root MySQL account. We give
> privs to one given DB only, not to the other ones. 

But what if there _are_ no other ones? Is there still a greater risk?

This is a Quick Start guide where someone is installing a box from scratch. There are no other DBs. Can you explain why using the root account is still a greater security risk than using a dedicated account?

Gerv

Comment 6

[Gervase Markham [:gerv]]

LpSolit: Oh, and if you have time to give feedback on the docs, that would be awesome :-) But I don't want to burden you.

Gerv

Comment 7

[Frédéric Buclin]

(In reply to Gervase Markham [:gerv] from comment #5)
> But what if there _are_ no other ones? Is there still a greater risk?

I'm against this idea in all cases, because my reasoning is: next step would be to not use a password at all because you are the only one accessing your machine, etc...


A quickstart guide must remain something which everybody can refer to to configure their Bugzilla installation, including in more realistic conditions where Bugzilla is not the single application interacting with MySQL/Pg/Oracle/Sqlite. We would shoot ourselves in the foot if we had to write a warning such as "Only follow these steps if you plan to use Bugzilla on your own machine for testing purposes only. It's totally not recommended to use the root account with production machines. In that case, you must find the documentation elsewhere. Good luck, and try to not open some security hole somewhere!".

I have installed and configured tens of Bugzilla installations, most for testing purposes on my local machine only, but also some in real environment. In that case, I want to look at the same documentation as usual to get the info I need without having to think "Damned, I must not do this here. Bad habit!".

Comment 8

[Gervase Markham [:gerv]]

I would still like you to explain what the additional risk is of using the root account on a production machine, as long as it's a dedicated machine.

I have no problem saying in the instructions "if Bugzilla is not the only application on this machine, don't use the Quick Start". (There are various other things about interacting with other apps we'd need to explain anyway. The Quick Start uses the web server document root - that probably wouldn't be good if there were other apps.)

Gerv

Comment 9

[:glob ✱]

(In reply to Gervase Markham [:gerv] from comment #8)
> I would still like you to explain what the additional risk is of using the
> root account on a production machine, as long as it's a dedicated machine.

there's no guarantee that a dedicated machine will remain so forever.  suggesting root simply isn't a good idea.

Comment 10

[:glob ✱]

(In reply to Gervase Markham [:gerv] from comment #3)
> > > yum install .. perl-DateTime perl-Template-Toolkit perl-Email-Send perl
> > 
> > yum also supports: yum install 'perl(Email::Send)'
> > 
> > this may be preferred as it guards us against changing of perl package names
> > between releases (we've been hit by this before).
> 
> OK - you want me to convert all the Perl package names to that form?

yes please

> > > To configure your Apache web server to work with Bugzilla while using mod_cgi, do the following:
> > > XXX Shouldn't we be using sites-available/sites-enabled here?
> > 
> > no; that's a debian/ubuntu specific dance.  each distro has their own method
> > for splitting httpd's conf file.
> 
> Well, shouldn't we recommend it for Debian/Ubuntu?

if we want to provide distro-specific instructions, then we should also provide those for centos/redhat/fedora systems.

> Does Red Hat require you to add all configuration to the distro-supplied
> httpd conf file, then?

it doesn't have a dedicated directory for apache vhosts, you either edit httpd.conf, or create a file in the /etc/httpd/conf.d directory.

> DBD::SQLite::st execute failed: Cannot determine local time zone

that's bug 604942 (read comment 9)

> > > XXX So this requires wildcard DNS? We should explain a bit about what is needed here.
> > 
> > yes
> 
> Do we just say "This requires you to set up wildcard DNS", or can we be more
> helpful?

that's probably good enough.  it's an easy term to search for should someone need more information.

Comment 11

[Gervase Markham [:gerv]]

(In reply to Byron Jones ‹:glob› from comment #10)
> > OK - you want me to convert all the Perl package names to that form?
> 
> yes please

I've done it, but you'll need to check it.

> > Well, shouldn't we recommend it for Debian/Ubuntu?
> 
> if we want to provide distro-specific instructions, then we should also
> provide those for centos/redhat/fedora systems.

I refer the honourable gentlemen about my "non-comprehensiveness" criteria :-) But feel free to supply additional text. In most places where it differs, we have entries for Fedora/Red Hat and Debian/Ubuntu, but I would take more, or e.g. a SuSE line if someone wanted to provide one. In one place, we have Gentoo instructions.

> > DBD::SQLite::st execute failed: Cannot determine local time zone
> 
> that's bug 604942 (read comment 9)

Well, the claim is this issue is fixed in DateTime::TimeZone 1.64, and I have 1.71..

Gerv

Comment 12

[:glob ✱]

(In reply to Gervase Markham [:gerv] from comment #11)
> I refer the honourable gentlemen about my "non-comprehensiveness" criteria
> :-)

ah; my point is very much in line with that; sorry i wasn't clear.
allow me to re-phrase my original feedback:

> To configure your Apache web server to work with Bugzilla while using mod_cgi, do the following:
> XXX Shouldn't we be using sites-available/sites-enabled here?

no - sites-available/sites-enabled is a debian/ubuntu specific dance.  we should either not mention it at all, or add a note along the lines of "Linux distributions differ with regards to recommended practices for configuring apache virtual-hosts (eg. Debian/Ubuntu uses sites-available).  Please refer to your distribution's documentation for more information."

Comment 13

[Gervase Markham [:gerv]]

I am happy for non-comprehensiveness to mean "gives details for some distros but not others". I don't think we should have an evenness criteria; the detailed information is useful for Ubuntu users even if we don't have similarly detailed info for Red Hat users yet.

Gerv

Comment 14

[Gervase Markham [:gerv]]

Comment on attachment 8489407 [details]
Placeholder for feedback round 1

We're now into the review process rather than feedback.

Gerv

Comment 15

[Gervase Markham [:gerv]]

The (poorly named) docs-relicensing branch at https://github.com/gerv/bugzilla-docs has now had all the reviews listed at https://etherpad.mozilla.org/bugzilla-docs-review, which cover the vast majority of the text. After over two months of seeking reviews, I think it's time to check in and continue work from there. Requesting approval.

(I've merged the trunk version of dkl's new API docs into the branch. A little tweaking will be required for the 5.0 version because his docs are slightly different there. But I can handle that.)

Gerv

Comment 16

[Gervase Markham [:gerv]]

To ssh://[e-mail]/bugzilla/bugzilla.git
   c0dbbc6..47342f0  master -> master

Gerv

Comment 17

[Gervase Markham [:gerv]]

To ssh://[e-mail]/bugzilla/bugzilla.git
   8ebd893..06033dc  5.0 -> 5.0

On the 5.0 branch, I turned off "TODOs" to make the docs look releaseable.

Thanks to everyone who helped with reviews!

Gerv

Comment 18

[:glob ✱]

(In reply to Gervase Markham [:gerv] from comment #16)
> To ssh://[e-mail]/bugzilla/bugzilla.git
>    c0dbbc6..47342f0  master -> master

next time please commit as a single revision on master rather than merging in a development branch.  merging brings in all your WIP commits.

Comment 19

[Gervase Markham [:gerv]]

Sorry :-( I didn't realise that this would have this effect; I assumed all the intermediate commits would be on an irrelevant "loop" in the DAG somewhere and not show up.

Gerv