961755 - Make API docs build again on website

Status: RESOLVED FIXED

(Bugzilla :: Documentation, defect)

Description

[Gervase Markham [:gerv]]

Since we moved to RST, the website's build-docs.pl has been failing (I assume) and the docs links are 404. I fixed most of them to point to ReadTheDocs in bug 960967, but RTD doesn't build the API docs.

To get them working again in the short term, we need to:

1) Make the new docs/makedocs.pl have an "API-only" switch
2) Update the website's build-docs.pl to use it when it detects an "docs/en/rst" directory.

Gerv

Comment 1

[Gervase Markham [:gerv]]

Attachment: Patch v.1

Includes one-line patch for bug 960968, to avoid a tedious need to un-bitrot it.

Gerv

Comment 2

[Gervase Markham [:gerv]]

Attachment: Patch v.2

This is better.

Gerv

Comment 3

[Gervase Markham [:gerv]]

The attached patch deals with point 1 in comment 0. As the website pulls stuff from source control to build it, we need to get that checked in before it's at all easy to develop a patch for point 2.

Gerv

Comment 4

[Teemu Mannermaa (:wicked)]

Comment on attachment 8362561 [details] [diff] [review]
Patch v.2

Looks good to me, although asking --pod-only and not having necessary Perl module makes the building "silently fail" or just not do anything but not sure if it's that big of a deal in the end.

And the other one liner fixes our documentation+pdf Tinderbox build that has done the same as normal documentation build lately. :)

Comment 5

[Gervase Markham [:gerv]]

Committing  to: bzr+ssh://bzr.mozilla.org/bugzilla/trunk/                      
modified docs/makedocs.pl
Committed revision 8874. 

Gerv

Comment 6

[Gervase Markham [:gerv]]

Oops. Need to do part 2.

Gerv

Comment 7

[Gervase Markham [:gerv]]

Attachment: Patch B, v.1

And this should update the website build process to use the new switch.

Gerv

Comment 8

[Teemu Mannermaa (:wicked)]

Comment on attachment 8362577 [details] [diff] [review]
Patch B, v.1

You don't seem to handle RST build for the CVS checkout so in effect you are not running it for trunk because according to default configuration, only 4.2 and 4.4 branches are built from bzr. Also, I can't see any code to rsync the resulting html/api output to be visible on the site. You probably need to share the code at the end of the current build_docs() sub with these subs or do the rst check inside the build_docs itself and "do the right thing" in there if found.

Comment 9

[Gervase Markham [:gerv]]

(In reply to Teemu Mannermaa (:wicked) from comment #8)
> Comment on attachment 8362577 [details] [diff] [review]
> 961755b.diff
> 
> You don't seem to handle RST build for the CVS checkout so in effect you are
> not running it for trunk because according to default configuration, only
> 4.2 and 4.4 branches are built from bzr.

Good point. I also had a local change to build docs for trunk from BZR instead of CVS. That needs to be part of the patch.

> Also, I can't see any code to rsync
> the resulting html/api output to be visible on the site.

Ah, right. That's what that code does! It's a mass of twisty variable names, all slightly different.

Gerv

Comment 10

[Gervase Markham [:gerv]]

Attachment: Patch B, v.2

OK, try this.

Gerv

Comment 11

[Teemu Mannermaa (:wicked)]

Comment on attachment 8362594 [details] [diff] [review]
Patch B, v.2

This is much better and I like moving to bzr for trunk. Although, the build system probably need manual intervention to force the build or remove the trunk CVS checkout so that the script doesn't fail. Not sure if there's any code that should be added to handle that case (or if we should just ask justdave or somebody from Mozilla IT to do manual things).

This change does break building the non-en languages but it seems that we haven't build any other languages for a long time anyway. Also this script is pretty much specific to our website.

You are not copying the needed style.css to the dest and that's needed for the POD docs to look pretty, AFAIK. You probably should add the three lines doing the copy.

It also fails creating new destination directories for a new version because mkdir can't handle missing intermediate directories, AFAIK. Simple fix would be to move the en to the rsync command line itself. This also helps building the $dest_dir for the style.css copy command.

Comment 12

[Gervase Markham [:gerv]]

(In reply to Teemu Mannermaa (:wicked) from comment #11) 
> This is much better and I like moving to bzr for trunk. Although, the build
> system probably need manual intervention to force the build or remove the
> trunk CVS checkout so that the script doesn't fail. Not sure if there's any
> code that should be added to handle that case (or if we should just ask
> justdave or somebody from Mozilla IT to do manual things).

It might need manual intervention. justdave: who can do that?

While you are on there, please install File::Which and File::Path, which are needed for LpSolit's recent patch to allow makedocs.pl to use rst2pdf in bug 961398.

Gerv

Comment 13

[Gervase Markham [:gerv]]

Attachment: Patch B, v.3

Comment 14

[Gervase Markham [:gerv]]

justdave: I need to hear from you about removing the old CVS tip checkout on the server before I check this in. Once you tell me it's removed (or tell me who to ask to get it done) then I can check this in and the API docs will build again, and the link on the website will work.

Don't forget to install File::Which and File::Path :-)

Gerv

Comment 15

[Dave Miller [:justdave]]

On RHEL6, File::Path is included in core Perl.  File::Which is perl-File-Which and is available from both RHEL-Optional and rpmforge-extras.

Bug 962726 filed to get webops to set this up.

Comment 16

[Gervase Markham [:gerv]]

Attachment: Patch B, v.4

Webops has installed File::Which but not (AFAICS) removed the unwanted tip CVS checkout. Turns out the fix to do so is a one-line change, so I made it. See line 43 of build-docs.pl (line 22 of the new patch). The rest is identical to the reviewed version.

Let's get these API docs back on the website :-)

Gerv

Comment 17

[Frédéric Buclin]

Comment on attachment 8365324 [details] [diff] [review]
Patch B, v.4

I know nothing about this script, so I will let wicked review it.

Comment 18

[Gervase Markham [:gerv]]

Checking in bin/build-docs.pl;
/www/bugzilla-org/bin/build-docs.pl,v  <--  build-docs.pl
new revision: 1.13; previous revision: 1.12
done
Checking in etc/docs-versions;
/www/bugzilla-org/etc/docs-versions,v  <--  docs-versions
new revision: 1.10; previous revision: 1.9
done

So now we wait to see if it uploads to the website, if it works, and if the API docs appear again...

Gerv

Comment 19

[Gervase Markham [:gerv]]

http://www.bugzilla.org/docs/tip/en/html/api/ is still 404 :-( Does anyone know how often the website updates from CVS, and how often the cron job builds the docs? justdave?

Gerv

Comment 20

[Gervase Markham [:gerv]]

URL is now working again! \o/

Gerv

Comment 21

[Mark Côté [:mcote]]

Ha, oh well, so much for the patch I already had for this. :)

Comment 22

[Gervase Markham [:gerv]]

mcote: Did you have a patch to convert the API docs to reST in some way? That might still be of interest...

Gerv

Comment 23

[Mark Côté [:mcote]]

I do! Yeah I wasn't clear as to whether that was solved or not.  Will work on adapting it to all the recent changes in the near future.  I'll file a separate bug for it.