Verify docs.services.mozilla.com is updating on Github push

RESOLVED FIXED

Status

P4
normal
RESOLVED FIXED
7 years ago
5 years ago

People

(Reporter: gps, Unassigned)

Tracking

Details

(Whiteboard: [qa-])

(Reporter)

Description

7 years ago
There were reports in IRC that the automatic hourly update/refresh of docs.services.mozilla.com on Mercurial push of services-docs isn't working.

Tarek said to file a bug so he can verify things when it isn't Friday night in France.
(Reporter)

Comment 1

7 years ago
:atoll suggested in IRC that the docs updating might be a good Jenkins job. Something to consider...
it's up and running - closing
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
looks like we have the issue again
Assignee: tarek → ckolos
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Updated

5 years ago
Summary: Verify docs.services.mozilla.com is updating on mercurial push → Verify docs.services.mozilla.com is updating on Github push
This fixed itself.
Status: REOPENED → RESOLVED
Last Resolved: 7 years ago5 years ago
Resolution: --- → FIXED
So I'm not entirely clear on what's supposed to be happening here, but I pushed something to docs repo on github yesterday, and it hasn't appeared on the live website today.  Do we expect it to appear automatically?
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
afaik, docs doesn't pull from github. webops, keepers of this functionality, use svn for docs.s.m.c.
OK, so we manually push from github to svn and then that goes out to the website automatically from there?  The title of this bug suggested that it should work automatically all the way through from github.
roping in webops on this. I honestly have no idea. :phrawzty can you assist?
Flags: needinfo?(dmaher)
Hello all,

First off, it may be helpful to provide some additional information about the environment in which we find ourselves.  I apologise if this digression is already known to you.

Generally speaking, we use internal source control from the admin nodes (which manage the webheads for a given cluster) to manage content.  In the past we used SVN for this purpose, though one some (newer) clusters, we use Git.  In either case, these repos are unrelated to Mozilla's general source control systems - it's really just for managing the content that gets assembled on the admin nodes and then pushed out to the wedheads.  In the case of the Static cluster[1] (which hosts docs.services.m.c, among others) we are using SVN.

In the specific case of docs.services.m.c, and unlike most of the other sites on the Static cluster, it would appear that the content on the admin node is managed from svn.mozilla.org[3] directly.  Looking at the contents of that repo, we can see that it contains a "compiled" site, which is to say HTML files instead of .rst sources.  Something, therefore, is acquiring the contents of the Github repo, processing it, and pushing it into svn.m.o.  I am currently engaged in tracking this mechanism down, and will update this bug once I have a clearer understanding of all of the moving parts.

[1] https://mana.mozilla.org/wiki/display/websites/Static+Cluster
[2] https://github.com/mozilla-services/docs
[3] http://svn.mozilla.org/projects/docs.services.mozilla.com/
(In reply to Daniel Maher [:phrawzty] from comment #9)
> Something, therefore, is acquiring the
> contents of the Github repo, processing it, and pushing it into svn.m.o.  I
> am currently engaged in tracking this mechanism down, and will update this
> bug once I have a clearer understanding of all of the moving parts.

Historically, that thing has been me. :P
Aaand I messed up the enumerated references in comment 9.  ¯\_(ツ)_/¯
> Something, therefore, is acquiring the contents of the Github repo, processing it, and pushing it into svn.m.o.

I've done a couple of these pushes by hand in the past.  There's an update.sh script in the docs.services.mozilla.com svn that can be run to achieve this push, but I've no idea if or where or when it's run automatically.
Well, that would explain why it "stops working" sometimes - if the process is manual, then there's no guarantee that it will be triggered.  As far as I can tell neither update.sh nor update.py are run automatically from anywhere.

This would appear to be a good candidate for automation[1].  I propose that we simply add a cron job that pulls from the Github repo, builds the content, and then pushes it to the webheads.  Committing it to the Mozilla SVN is optional; however, if there are reasons for retaining this step (external tooling, perhaps?) then it can be retained as well.

Objections ?  General commentary ?

[1] http://imgur.com/KVun06L
Assignee: ckolos → dmaher
Status: REOPENED → ASSIGNED
Component: Web Site → WebOps: Other
Flags: needinfo?(dmaher) → needinfo?
Priority: -- → P4
Product: Mozilla Services → Infrastructure & Operations
QA Contact: nmaul
Version: unspecified → other
+1 from me.
Flags: needinfo?
(In reply to Daniel Maher [:phrawzty] from comment #13)

> Objections ?  General commentary ?

+1 from me. I don't know of any reason why the SVN stage needs to continue to exist, but I acknowledge my limited frame of reference :P
+1 automation ftw
As might be expected, the required packages for Sphinx support do not exist in any of the standard repositories.  I built packages for each dependency, and wouldn't you know, the chain requires for python-imaging and python-pillow, which contain conflicts (they both supply the PIL).

Next up: virtualenv.  On the plus side this means I can avoid distribution packaging concerns (much to the chagrin of OpSec); however, it comes with its own set of special problems - namely python 2.6, and the fact that subprocess.check_output isn't a thing, which means the supplied update.py will fail.

So instead there's Yet Another Bash Script (build_site.sh) which will clone the repo, install the necessary virtualenv components, and then build the site.  I've modified the cron job to reflect this change in the work flow; said cron job runs every 15 minutes.  Note that the Mozilla SVN repo is not part of this work flow.  Should this functionality be required, it can be added in to build_site.sh as necessary.
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago5 years ago
Resolution: --- → FIXED
Thanks for making this happen.

Yeah, having a cron automatically running was the goal since day one. And at some point I ended up having a cron on my own server and it stopped working one day :)

For the record Pillow is a fork of Imaging. So if the first is installed you should be looking good
(In reply to Tarek Ziadé (:tarek) from comment #18)
> For the record Pillow is a fork of Imaging. So if the first is installed you
> should be looking good

Yes, it is - but RPM doesn't know that. :P  LOL dependency hell.
If the RPM package for python-pillow does not specify "Provides" and "Obsoletes" directives, that's a packaging bug
So I guess we need a longer term plan for when this pipeline breaks...

from IRC on 4/7/2014:
Whiteboard: [qa-]
14:15 < alexis> ckolos: hey, do you know if the cron that updates docs.s.m.c is working okay?
14:17 < alexis> bobm: ^ ?
14:19 < ckolos> alexis: unknown.
14:20 < bobm> same here
14:20 < alexis> who should I ask about that?
14:21 < ckolos> https://bugzilla.mozilla.org/show_bug.cgi?id=964959
14:21 < ckolos> that's handled by the web ops team
14:22 < alexis> ah ok
14:22 < alexis> normally I'm happy to see that stuff is handled by phrawsty, but I'm not in this tz today.
14:22 < jbonacci> looking
14:23 < ckolos> there was another bug opened about it but my bz-fu is not working well
14:27 < ckolos> alexis: https://bugzilla.mozilla.org/show_bug.cgi?id=694616
14:27 < jbonacci> ckolos thanks
14:27 < tarek> yup
14:28 < alexis> ckolos: thx
14:29 < jbonacci> I touched both bugs to get them back into my queue in case this conversation comes up again
14:30 < tarek> I am still not sure what's the current status though
14:30 < tarek> e.g. is it syncing or not
14:31 < jbonacci> tarek yea I know
14:31 < jbonacci> we have two bugs but not process
14:31 < jbonacci> :-P
14:35 < tarek> OH: This docs automatic updates are talked about since 2011 :)
14:38 < jbonacci> zactly
I fixed this yesterday (in another bug that :alexis has visibility on).

In the interest of full disclosure, I no longer work in webops, so I'm removing myself from this bug.
Assignee: dmaher → server-ops-webops
You need to log in before you can comment on or make changes to this bug.