Port /about/forums pages from SVN to bedrock and/or other locations

RESOLVED FIXED

Status

P2
normal
RESOLVED FIXED
6 years ago
3 years ago

People

(Reporter: sgarrity, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(URL)

Attachments

(1 attachment)

(Reporter)

Description

6 years ago
The only files left in the /about dir on the old mozilla.org SVN repo are in the /about/forums dir.

We need to determine which files are still relevant, and where they should be ported.

The files are:

http://www.mozilla.org/about/forums/index.html
http://www.mozilla.org/about/forums/etiquette.html
http://www.mozilla.org/about/forums/cancellation.html

There are also three Perl scripts that live the /about/forums dir (checklist, commit, and gen-ng-info). These seem to be scripts that grab the list of forums from news.mozilla.org, dump them to a text file, and generates the list for the /about/forums page.

The etiquette.html and cancellation.html pages seem like they could live under http://mozilla.org/about/governance/policies/ but it might also make sense for them to stay with the overview page (index.html) where ever it ends up.

So, Forums page (http://www.mozilla.org/about/forums/). Should it be ported to Bedrock as is? Does it belong elsewhere?
The way this works is that groups are added to raw-ng-list.txt, with a classification (indicated by position), a name and a description. "gen-ng-info" turns that raw list into some HTML with appropriate subscription links etc. "commit" runs gen-ng-info, uses a script called "replace-between" to stick the generated HTML into index.html, and then checks all changed files in to the SVN repo.

"checklist" grabs the list of newsgroups from news.mozilla.org to check and see if the raw-ng-list.txt file has got out of date compared with reality. So when someone asks me to update the list, I run checklist and fix all the problems it finds.

I'm certainly open to other ways of doing this which do not require more effort. 

Haven't we had this discussion before by email? Jennifer?

Gerv
Hi Gerv and Steven-

Yes - we've talked about this previously.

Justin is working on a publishing solution that will allow us to publish the about/forums content on Bedrock.

Thx,
Jen
Depends on: 874137
Moving this one to the Icebox on Kanban until the legacy publishing system is ready to go
Priority: -- → P2
Hi Gerv-

We'd like to propose the a new work flow to publish the forums index, etiquette, and cancellation pages directly to Bedrock and remove it from the SVN repo:

1.  Content owner will file a bug with information about the requested updates to the page
2.  Content owner or Bedrock developer will make the update in Github and file a pull request
3.  Another Bedrock developer will review the PR
4.  Updates will be published on Bedrock

Will this work for these pages, Gerv?

Thx,
Jen
Flags: needinfo?(gerv)
Hi Jennifer,

I was under the impression that you were developing a self-publishing system for Bedrock, and that this is what we've been waiting for to take action on this bug. See comment 2. Does that system now exist, or not?

Having to file bugs and make Github PRs which someone else has to approve is really not an ideal workflow for me, particularly if something better is actually available... The etiquette and cancellation pages rarely change, so it's not so much of an issue, but the forums index gets updated regularly.

Gerv
Flags: needinfo?(gerv)
Hi Gerv-

We did develop a self-publishing system to publish Firefox release notes, but it isn't a turn-key solution and would require not-insignificant back-end development hours to be applied to the Forums and Credits workflow (I'll comment separately in the Credits bug).

In other words, nothing better is readily available.  We do publish almost daily on www.mozilla.org, so we can guarantee a pretty quick turn around.  And we  could incorporate the Forums script into the Git process if that would help you.

I think we need to move forward in this direction.

Thx,
Jen
Flags: needinfo?(gerv)
(In reply to Jennifer Bertsch [:jbertsch] from comment #6)
> Hi Gerv-
> 
> We did develop a self-publishing system to publish Firefox release notes,
> but it isn't a turn-key solution and would require not-insignificant
> back-end development hours to be applied to the Forums and Credits workflow
> (I'll comment separately in the Credits bug).
> 
> In other words, nothing better is readily available.  We do publish almost
> daily on www.mozilla.org, so we can guarantee a pretty quick turn around. 
> And we  could incorporate the Forums script into the Git process if that
> would help you.
> 
> I think we need to move forward in this direction.
> 
> Thx,
> Jen

What if Gerv just runs his normal perl scripts and the output of the script is what is submitted via Github's pull request? This will require no extra magic than we have now and using the same workflow [1] that everyone in the world uses for contributing to mozilla.org. If we have an existing workflow for content updates and contributions on bedrock that is good enough for any contributor (paid or volunteer), why would we have anything more complex than that for the forums page?

The self-publishing system [2] requires substantial technical investment to set it up and only make sense for updates that are frequent and with many changes each publish. The ROI of setting up Nucleus for the forum updates will be negative for quite a while because the updates appear to only happen every month or two. Firefox release notes uses Nucleus because updates happen every 6 weeks and 100% of the content changes each time.

[1] https://bedrock.readthedocs.org/en/latest/contribute.html
[2] https://wiki.mozilla.org/Websites/Mozilla.org/Publishing
+pmac and jgmize
We are going to begin work moving the output into Github and the templates into Bedrock in the next two weeks.

Steven - I'll ping you about getting starting.

Thx,
Jen
I'm sad that it's possible in 2014 to set up a copy of Wordpress in 5 minutes and yet the combined might of our web team can't come up with a self-publish solution for our main website which doesn't require a lot of hassle for them to set up for each new set of pages which want to use it :-|

If we can do what Chris More says in comment 7 (let me pull a git repo which I can run my scripts to update, and then submit a pull request), then that's OK.

Gerv
Flags: needinfo?(gerv)
(In reply to Gervase Markham [:gerv] from comment #10)
> I'm sad that it's possible in 2014 to set up a copy of Wordpress in 5
> minutes and yet the combined might of our web team can't come up with a
> self-publish solution for our main website which doesn't require a lot of
> hassle for them to set up for each new set of pages which want to use it :-|
> 
> If we can do what Chris More says in comment 7 (let me pull a git repo which
> I can run my scripts to update, and then submit a pull request), then that's
> OK.
> 
> Gerv

We will do what Chris More suggests in comment 7.
(Reporter)

Comment 13

5 years ago
Gerv: Do you have a preference between these two options?

A. You provide output from your gen-ng-info script (including HTML tables, etc.) and we include it in our template, or;

B. You provide raw-ng-list.txt and we move the HTML-ification from gen-ng-info into bedrock

I don't feel strongly either way. Just taking your output from gen-ng-info (HTML and all) is probably the quicker option.
Flags: needinfo?(gerv)
I don't have a preference. Happy to provide HTML if that's easier for you guys. And it gives me more flexibility to update the list format if things change.

Gerv
Flags: needinfo?(gerv)
(Reporter)

Comment 15

5 years ago
Here's a PR to add the the template for /about/forums/ to bedrock: https://github.com/mozilla/bedrock/pull/2116

This will still need to include the generated HTML from Gerv's script somehow. Gerv, it would also be helpful if the tables in your output could get the CSS class "table" added. I could work around that with the output you have, but it would be cleaner to just add the CSS class.

Next, I'll update the PR to add the etiquette and cancellation pages, and the necessary redirects.
Can we really not just include the current output and then update the HTML in PRs after that? It's an infrequently changing list of things. Coding automation for this seems silly. It'd also be MUCH easier to do l10n (if that's required) if the strings were in the template.

If we can't do that, then it looks like we can do something similar to what we've done to credits, which is to consume the data file (raw-ng-list.txt) and provide the data to the template context where we'll handle the HTML (like :sgarrity mentioned in comment #13 option B). I'd much rather have the HTML in our templates so that we can control styling issues like that mentioned in comment #15. I'm much more trusting of formatted data I can verify than HTML that we'd have to parse and sanitize.

This leaves l10n. Are we planning on translating this page (forums)?
Flags: needinfo?(jbertsch)
I'd be fine with you guys working directly with raw-ng-list.txt if you prefer.

The auto-generated bit won't be translated. The header could be, although perhaps we should add a warning that, except in regional groups, the standard language is English.

Gerv
We do not need L10N for this page.  Thx!
Flags: needinfo?(jbertsch)
(Reporter)

Comment 20

4 years ago
Created attachment 8498869 [details] [diff] [review]
bug-896977-remove-about.diff
Attachment #8498869 - Flags: review?(pmac)
(Reporter)

Comment 21

4 years ago
(In reply to Steven Garrity [:sgarrity] from comment #20)
> Created attachment 8498869 [details] [diff] [review]
> bug-896977-remove-about.diff

This patch removes the remaining /about/forums/ files, now that these pages are in bedrock.
Comment on attachment 8498869 [details] [diff] [review]
bug-896977-remove-about.diff

Review of attachment 8498869 [details] [diff] [review]:
-----------------------------------------------------------------

Some of these files are still the source of data for the new pages in bedrock. Gerv maintains naw-ng-list.txt and bedrock gets it out of SVN from this location. We should check into moving it elsewhere so that we can continue cleaning up this directory, but I know at least the following files are scripts and may be in use:

gen-ng-info
commit
checklist
replace-between
raw-ng-list.txt

We should discuss with Gerv moving what he wants of this stuff to a new and more permanent location in the SVN repo and let him clean up the rest.
Attachment #8498869 - Flags: review?(pmac) → review-
Hi Gerv-

Do you want to discuss this in the bug (see above comment) or should I try to find time to meet on vidyo?

Thx,
Jen
Flags: needinfo?(gerv)
I don't think this needs a Vidyo meeting. I'm happy for this info to move elsewhere in SVN or even to its own repo if you just tell me where it's gone.

If Bedrock works directly on raw-ng-list.txt then we only need the following files to remain:

raw-ng-list.txt
checklist

Gerv
Flags: needinfo?(gerv)
(Reporter)

Comment 25

4 years ago
Gerv: Do you have a preference where this would end up? Do you still want it in SVN, or would a github repo work? Could it just live directly in bedrock?
Somewhere where I can check in without having to file a pull request and wait would be strongly preferred. SVN is fine. Github is fine if it's now OK by policy to throw random repos up there.

Gerv
Hi Pmac-

Please review comments 22-26, and let's brainstorm a location.

Thx,
Jen
Flags: needinfo?(pmac)
I'd suggest keeping it in SVN. It's working fine. We should just move it to a new folder. I don't really see a good place for such a thing in the existing hierarchy though. Perhaps a new top level "data" folder? 


/data/forums/raw-ng-list.txt

and

/data/credits/names.csv


Alternately data like this could be considered a "project" and go somewhere like...

/projects/mozorg_data/credits/names.csv
/projects/mozorg_data/forums/raw-ng-list.txt

Either is fine with me. Once we move it's a simple change in bedrock settings.
Flags: needinfo?(pmac)
(In reply to Paul McLanahan [:pmac] from comment #28)
> I'd suggest keeping it in SVN. It's working fine. We should just move it to
> a new folder. I don't really see a good place for such a thing in the
> existing hierarchy though. Perhaps a new top level "data" folder? 
> 
> 
> /data/forums/raw-ng-list.txt
> 
> and
> 
> /data/credits/names.csv
> 
> 
> Alternately data like this could be considered a "project" and go somewhere
> like...
> 
> /projects/mozorg_data/credits/names.csv
> /projects/mozorg_data/forums/raw-ng-list.txt
> 
> Either is fine with me. Once we move it's a simple change in bedrock
> settings.

Hi Gerv-

Do you have a preference between either of these locations?

Thx,
Jen
Flags: needinfo?(gerv)
Note that either of these locations would require someone with more privileges in SVN than I to create them and move things. I only have write permissions in projects/mozilla.com.
(In reply to Paul McLanahan [:pmac] from comment #30)
> Note that either of these locations would require someone with more
> privileges in SVN than I to create them and move things. I only have write
> permissions in projects/mozilla.com.

Noted.  Maybe sgarrity does? :)  If not, we'll find someone else.
Flags: needinfo?(steven)
I have no preference as to the location, as long as I have write permissions there :-))

Gerv
Flags: needinfo?(gerv)
This is long done :) I'm closing this and opening a new bug for moving the forums data and scripts to a new git repo somewhere since SVN will be shutting down next year.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Flags: needinfo?(steven)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.