Closed Bug 801909 Opened 13 years ago Closed 11 years ago

Move /credits & /credits/faq into bedrock

Categories

(www.mozilla.org :: Legacy PHP system, defect, P3)

Product:
www.mozilla.org
Component:
Legacy PHP system
Platform:
x86
macOS
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: bensternthal, Assigned: pmac)

References

Details

(Whiteboard: u=user c=legacy p=2 [kb=377757] r=130270)

Attachments

(1 file)

Link to Github pull-request: https://github.com/mozilla/bedrock/pull/2131
44 bytes, text/x-github-pull-request
Details | Review
This bug captures moving /credits and /credits/faq from legacy into bedrock. In /credits there are some perl scripts that are used to generate the html and svn commit the file. Gerv, is the only user of these scripts.. so not 100% sure on how we should accommodate this.
Ok im working on it!
Assignee: nobody → bsternthal
We should probably use the Perl scripts as inspiration for the Python that will generate this page. This should be a "./manage.py" command. We can then add it to the deployment process.
Once this is up on demo I will talk to Gerv about these scripts. I tend to think they are no longer necessary. It's pretty trivial to add users to this page manually, I do not really see the need to generate this page from a txt document. The other part of what these scripts do is related to SVN and commit statements. This workflow will change if we decide to move this to bedrock/git, again another reason I think we can eliminate the scripts. I'll know more once I do a show and tell with gerv.
The point of these scripts is that I can open up my email folder called "credits" which has a big pile of emails in, and (if people have formatted it correctly, which they often do) copy and paste the line onto a command line which says: ./commit 'Gervase Markham <gerv@gerv.net>: "Gerv did stuff"' The scripts then extract the name, put it into the correct place in the names.txt file, regenerate index.html from names.txt in sorted order, and check in both files with the correct checkin comment which keeps a record of the person's contribution. All of these functions are valuable, and the script saves me significant time. I get lots of credit requests. I wrote these scripts because it was very tedious and error-prone to manually try and insert a name into the right place in a very long list in an HTML file in alphabetical order. I do not want to have to go back to doing that. In addition, this: http://viewvc.svn.mozilla.org/vc/projects/mozilla.org/trunk/credits/index.html?view=log is a detailed list of the contributions of everyone listed, back to Thu Aug 26 22:22:57 1999 PDT (13 years, 2 months ago). So far, we have managed to preserve that across various SCM moves and website CMS changes. The new system should not break the availability of that data, or prevent it being added to. It's part of our history. Lastly, I need to be able to maintain this page, rather than it having to be done by someone else. Happy to discuss how we can achieve these things :-) Gerv
Interesting... We can definitely support some type of script that can generate this page, ideally it's something that you or whomever could update. One thing I am not sure of, preserving the SVN history in GIT. I know you can do that when importing an entire repo but not sure you can do that for a file. Perhaps the history could be exported to a flat file and made available? I think we need to investigate the above a little to see what our options are. Rik any thoughts on this one?
Looks like we could do something similar to what we do w/ product_details. 1. Gerv keeps doing exactly what he's doing. 2. We write a manage.py command to pull down latest names.txt file and generate our new template. 3. profit The only change would be that generating that index.html file by gerv's script would be superfluous, but wouldn't cause any problems either. So that part could be removed from his script or not, we wouldn't care.
A couple problems I see with the above 1. We are adding an additional dependency on the legacy svn side 2. It makes bedrock more complicated 3. Credits only gets updated when we do a bedrock deployment (might not be an issue) I think the above does get us the win of /credits living in bedrock with an updated look but we make things more complicated in the process. A big problem here is the SVN history, I can't think of a way we can preserve that if this moves into GIT outside of a flat file or something. If that's true and the SVN log must remain, we may just keep this in legacy for now and move on to the other directories.
We have A LOT of SVN dependencies in bedrock. All of our translations are in SVN and won't be moving due to lock in w/ our contributors and tools. All of the product version and language information is in SVN and likely won't be moving any time soon. This is just another thing in a line of things that we know how to do, and will continue to have to do in the future. It feels like it's making bedrock more complicated, but it's really not since we're already well used to using SVN, so it's not adding a dependency, only using an existing one. We can easily have it update outside of a deployment exactly like product_details and l10n does now. I really do think this is the way forward for now rather than adapting gerv's scripts to work with git. If we did want to adapt the scripts, we could use git and preserve the commit history by creating a new git repo just for this and importing the full history of the names.txt file. I don't really see any advantage here since it would require work, and since that would make it so that it really would require a bedrock push to update. When in the distant future we want to sunset SVN, this won't make that any harder because so many larger and more complex things already rely on it.
3) Is not really a problem assuming bedrock pushes happen at least every couple of weeks. When and if we do move, I agree that a dedicated repo with history import is the way to go. Gerv
I like the sound of that plan.
Ben: The code you prepared gave a Sandstone design to the credits page. Because it is accessed through about:credits in Firefox, I don't think we want to change the design here.
Assignee: bsternthal → pmac
Whiteboard: u=user c=legacy p=2
I'd like to move forward with adding the Sandstone design to the credits page. The Sandstone look-and-feel (and global navigation) allows people who view the credits page to more easily visit mozilla.org and learn more about the Mozilla project. Sandstone will also help to visually tie the accomplishments of the people listed in the credits to the successes of Mozilla.
jbertsch: as maintainer of that page, I'm not in principle opposed, but you need to be aware of a few constraints: - You mustn't change the content. - The page is available via every Mozilla browser product, and some non-browser products too. That includes things which aren't Firefox or Thunderbird. So if a menu bar has links for Firefox and Thunderbird but not for the actual product the page is appearing in, that would be bad. - This includes our mobile browsers, so the design needs to work well there too. (The current one doesn't work very well, so this would be an improvement which is necessary anyway.) If you could do a mockup, that would be helpful. Gerv
Based on that let's keep the existing design and not update to sandstone. Also, I was thinking (early this morning) we should make sure this page does NOT have google analytics on it.
Thanks for weighing in, Gerv. Ben - I agree. Let's keep the existing design and not update to sandstone.
Priority: -- → P3
Ben and Gerv - please recommend next steps in light of bug 850874
Depends on: 850874
Flags: needinfo?(booboobenny+bugzilla)
I do not think 850874 is valid. If we ever do decide to update the design of this, first step would be to figure out all the mozilla products that reference it so we can test.
Flags: needinfo?(booboobenny+bugzilla)
Catching up on this bug. It appears everyone agrees on a few things: The visual design of the page should not change, Gerv should be able to continue using his Perl scripts, we should continue to use Subversion (at least for now), and the Google Analytics tracking code should be removed. I would be happy to migrate the commit history to Git and update the scripts accordingly if we want to kill two birds with one stone. Aside from that and removing the GA tracking code, is any development work needed? It looks like the rest can be handed over to IT. There is also the /credits/faq page. Would love to migrate that to Sandstone if someone could point me in the right direction.
Assignee: pmac → jkarahalis
Hi John- Maybe this one falls under Justin's more complex publishing solution and should have been removed from the "ready" state? Let's hold off until Justin replies - sorry about that! Justin - is that correct? Thx, Jen
Flags: needinfo?(hoosteeno)
tl;dr Yes, let's hold off working on this bug. I'll take the card off the board. I created two bugs yesterday that relate to this bug: bug 874137 This describes a broader effort to handle "self-published" content like /credits. The implementation we're thinking of resembles :pmac's solution in comment 6. I think it should block this bug, since it is blocking similar migrations of other such content. bug 874168 This is just my pitch for building an independent web stack devoted to content with in-product dependencies. I think it shouldn't block this bug, but it sure would be nice to see.
Depends on: 874137
Flags: needinfo?(hoosteeno)
Whiteboard: u=user c=legacy p=2 → u=user c=legacy p=2 [kb=377757]
Assignee: jkarahalis → nobody
Can We possible work on this as I think this would look better on bedrock andspaced out a bit better also link to the Mozillians site Just done a test on that site and got the following Result Method used: Flesch-Kincaid (English). Flesch-Kincaid Grade level: 42. Flesch-Kincaid Reading Ease score: -55. http://www.standards-schmandards.com/exhibits/rix/index.php The Flesch-Kincaid Reading Ease score indicates how easy a text is to read. A high score implies an easy text. In comparison comics typically score around 90 while legalese can get a score below 10. The Flesch-Kincaid Grade level indicates the grade a person will have to have reached to be able to understand the text. E.g. a grade level of 7 means that a seventh grader will be able to understand the text. http://en.wikipedia.org/wiki/Flesch-Kincaid
Hi Gerv would be good to get your feedback on this page as i believe you own it
Flags: needinfo?(gerv)
Flags: needinfo?(gerv)
Hi Gerv- As I mentioned on the Forums bug, we would love to move the Credits page (along with the Credits FAQ) into our Github workflow. We publish almost daily, so keeping the pages up to date should not be a problem. Thx, Jen
Flags: needinfo?(gerv)
We are going to begin work moving these pages into Github/Bedrock in the next two weeks. Steven - I'll ping you about getting started. Thx, Jen
Jennifer: this bug lists various constraints on this page. Are they still satisfiable? Will I still be able to use my Perl scripts to manage the page? I can switch them to work with a git repo instead of SVN without too much trouble, but I need to make one checkin per added name (of which there are many per month), and I do not want to have to make a pull request per checkin. Gerv
Flags: needinfo?(gerv)
From a purely technical perspective I really don't care in which VCS the data is kept. SVN works just fine for this, and I'm fine if it stays there since it requires less busy work for Gerv. The only change we need to make is that its current location within our SVN repo is going to be decomissioned. So it would be best if it moved to its own area and out of the "projects/mozilla.org/trunk/credits" area. Perhaps a new project: "projects/credits"? All we really need is the "names.csv" file. We could have a script that updates this file on deployment, or even on a schedule via cron if that is required. But we'd write a script on our side to produce the HTML from the data in the CSV based on a template in bedrock. We've talked about this before, but I recently noticed a new import script in the repo to pull from a google docs spreadsheet. If you'd rather us integrate straight from there we could investigate that as well. But if you're happy keeping that CSV file up-to-date, then we'll be perfectly happy consuming it. That sound reasonable?
Flags: needinfo?(gerv)
I'd like to keep any remaining scripts in the same repo as the names store. (Some of the scripts help add names to the store; others format it as HTML.) There is a Google Docs spreadsheet, but pulling from it is not an automatic process. There's cleanup and triage required. The doc is the raw data. So, it would be best if you pulled from the CSV file. Gerv
Flags: needinfo?(gerv)
Sounds good to me. Sounds like we just need to move your SVN repo exactly as it is to another area of the SVN server and we'll be good to do our thing in bedrock. And technically we don't even require that. We'll just make the URL from which we pull the CSV file configurable and do our thing. The only reason it needs to move is that we'll soon be abandoning and deleting every other thing in the "projects/mozilla.org" directory there. It would be an odd place for it to continue to live after that.
Assignee: nobody → pmac
Will the non-name parts of this page be translated? This would mainly be the title and a couple of sentences that appear before and after the list of names? If not, I can keep the URL just /credits. But if so, then we can do our normal redirect to the most appropriate and available locale (e.g. /de/credits/).
Flags: needinfo?(jbertsch)
We've never had l10n of this page, and the current URL is baked into all our products back to the first release. I think we should leave things as they are. English is the primary working language of the Mozilla project. Gerv
It's just a potential upgrade. The URL would remain the same, but it would redirect to the URL with the locale. /credits/ -> /en-US/credits/ or /credits/ -> /de/credits/ ... depending on your Accept-Language header. It would be easy enough to wrap the strings and allow the locale URLs so that it would be ready to translate when and if we want. Otherwise it can stay like this.
Please leave it like this. Thanks :-) Gerv
All of the basics are done. The page is in bedrock, and there is a command for grabbing the latest names.csv from SVN. :sgarrity should double check my code and make the FAQ page prettier.
PR against pmac's branch for basic FAQ page styles: https://github.com/pmclanahan/bedrock/pull/3
Commits pushed to master at https://github.com/mozilla/bedrock https://github.com/mozilla/bedrock/commit/8cc990ac66d4000d7421f9046ce4b75664237731 Bug 801909: Port /credits and /credits/faq to bedrock. * Add credits CSV file processing. * Add FAQ and Credits templates. * Add cron command for updating credits.csv from svn. * Add new app for updating external files. * Add updating external files to deployment. * Add apache rule and cache headers. * Add support for last-modified header based on date from web server. https://github.com/mozilla/bedrock/commit/08fbb1e36be8592ed821e63d9db37911dcb2a4b2 Basic styles for Credits FAQ page (Bug 801909) https://github.com/mozilla/bedrock/commit/9e1c35461a6c9ce50efeda089e515bc7477cb4e4 Merge pull request #2131 from pmclanahan/port-credits-801909 Bug 801909: Port /credits and /credits/faq to bedrock.
Woo! We should have this in production soon. Thanks all!
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Credits is now powered by bedrock in production: https://www.mozilla.org/credits/
Status: RESOLVED → VERIFIED
/credits/faq/ is down. Likely an apache config thing. I'm investigating.
Status: VERIFIED → REOPENED
Resolution: FIXED → ---
Removed special handling for /credits/* from .htaccess in r130270.
Whiteboard: u=user c=legacy p=2 [kb=377757] → u=user c=legacy p=2 [kb=377757] r=130270
Pushed change from comment #39 to prod in r130273.
Commits pushed to master at https://github.com/mozilla/bedrock https://github.com/mozilla/bedrock/commit/00ba6ef6578040ea2576bc3605a805cd279b0d43 Bug 801909: Fix 404 on /credits/faq/ https://github.com/mozilla/bedrock/commit/235b04d9deb438715cf46f6fe04db778e978bb97 Merge pull request #2161 from pmclanahan/fix-credits-faq-801909 Bug 801909: Fix 404 on /credits/faq/
404 fix has been pushed to production and all is well once again.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Commits pushed to master at https://github.com/mozilla/bedrock https://github.com/mozilla/bedrock/commit/cc55a4708c1e3d7aa11e7836907f83bbb8063d38 Bug 801909: Add validation to externalfiles. * Move to a real management command. * Add tests for updating and validation. * Add ability to specify file class in settings. https://github.com/mozilla/bedrock/commit/ce803f06d8c798f2cad295ea258e5d0ac8bba0f4 Merge pull request #2164 from pmclanahan/fix-credits-missing-500-801909 Bug 801909: Add validation to externalfiles.
Clearing needinfo
Flags: needinfo?(jbertsch)
pmac: it's not obvious from the patch - how often does the cronjob run? When I update the names, I used to tell people "within the hour", but it's been 9 hours now. Is it daily? Gerv
Flags: needinfo?(pmac)
Right now it's only on deploy. We could make it run on a schedule if that'd be preferable, but we do push at least 3 or 4 times per week (usually). Let me know if we should revisit this.
Flags: needinfo?(pmac)
pmac: yes, please. If we can't use some kind of hook to detect checkins, and we can't provide me a big red button to push, then it would be great to run the job regularly. If it's not resource-intensive, why not every hour? Gerv
See Also: → 1142850
For the record, it looks like the CSV files of names is still stored in SVN and referenced from bedrock here: https://github.com/mozilla/bedrock/blob/master/bedrock/settings/base.py#L487 Since it is referenced with the SVN URL, and not a www.mozilla.org/ URL, this should continue to work even after the PHP site is no longer served from www.mozilla.org.
What :sgarrity said in comment #48 is right. However, I've gotten word that there is a concerted effort to shut down SVN completely sometime next year. So we should think about moving this process to git.m.o or github. If we were on github there are possibly some cool webhook things we could do to update the credits and forums data.
Feel free to move it, although it's pretty important to preserve the history of that file, which contains all of the citations of people over the years. That shouldn't be too hard, I hope - there seem to be lots of good SCM-munging tools out there these days. If you file a bug for this work, CC me - I'll need to know if it moves. git.m.o would be a fine home. Gerv
Cool. This isn't a rush since SVN should be around at least through Q1 2016, so we likely won't revisit this until Q4 this year at the earliest. I'll go ahead and file bugs for moving credits and forums to git though and CC you gerv. And yes, I'm confident that preserving history will be no problem.
Blocks: 1188890
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: