Closed Bug 1026184 Opened 10 years ago Closed 10 years ago

Set up repo and move files for /security migration to Bedrock

Categories

(www.mozilla.org :: Bedrock, defect)

Product:
www.mozilla.org
Component:
Bedrock
Version:
Production
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: malexis, Assigned: pmac)

References

(Blocks 1 open bug)

Details

(Whiteboard: [kb=1414973] )

Attachments

(2 files, 1 obsolete file)

Link to Github pull-request: https://github.com/mozilla/bedrock/pull/2305
44 bytes, text/x-github-pull-request
Details | Review
Link to Github pull-request: https://github.com/mozilla/bedrock/pull/2391
44 bytes, text/x-github-pull-request
Details | Review
Whiteboard: [kb=1414973]
Repo created:

https://github.com/mozilla/foundation-security-advisories/
Assignee: nobody → pmac
Initial script and import of the MFSAs is done and committed to the repo:

https://github.com/mozilla/foundation-security-advisories

Check it out when you get time and let me know how you think it went. I think the results are quite good.

The Known Vulnerabilities are not nearly as numerous, so I'm thinking it's probably not worth it to adapt the script for them, and we can just manually convert them, or whichever of them you wish to convert. 

For now while we still have the HTML in SVN in production, this script can and should be run to rewrite all of the files, then we can commit the differences. Once the move is complete the script can be deleted.
Flags: needinfo?(jmize)
Flags: needinfo?(abillings)
Status: NEW → ASSIGNED
Is everything in the span tags going to turn into metadata? Also, I notice an extra line before the product version numbers (which won't necessarily matter if it is metadata).

Otherwise, it looks fine to me. Dan?
Flags: needinfo?(abillings) → needinfo?(dveditz)
(In reply to Al Billings [:abillings] from comment #3)
> Is everything in the span tags going to turn into metadata? Also, I notice
> an extra line before the product version numbers (which won't necessarily
> matter if it is metadata).

We can do that. The only bit that I've made metadata so far is the title which I'm extracting from the PHP bit. I can easily parse and move that info up to metadata which would give us more control over the layout in future.
We discussed this in the meeting.

I'd like the non-description (and non-buglink) fields to be metadata. So, Title, Reporter, Announcement Date, Security Rating, Affected Products, Product Versions.
So we did. I've almost got that done. Should have updates for you tomorrow.
I've finished the update to the script and reimported all of the announcements. It's all done in:

https://github.com/mozilla/foundation-security-advisories/commit/ca161d0bc200f5a1d88c03566151fb826ee7a493

Let me know if you see any issues. I believe this is pretty good. We've got two "title" items in metadata. The "page_title" is from the PHP variable, and the "title" is from the <span> in the document. Not sure we need them both, but I figured it couldn't hurt.
Those two titles are duplicate. I cut and paste the second into the first. :-)

I assume "fixed_in" will take multiple values (it is free text?)? Actually, the same goes for Product and Reporter.

The text and metadata looks good on the samples.
(In reply to Al Billings [:abillings] from comment #8)
> Those two titles are duplicate. I cut and paste the second into the first.

page_title is slightly different, it's "MFSA" + mfsa_id + title. It should be a redundant bit of meta data, but when you do create the page's <title> element make sure it's composed of the right parts, not just copying title metadata alone.

I guess it looks good? How do the metadata bits at the top get converted to HTML and styled? Is there an example of what it looks like?
Flags: needinfo?(dveditz)
(In reply to Daniel Veditz [:dveditz] from comment #9)
> page_title is slightly different, it's "MFSA" + mfsa_id + title. It should
> be a redundant bit of meta data, but when you do create the page's <title>
> element make sure it's composed of the right parts, not just copying title
> metadata alone.

I'll keep the "title" one then (from the HTML), and recreate the other in the template where needed. I agree it's redundant since we have all of the information to recreate <title>.

> I guess it looks good? How do the metadata bits at the top get converted to
> HTML and styled? Is there an example of what it looks like?

The "what it looks like" bit is still to come. We'll do that in a pull-request to bedrock and show it to you before we merge it to master. Having the metadata as just metadata allows us some flexibility to make it look however we want, and change it as needed w/o having to parse and reprocess all of these things.
There are some files that don't put a "Title:" item in the metadata section, and there are some that have some HTML in the title (e.g. <attr> and <kbd> tags). So I'm opting to use the title from the page when I get it (we'll strip tags dynamically when filling in the <title>), and when I don't have that because it was elsewhere on the page (an <h2> usually), pull it in from the PHP variable. I've committed these changes. Most pages the result is just that the "page_title:" medatadata item was removed, but you can see the ones where the "title:" changed as well, and those are the ones that had some difference in the two. We can handle any further anomalies on a one-off basis hopefully. Adding more special cases to the script is also possible, but we're nearing the point of diminishing returns on this I believe.
What you have so far looks good to me, :pmac, and +1 on diminishing returns for special cases.
Flags: needinfo?(jmize)
What are the next steps here? We shipped last week. We have five weeks until we ship again. If we don't want to wait six or seven weeks, we need to do this soon. :-) I don't want to do this near the end of a ship cycle.
Hi Al-

We think we can get this done in that time frame.  What is your deadline to have it done before the next release?

Thx,
Jen
We ship on September 2 so we'd need this stable and usable by, say, August 19, giving us two weeks to have it waiting for us and usable (and to shake out any style issues, etc.).
Thanks, Al.

Hi Mike- Could you please regroup with Josh and Pmac about what needs to be done when to make this date?  Thx!
Flags: needinfo?(malexis)
Al,

I'm back on this. In my investigation of the /known-vulnerabilities/* pages, it seems they're just a different view of the data in /announce/; right? So I should be able to just generate the pages for the product announcements as well as for the individual product versions.

If there is other data in the K-V area that isn't in the announcements please let me know, but I think we can do all of the page generation we need just based on the data from the documents we now have in the repo on github.

If this is true I'll start on the "bedrock" side of this thing.
Flags: needinfo?(abillings)
(In reply to Paul McLanahan [:pmac] from comment #17)

> I'm back on this. In my investigation of the /known-vulnerabilities/* pages,
> it seems they're just a different view of the data in /announce/; right? So
> I should be able to just generate the pages for the product announcements as
> well as for the individual product versions.

That's correct. They're just a roll up of the advisories for an affected product. I create them by hand when I have the final advisory list.
Flags: needinfo?(abillings)
Fantastic. I've started on the bedrock (website) side of things. The plan then, just so we're clear, is:

1) website checks for new stuff from the repo on github every 30min or so (TBD)
2) website uses data contained in the markdown files to display the various lists of advisories.
3) website uses markdown content for the individual advisory pages.

This sound about right to you? The static pages (e.g. /security/) will just be static templates in our repository and can be modified by filing a bug to have us do it, or by sending us a pull-request with the changes.
Al,

One more thing: Do we want to take this opportunity to alter the URLs for the advisories? Clearly we'll redirect the old urls to the new, but since we need to do that regardless (at least from *.html -> */), we could do something like:

current:
/security/announce/2014/mfsa2014-01.html

proposed:
/security/advisory/2014/01/
or even:
/security/advisory/2014-01/

But if we did the first proposal above, we could also use the "advisory/2014/" as a list per-year as well.

If you have other suggestions that would be great too. The above is all the code needs to find the correct info, so just let me know if you have a preferred structure. I just feel like there is a lot of redundant info in the current URL structure and now is the best time to change it.
Flags: needinfo?(abillings)
I don't want to move them because of all the links to them, even with a redirect, but let's get Dan's opinion.
Flags: needinfo?(abillings) → needinfo?(dveditz)
I'd really like to at least remove the ".html" from the ends. None of the other bedrock urls have file extensions. We can keep 'em if we have to, but we do enjoy pretty URL design around here :)
I'd really prefer to make the url changes that Paul suggests in Comment 20, with redirects, in order to keep the site information architecture clean.
(In reply to Paul McLanahan [:pmac] from comment #20)
> current:
> /security/announce/2014/mfsa2014-01.html
> 
> proposed:
> /security/advisory/2014/01/
> or even:
> /security/advisory/2014-01/

I don't mind moving from /announce/ to /advisory/, but where does the current security/announce/ list end up? leaving it as the sole content in /announce/ would be silly, have a list of hundreds of advisories in a directory called /advisory/ singular seems a little odd. Maybe use "advisories" as the name instead?

I hate the /2014/01/ format. These things are names that people reference elsewhere and the slash gives too much separation. /2014-01/ would be OK but I'd prefer to keep the 'mfsa' in the link. /mfsa2014-01/ without the .html is my preference.

One reason for the extra /year/ directory was that I was concerned it would be inconvenient to deal with thousands of advisories in a single directory. We've got about 750 advisories now and adding 100 or so a year. If that's not really a problem these days I guess we could lose that bit of the URL.
Flags: needinfo?(dveditz)
(In reply to Daniel Veditz [:dveditz] from comment #24)
> I don't mind moving from /announce/ to /advisory/, but where does the
> current security/announce/ list end up? leaving it as the sole content in
> /announce/ would be silly, have a list of hundreds of advisories in a
> directory called /advisory/ singular seems a little odd. Maybe use
> "advisories" as the name instead?

"advisories" it is.

> I hate the /2014/01/ format. These things are names that people reference
> elsewhere and the slash gives too much separation. /2014-01/ would be OK but
> I'd prefer to keep the 'mfsa' in the link. /mfsa2014-01/ without the .html
> is my preference.

"/security/advisories/mfsaYYYY-XX/" works for me.

> One reason for the extra /year/ directory was that I was concerned it would
> be inconvenient to deal with thousands of advisories in a single directory.
> We've got about 750 advisories now and adding 100 or so a year. If that's
> not really a problem these days I guess we could lose that bit of the URL.

With this change we're separating the file structure from the URL structure. Currently (as you can see in the github repo) the old structure is pretty much intact. If you'd like that changed it's easy at the moment. If you like it the way it is, that's fine too. That part is purely for you and your workflow. I'm just using the files as a data source to update the database in bedrock. The website will display the data from the database, so both the file structure for the source data and the URLs can be anything we like.
Blocks: 504882
As part of the port I'm moving static pages (like the bug-bounty and faqs), but there are some clearly old pages that likely need to be archived or just deleted (e.g. /security/shell.html). Does anyone have ideas on which need porting, archiving, or deleting within the /security/* section?

http://viewvc.svn.mozilla.org/vc/projects/mozilla.org/trunk/security/
This still needs a bit more work, but the main parts are done, and it can start being reviewed and styled.
(In reply to Al Billings [:abillings] from comment #15)
> We ship on September 2 so we'd need this stable and usable by, say, August
> 19, giving us two weeks to have it waiting for us and usable (and to shake
> out any style issues, etc.).

I wanted to mention we've kind of hit this date. Where are we at? Is this going to be usable when we ship in two weeks?
It's in review. I asked about other pages in comment #26 but haven't heard anything. We could switch to only moving certain pages to bedrock for now and deal with others later, but it makes the configurations a bit more complex.

Other than that we're reviewing the code and I'm trying to get some help making sure the look & feel is good enough (right now I think it could still use some work).

Safe bet (in my opinion) is to assume it won't be ready and to go for the normal procedure. The import script should grab any new advisories after that so it shouldn't require any extra work on your part. 

I'm pinging mike again to see if he agrees.
(In reply to Paul McLanahan [:pmac] from comment #26)
> As part of the port I'm moving static pages (like the bug-bounty and faqs),
> but there are some clearly old pages that likely need to be archived or just
> deleted (e.g. /security/shell.html). Does anyone have ideas on which need
> porting, archiving, or deleting within the /security/* section?
> 
> http://viewvc.svn.mozilla.org/vc/projects/mozilla.org/trunk/security/

Jen, are you OK with pushing without some of these old pages to meet our deadline?

We'll review the PR in tmrw's triage if no one gets to it by then and target to go live this week.
Flags: needinfo?(malexis) → needinfo?(jbertsch)
I'm OK with that.
Flags: needinfo?(jbertsch)
Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/a730fe41dac8c5f7f50983e0123ed9fc731ed7bc
Bug 1026184: Port /security to bedrock.

Add an update_security_advisories command to pull
in and parse the advisroies from the git repository.

https://github.com/mozilla/bedrock/commit/03ea497949b496eba24f7b855f37ef6713992642
Merge pull request #2231 from pmclanahan/port-mofo-security-advisories-1026184

Bug 1026184: Port /security to bedrock.
We've pushed the initial version of the bedrock-based pages to production:

https://www.mozilla.org/b/security/

Note the "/b/" in the URL. That is there because we've not yet replaced the old urls. We've deployed the new pages and done the database schema migrations and the initial import of data. Now we want to make the pages a bit more beautiful and install scripts that will update the data from the new github repository every half hour.

All this to say that we're nearly there, and it's my opinion that you should feel good about moving forward with the advisories for the next release going into the github repo. Docs for doing so are in the README:

https://github.com/mozilla/foundation-security-advisories#writing-new-announcements
Looking at these, I see two things missing from the rendered content.

We need the date and the reporter name rendered above the content body (as well as the reporter name that I put by hand in the content body).
Oh yeah. The individual advisory pages are woefully inadequate so far. I basically threw some of the data in a template as a useage example. Our friend Steven Garrity will soon make them pretty and brimming with information.
Any further update here? Clearly we didn't make the last release but I am hopeful for the next one.
(In reply to Al Billings [:abillings] from comment #37)
> Any further update here? Clearly we didn't make the last release but I am
> hopeful for the next one.

Steven Garrity's PR to apply styling to the pages (https://github.com/mozilla/bedrock/pull/2279) is still in the review process, but that looks close to complete based on the discussion in the PR.
And to add to what jgmize said, we're feeling pretty good about what we have. Might need to tweak it a bit more, but I'll try to have a demo up for you to see and try soon so that we can crush any remaining bugs or styling issues.
Al and Daniel,

We've got a version out now that I think is good for launch. If you agree, we can make the changes to the apache config to replace the existing URLs with it. For now it's behind the "/b/" url prefix where bedrock things are before we configure them. You can see the latest-and-greatest at:

https://www.allizom.org/b/security/

Let us know what you think and we can make changes or prepare the changes required to move this live.
Flags: needinfo?(dveditz)
Flags: needinfo?(abillings)
Visually, things are looking good. Your script seems to be barfing on metadata for products and versions though.

Compare:
https://www.mozilla.org/security/announce/2014/mfsa2014-66.html (Firefox and Thunderbird)

https://www.allizom.org/b/security/advisories/mfsa2014-66/ (Only lists Thunderbird)
Flags: needinfo?(abillings)
This means your rollup pages are rather empty too.

https://www.allizom.org/b/security/known-vulnerabilities/firefox/ only shows Firefox 28.0.1 and earlier and not much in some of these.
Actually, the rollup pages are missing versions and advisories all over. Thunderbird at https://www.allizom.org/b/security/known-vulnerabilities/thunderbird/ doesn't show https://www.allizom.org/b/security/advisories/mfsa2014-66/ even though metadata (as displayed in -66) should be correct.
(In reply to Al Billings [:abillings] from comment #41)
> Visually, things are looking good. Your script seems to be barfing on
> metadata for products and versions though.
> 
> Compare:
> https://www.mozilla.org/security/announce/2014/mfsa2014-66.html (Firefox and
> Thunderbird)
> 
> https://www.allizom.org/b/security/advisories/mfsa2014-66/ (Only lists
> Thunderbird)

Ah yes. I see you've changed how you do multiple "fixed in"s. Prior to these it was a single "fixed in" label with blank labels following, now it's multiple "fixed in" labels. I can alter the import script and run it again.
Based on mfsa2014-66 it looks like we also need to bring over some "p.note" styling for the new pages. I'll talk to Steven Garrity about that next week.
I believe I've got most of these issues fixed, though not live yet. The import wasn't dealing properly with some ways of indicating multiple lines of "Fixed in". These have now been reimported and are correct in the foundation-security-advisories repo.

The issue with the missing Thuderbird advisories was due to the fact that "31" is the first time you've not added the ".0" onto the end of the version. Our version comparison class expects at least "31.0" and so was failing making it seem that "31" was less than all other versions. Since the product advisories list pages have a minimum version they show, those were falling off the end. I've submitted the following PR to fix this:

https://github.com/mozilla/bedrock/pull/2298

So we should be ready for another look early next week. Thanks for jumping on the review so quickly.
Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/b992e64f86aa2e6fd992ca95f51bb7c4f45802d3
Bug 1026184: Fix security product version compare.

Would fail to create a proper version if version
string contained no minor version (e.g. "31").

https://github.com/mozilla/bedrock/commit/fff80e6dc1c0f33ede214fa022e4ba2a81dc8de5
Merge pull request #2298 from pmclanahan/fix-security-advisory-issues-1026184

Bug 1026184: Fix security product version compare.
I apologize for inconsistencies. There is a bit of freeform text involved. I use the template to try to avoid doing something stupid.

Once the old versions are imported correctly, we just have to make sure we do it correctly going forward.  

If we construct the metadata area in the future in such a way to avoid ambiguities, it might be a good thing, even if the source looked redundant. I'm not sure the best way you might want to structure this in the future.
Advisory redirects from the old to new names seem to work fine, but old "known-vulnerabilities" links are broken. For example

  /security/known-vulnerabilities/firefox15.html
became
  /security/known-vulnerabilities/firefox-15/

but there's no redirect.

The anchor format in the known-vulnerabilities pages has changed (sprouted a hyphen), but now there's a nice '#' to let people know there's an anchor so that's a win. It's kind of big and ugly though. Could we shrink the '#'? Make each headline itself the link?

Not keen on using purple for "high" bugs. If you can't make red-orange-yellow work in the site palette then I'd rather have "high" be the yellowish ones (slightly more saturated if possible) because that's more of a warning color than purple. The moderates could be a pale purple or blue if that all we can use, but they should be paler than the current "high" purples you've got now.

The advisory/known-vulnerabilities lists seem extremely stretched in the vertical direction. They do match the release notes style so maybe I just need to get used to it.
Flags: needinfo?(dveditz)
(In reply to Daniel Veditz [:dveditz] from comment #49)
> Advisory redirects from the old to new names seem to work fine, but old
> "known-vulnerabilities" links are broken. For example
> 
>   /security/known-vulnerabilities/firefox15.html
> became
>   /security/known-vulnerabilities/firefox-15/
> 
> but there's no redirect.

Ah! Good catch. I'll add that.
 
> The anchor format in the known-vulnerabilities pages has changed (sprouted a
> hyphen), but now there's a nice '#' to let people know there's an anchor so
> that's a win. It's kind of big and ugly though. Could we shrink the '#'?
> Make each headline itself the link?

We can revert to the old anchors. Depending on the browser the anchor may survive a redirect, to those old links that might be out there could work.

> Not keen on using purple for "high" bugs. If you can't make
> red-orange-yellow work in the site palette then I'd rather have "high" be
> the yellowish ones (slightly more saturated if possible) because that's more
> of a warning color than purple. The moderates could be a pale purple or blue
> if that all we can use, but they should be paler than the current "high"
> purples you've got now.

I'm sure we can work that out. I think we were indeed trying to stick with a color palate we already had, but we can certainly tweak more. Steven?

> The advisory/known-vulnerabilities lists seem extremely stretched in the
> vertical direction. They do match the release notes style so maybe I just
> need to get used to it.

Meaning that each row in the list is a bit tall? We can reduce the padding around them, but I personally like the space. It does make the page a bit tall, but it was already pretty tall to begin with. Perhaps Steven can advise us on this one as well?
Flags: needinfo?(steven)
(In reply to Al Billings [:abillings] from comment #48)
> I apologize for inconsistencies. There is a bit of freeform text involved. I
> use the template to try to avoid doing something stupid.

Oh no worries. I hope I didn't come across as accusatory. I of course realize that the import would be fragile. Once we're on the new system hopefully we'll be able to more easily be more consistent.

> Once the old versions are imported correctly, we just have to make sure we
> do it correctly going forward.  

Exactly.

> If we construct the metadata area in the future in such a way to avoid
> ambiguities, it might be a good thing, even if the source looked redundant.
> I'm not sure the best way you might want to structure this in the future.

Please take a look at the current files and let me know if you have ideas. I've reduced ambiguity and repetition as much as I thought I could so far. For example, I did have the ID as a data item in the file, but the filename itself has that, so I removed it. The metadata no longer needs "products" as the same can be gleaned from "fixed_in". If you see other areas that could be improved in this way I'm very happy to implement them.

I was thinking that just copying an old file and changing the values would be the way to create a new one, but of course a little CLI program could easily be constructed to help in the consistent creation of new files if you like. That'd be a project for post-launch, but anything like that we can do to make this transition easier seems in scope to me.
We've just pushed an update to stage that I think addresses many concerns. Please have a look and we can continue this tomorrow.

https://www.allizom.org/security/
Flags: needinfo?(steven)
Actually I was wrong and we're waiting on one more pull-request to pass review and be merged. Should have everything updated later today.
Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/926a786379997215ce98242524af36846185599b
Bug 1026184: Fix known-vulnerabilities anchors and add redirects.

* Add flag for clearing the security advisories data.
* Default to stripping '.0' from versions for consistent naming.

https://github.com/mozilla/bedrock/commit/e20a6af0edc4f7b399fa2ef385fcc75dcb9a2f59
Merge pull request #2305 from pmclanahan/add-redirects-known-vulnerabilities-1026184

Bug 1026184: Fix known-vulnerabilities anchors and add redirects.
Apparently, we haven't had any advisories since September, 2011. :-)

https://www.allizom.org/security/announce/
:abillings I'm re-importing all of the announcements now :)
Okay. All advisories should be imported now. Time for a test.
Not seeing any differences.
Oh. Stage seems woefully behind on its copy of the mozilla.org SVN repo. But what I meant was the new bedrock version of the security center at https://www.allizom.org/b/security/
The latest code and advisories are now in production. The cron job to update the advisories hourly on prod is also in place and working:

https://www.mozilla.org/b/security/advisories/

So we can switch to this version at least for these pages any time you're ready. There are still some older pages that we need to decide what to do with (like those mentioned in comment #26). We have an archival process for things that are of historical interest, or of course they can be ported as is, or deleted entirely if the information is just woefully out of date. I can probably get you a list of these pages if you need it.
Where would archived pages go?

I just noticed, looking at the source, that the "author" metadata wasn't preserved. I'm not sure Dan or I cared but that could be used to tell which of us wrote an advisory even though we never displayed it. Since we don't display it, it may not be worth adding to the header metadata in the markdown source files.
(In reply to Al Billings [:abillings] from comment #62)
> Where would archived pages go?

There's an archive site on SVN where we put things that need to stay online, but don't need their templates updated and won't ever change again. For example:

http://website-archive.mozilla.org/www.mozilla.org/devpreview_releasenotes/projects/devpreview/system-requirements.html

Many firefox release notes are there as well:

http://website-archive.mozilla.org/www.mozilla.org/firefox_releasenotes/en-US/firefox/3.6/releasenotes/
 
> I just noticed, looking at the source, that the "author" metadata wasn't
> preserved. I'm not sure Dan or I cared but that could be used to tell which
> of us wrote an advisory even though we never displayed it. Since we don't
> display it, it may not be worth adding to the header metadata in the
> markdown source files.

Interesting. I see the author meta tag in the files. Is that how they all are? I think it would be good to have author info in the metadata of the file, as well as a field in the DB in case we ever want to query for them that way. I'll see about making those changes as soon as I can.
(In reply to Paul McLanahan [:pmac] from comment #63)

> Interesting. I see the author meta tag in the files. Is that how they all
> are? I think it would be good to have author info in the metadata of the
> file, as well as a field in the DB in case we ever want to query for them
> that way. I'll see about making those changes as soon as I can.

I don't know about all. Dan started adding it at some point and I kept using it (changing it to me) when I started writing stuff a couple of years ago.
Great. Not even sure why I asked about "all". It can and should of course be optional.
Dan points out that my memory is mistaken and I added this when I took over writing advisories to make it clear it was me. You could just mark the older ones as him or leave it blank.
If the "Products" section makes any sense (as a first-glance "do I need to worry about this one") it should go before the Fixed-in section. Otherwise it just looks redundant with the "Fixed-in" section.

I'd /almost/ say we could kill that section, but every once in a while we put a specific affected product version in it as a way of saying the bug was not in all old versions. We usually repeat that information in the body though so...

The title blocks on the known-vulnerability pages are wrong: currently it repeats "Mozilla Foundation Security Advisories" from the main index; it should instead boldly announce which product is covered in that list. You can sort of guess from the sub-headings below that, at least the difference between Firefox and thunderbird. But the distinction between "modern firefox" and the "Firefox 1.5" page might be lost. The old title was "Security Advisories for <product>". I'm not against some other way of stating that as long is the product name (and sometimes version) is in there clearly.

The known-vuln pages for obsolete versions of Firefox and Thunderbird (e.g. Firefox 1.5, Thunderbird 2.0) had a big warning box saying those versions were no longer supported and that people should upgrade. It's probably still a good idea to have something like that.
Once these are addressed, I'd like to switch the live system over to the new Bedrock based pages. Then we can discuss what to do with the rest of the security pages.
Are we ready to go? We've shipped another release and there are more advisories to import.

Should I open a new bug to discuss the other security pages and the porting of them to github (whether this repo or another)?
Flags: needinfo?(pmac)
From my perspective we are. I think there are still some things :dveditz mentioned in comment #67 that need to be addressed, but after that and I get the apache rules sorted we should be ready. And filing a new bug for the remainder of the pages sounds good to me.

I should have time today to get a new PR in for the things needed to get us live.
Flags: needinfo?(pmac)
(In reply to Daniel Veditz [:dveditz] from comment #67)
> If the "Products" section makes any sense (as a first-glance "do I need to
> worry about this one") it should go before the Fixed-in section. Otherwise
> it just looks redundant with the "Fixed-in" section.
> 
> I'd /almost/ say we could kill that section, but every once in a while we
> put a specific affected product version in it as a way of saying the bug was
> not in all old versions. We usually repeat that information in the body
> though so...

It's fine with me to ditch the products section. They're the same list from the import right now. Completely up to you guys though. I can at least move it to above Fixed In. I can also make it a comma separated list instead of a UL as well. 

> The known-vuln pages for obsolete versions of Firefox and Thunderbird (e.g.
> Firefox 1.5, Thunderbird 2.0) had a big warning box saying those versions
> were no longer supported and that people should upgrade. It's probably still
> a good idea to have something like that.

That should be doable. Such a warning is a bit more tricky now though since the same template is used for displaying versions, and since you can really go to any specific product version page, we only link to the old ones.

https://www.mozilla.org/b/security/known-vulnerabilities/firefox-32/

We can use product-details to know the latest release version, and ESR though, so it should work pretty well. What do you think?
In the above PR I've added an outdated warning to any list page for a particular version based on product details if available for the product, or on just the hardcoded most recent for SeaMonkey, and anything else is considered old. The message is simple for now, but does include a link to the appropriate page per product to upgrade if users so choose.

I also went ahead and changed the "Products" data on the individual advisory page to be a comma separated list and moved it above "Fixed In". I think this is good for now.

This also improves page titles (title tag as well as the h1 on the page) to be more in-line with what we had for the specific version known-vulnerabilities lists.

This stuff isn't live yet, but if you'd like to check it out before merge I may be able to secure a demo server for testing. This would probably be good regardless just to ensure proper function of the redirects from old URLs and the new apache settings.
Changes are live for preview here:

https://www-demo5.allizom.org/security/
I think we're about ready. Let me know what you think about what is on demo5 (linked above).
Flags: needinfo?(dveditz)
Flags: needinfo?(abillings)
I notice that "Fixed In:" with a list of items has a bullet point per item even though it is one per line. I'm not sure we need a rendered bullet point since the list is already in its own box (by color).

Other than that, it looks great and we should pull the trigger to go live. I like the styling on our other pages under /security as well. Are they going to be served from the same repo (I think they should) with the same style? I want to go there in the end anyway.
Flags: needinfo?(abillings)
(In reply to Al Billings [:abillings] from comment #76)
> Other than that, it looks great and we should pull the trigger to go live. I
> like the styling on our other pages under /security as well. Are they going
> to be served from the same repo (I think they should) with the same style? I
> want to go there in the end anyway.

Not sure what you mean here. They're just normal pages in bedrock for now. If you mean you'd like to control the content via files in the foundation-security-advisories repo then we can discuss that. Unless the content in them will change frequently it's probably not necessary, as a quick bug or PR for a change will be easy and fast. I'm certainly open to whatever you need though.
(In reply to Paul McLanahan [:pmac] from comment #77)
> 
> Not sure what you mean here. They're just normal pages in bedrock for now.

Well, right now they're being served via SVN and we were discussing, immediately, moving the advisories and related pages to Bedrock. 

Eventually, I'd like to edit all of my pages and push changes without needing SVN anymore. I'm happy to have an advisory only repo for the advisory pages at this time though.
"they" being all the other security pages, like the Bounty FAQ page, for example or /security index.
(In reply to Al Billings [:abillings] from comment #79)
> "they" being all the other security pages, like the Bounty FAQ page, for
> example or /security index.

That's what I thought. Like I said, they're normal pages in bedrock for now, and will need to be edited there when these latest changes I made are pushed to production. Those pages are part of the initial push we're about to do, as you can see on demo5. Those pages content can be moved to markdown and into the repo with the advisories, but it'd be a lot easier if the maintenance of them could be handled with pull requests to bedrock. But again, if you're not happy with that we can port them into your repo and you can manage their content there. It just seemed to me like overkill since they don't seem to change very often.

Do you want to chat about this further before we launch? I can setup something if you'd like.
No, we can launch. I'm leaving the country late Wednesday night so I probably won't have time to chat.

I don't think Dan or I know how to edit pages in Bedrock since we use SVN right now to make changes. If you can point us to docs and make sure we have edit access to them via Bedrock, that would be good.
Bedrock is in github, and we'd just need a pull-request so that we can make sure that the changes look good from a code perspective, and that it merges cleanly and tests pass. Contribution docs are available[0]. In those docs are also instructions for getting bedrock running so you can see and test your changes if you want, but for pure content changes this is likely overkill.

Unfortunately "edit access" is limited as we try to keep the pool of people who can push to the repo as small as possible to avoid conflicts and reduce surprises when preparing a production push. We get around this limitation with the pull-request process like every other project on github. So the easiest thing will be to prepare and submit a pull request against bedrock's master branch, ping me or any other bedrock dev about it in #www or via github with an @ mention (@pmclanahan) in a comment, and we'll get it merged and in prod asap. 

If you do decide that this process isn't good enough, we can discuss pulling page content from other sources, but I hope we can work something out to keep them simple. 

[0] http://bedrock.readthedocs.org/en/latest/contribute.html
Ok. Let's get this live.

Dan and I had checkin privs on mozilla.org via SVN but if the permissions are more restrictive now, we'll work around it.
The code review for my latest changes should be in progress, and as soon as that is done and it merges we'll be ready for prod. I'll update this bug when it happens.

Thanks Al.
After tomorrow, needinfo? dveditz for anything until 11/7. I'll be back after that.
Commit pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/f4f7dd4a442aaadc81d5dcd2333dfa44795a0fe6
Fix Bug 1026184 and Fix bug 1026184 for /security

* Fix titles and add obsolete warnings for known-vulnerabities
* Change "Products" entry from UL to comma separated list.
* Add management command for security advisories to deployment for dev/demo.
* Add apache rules for new security center.
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
This just went to production:

https://www.mozilla.org/security/

All updates to advisories should happen in the new github repository from now on:

https://github.com/mozilla/foundation-security-advisories

There is a cron job on our servers that will update the site based on the master branch of that repo every hour. Let's close this bug out and open new ones for any further issues or new features.
Flags: needinfo?(dveditz)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: