Closed Bug 1244308 Opened 8 years ago Closed 8 years ago

Spammy URLs should 404

Categories

(developer.mozilla.org Graveyard :: General, defect)

Product:
developer.mozilla.org Graveyard
Component:
General
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: shobson, Assigned: willkg)

Details

(Keywords: in-triage) 

Google has started emailing us (admin contact for Google Analytics account) about URLs like http://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Live_(Online)_Teen_Wolf_Season_5_Episode_13_:_Codo

Which is was spam and was deleted by the content team but the URL remains.

These URLs need to be removed from the site or Google needs to be prevented from accessing them.

This is affecting our standing in the search results.
This seems legit, the expected behavior for when a document is logically deleted should be:

1. if the user is a super user, show the restore page with a HTTP status code of 200
2. if the user is not a super user, show the 404 page
Grabbing this to look into now.
Assignee: nobody → willkg
Status: NEW → ASSIGNED
Looks like that url is already returning an HTTP 404 and the page is tagged with x-robots-tag: noindex.

I'll fix it so that the http response content is the regular 404 page if the user is not a super user.
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/74152c73ac9c6aaa54be462ad05aa2f5bff95e3f
Fix bug 1244308: Return 404 content if user can't restore the document

Previously, MDN would return an HTTP 404 with an HTML document that
has some stuff on it and lets people restore the document. That's not
great because somehow those pages are getting indexed or otherwise
making search bots grumpy.

Instead of doing that, this changes MDN to return a straight-up
regular 404 page unless the user can restore documents in which case
they get the HTML page that lets them restore documents.

https://github.com/mozilla/kuma/commit/b4b8e479e8e8400ab9bbbb96042fb66381b63d77
Merge pull request #3777 from mozilla/1244308-404

Fix bug 1244308: Return 404 content if user can't restore the document
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Product: developer.mozilla.org → developer.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.