937983 - [SEO] Consider delisting /firefox/all/ from search results

Status: RESOLVED FIXED

(www.mozilla.org :: General, defect)

Description

[Kohei Yoshino [:kohei]]

Google is currently including /firefox/all/ in the sitelinks of /firefox/  at least in the en-US and en-CA locales:

https://www.google.com/search?q=firefox&hl=en&gl=us
https://www.google.com/search?q=firefox&hl=en&gl=ca

While /firefox/all/ is useful, we'd like people to certainly visit /firefox/ for download. We can ask Google to delist /firefox/all/ from the sitelinks by a) demoting the URL in the Webmaster Tools, or b) adding <meta name="robots" content="noindex"> to that page.

Adding noindex might be a better option as demoting doesn't always work.

Comment 1

[Chris More [:cmore]]

Let's try to demote it in Google's Webmaster tools and see what happens. I can do that soon.

Comment 2

[Chris More [:cmore]]

I tried a demotion. It will only be temporary, so lets' just see first what type of impact it has. It will probably take a few days.

Comment 3

[Jennifer Bertsch [:jbertsch]]

I'm going to mark this one resolved->fixed for now.

Cmore - please re-open if additional action is required.

Comment 4

[Kohei Yoshino [:kohei]]

I just checked the search results but /firefox/all/ are still indexed.

> https://www.google.com/search?q=firefox&hl=en&gl=us
> https://www.google.com/search?q=firefox&hl=en&gl=ca

Comment 5

[Chris More [:cmore]]

(In reply to Kohei Yoshino [:kohei] from comment #4)
> I just checked the search results but /firefox/all/ are still indexed.
> 
> > https://www.google.com/search?q=firefox&hl=en&gl=us
> > https://www.google.com/search?q=firefox&hl=en&gl=ca

Well, then the <meta name="robots" content="noindex"> will probably be the only option. I looked at all inbound keywords to this /firefox/all/ page and none of the keywords look like anything specific to firefox in other languages. They are all variations of Firefox, download, and free.

Comment 6

[Kohei Yoshino [:kohei]]

Attachment: PR on GitHub

Sent a PR to add <meta name="robots" content="noindex">

Comment 7

[[github robot]]

Commits pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/8a837a51d0b1e6bb022435fc781dc21f4e3a3e6a
Bug 937983 - [SEO] Consider delisting /firefox/all/ from search results

https://github.com/mozilla/bedrock/commit/8fe900a81b61e2564ddd2f8e4ea02fc6c04784f1
Merge pull request #1442 from kyoshino/bug-937983-firefox-all-noindex

Bug 937983 - [SEO] Consider delisting /firefox/all/ from search results

Comment 8

[Kohei Yoshino [:kohei]]

Let's keep this open until we make sure that /firefox/all/ is gone from search results.

Comment 9

[Chris More [:cmore]]

(In reply to Kohei Yoshino [:kohei] from comment #8)
> Let's keep this open until we make sure that /firefox/all/ is gone from
> search results.

agree! :)

Comment 10

[raymond [:retornam] (inactive)]

pushed to stage. Marking as resolved. I'll leave this open for Kohei to verify

Comment 11

[Chris More [:cmore]]

The page is still in Google results and it may take a little longer. I received the follow automated email from Google so it seems that they are aware of it:

"Dear webmaster of http://www.mozilla.org/en-US/firefox/all/

We wanted to bring to your attention an issue with http://www.mozilla.org/en-US/firefox/all/ and its availability to search engines (including Google’s). Currently, the page contains a “noindex” robots meta tag, that causes it to be removed from our and other search engines' search results.

On http://www.mozilla.org/en-US/firefox/all/ it is currently showing:
<meta name="robots" content="noindex">

This will affect Google's ability to make this content available through search results. If you are unaware that you are preventing Google from indexing your site, we hope you find this notification useful. If you are blocking Google intentionally, please feel free to contact me with any concerns you may have.

For more information about how to use the Robots Exclusion Protocol to control how Googlebot accesses your site, please see our Help Center:  http://support.google.com/webmasters/bin/answer.py?answer=79812 and https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag

If you are not the appropriate person to contact for this issue, it would be greatly appreciated if you pass this information to them. Feel free to contact me with any questions you may have.

Thank you,
Gary Illyes

PS We are sending a copy of this message to Google Webmaster Tools for this website for verification purposes."