Closed Bug 1438302 Opened 8 years ago Closed 8 years ago

Broken Google search entry for UK Firefox download page

Categories

(www.mozilla.org :: General, defect)

Product:
www.mozilla.org
Component:
General
Version:
Production
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: emorley, Unassigned)

References

(Blocks 1 open bug)

Details

Attachments

(5 files)

Link to GitHub pull-request: https://github.com/mozilla/bedrock/pull/5461
44 bytes, text/x-github-pull-request
Details | Review
Link to GitHub pull-request: https://github.com/mozilla/bedrock/pull/5463
44 bytes, text/x-github-pull-request
Details | Review
Link to GitHub pull-request: https://github.com/mozilla/bedrock/pull/5464
44 bytes, text/x-github-pull-request
Details | Review
Link to GitHub pull-request: https://github.com/mozilla/bedrock/pull/5466
44 bytes, text/x-github-pull-request
Details | Review
Link to GitHub pull-request: https://github.com/mozilla/bedrock/pull/5470
44 bytes, text/x-github-pull-request
Details | Review
STR: 1) In Firefox or Chrome from the UK, search for the phrase "download firefox" on https://www.google.co.uk Expected: * Top search result is the Mozilla download page * The result's metadata is in English Actual: * Top search result is the Mozilla download page * The result's metadata is not in English and looks like some phishing page Screenshot: https://screenshots.firefox.com/FdVwbhRSI0o0xMdO/www.google.co.uk
The Google cached copy of the page isn't in English either: https://webcache.googleusercontent.com/search?q=cache:UHmotBebfZgJ:https://www.mozilla.org/en-GB/firefox/new/ Interestingly if I follow the "Current page" link from the cached page, it takes me to: https://www.mozilla.org/sr/firefox/new/ ...so presumably the bug is that it thinks the `sr` page is same page but a canonical URL for the `en-GB` page?
Flags: needinfo?(rraue)
Confirmed that I see this in the UK too (on both Firefox and Chrome, logged-in or not): https://screenshots.firefox.com/3erRi0TNGnhvylt5/www.google.co.uk
On our site everything is good. The canonicals are right, the hreflang as well. The only problem I see is that en-GB is duplicate content, but not to sr but en-US. The traffic is stable on en-GB and sr, I will observe this and maybe its just a temporary hickup. But we should deal with the duplicate content issue. I will file a seperate bug for this.
Flags: needinfo?(rraue)
I digged deeper last week and I am still sure, this problem (now not Serbian but Indonesian) is a Google problem. But I found some issues which could have lead to this Google problem: 1. All our en-GB pages are nearly complete duplicates to the en-US equivalents 2. Google doesnt like duplicate content, they recomment canonicals from duplicates to the original 3. Our Search traffic is going down for en-GB/firefox/new/ https://drive.google.com/file/d/1MUtEjYM9hTk56NanpiJ9IM5lH9BaWUyp/view?usp=sharing 4. A lot of pages (29 out of 110) in /en-GB/ are flagged as duplicate content and not even indexed https://docs.google.com/spreadsheets/d/16Q7dqjoExW-zSC44uDxCqbxTt6ZUt_bTmug8RGwdNG8/edit?usp=sharing 5. But if I understand it right, we want a UK version in the index (it could be a strategical decision to change to an en version, without localisation of ca, us, gb, that would make it easier but harder in the future to target these markets with a specific campaign or version) We should try the following to convince Google its not totally unique but localised content. We can be only 100% sure, that something like the reported bug will not happen again, when the content is really unique, but we can try first, if some changes will not do it as well: The most important pages are /en-GB/, /en-GB/firefox/ and /en-GBfirefox/new/. Apple.com for example has very similar content as well but indicates in the <title> which version it is with (United Kingdom) or (UK). We should do this as well for this three pages to test. For all en-GB pages we should change <h2 class="nav-logo"><a href=“/en-GB/">Mozilla</a></h2> to <h2 class="nav-logo"><a href=“/en-GB/“>Mozilla United Kingdom</a></h2> or Mozilla (UK) /en-GB/firefox/new/ doesn’t have the navigation, here we should change: <h2><a class="firefox" href="/en-GB/firefox/"><img class="" src="/media/img/logos/firefox/logo-quantum-wordmark-white.bd1944395fb6.png" srcset="/media/img/logos/firefox/logo-quantum-wordmark-white-high-res.3ccf0070280e.png 1.5x" width="133" alt="Firefox" height="50"></a></h2> <h2><a class="mozilla" href="/en-GB/"><img src="/media/img/logos/mozilla/wordmark-dark.731d4dab7347.svg" alt="Mozilla" width="101" height="34"></a></h2> To <h2><a class="firefox" href="/en-GB/firefox/" title=“Firefox (UK)”><img class="" src="/media/img/logos/firefox/logo-quantum-wordmark-white.bd1944395fb6.png" srcset="/media/img/logos/firefox/logo-quantum-wordmark-white-high-res.3ccf0070280e.png 1.5x" width="133" alt=“Firefox (UK)“ height="50"></a></h2> <h2><a class="mozilla" href="/en-GB/" title=“Mozilla (UK)”><img src="/media/img/logos/mozilla/wordmark-dark.731d4dab7347.svg" alt=“Mozilla (UK)“ width="101" height=“34"></a></h2> These are small changes, but if could help Google to differentiate the pages better. Additional we should solve the crawling locales problem in this bug, so crawlers can navigate between the pages as well not just via hreflang: https://bugzilla.mozilla.org/show_bug.cgi?id=1439377
Ok, so we have three steps here: 1.) Change the <title> attributes for /en-GB/, /en-GB/firefox/, and /en-GB/firefox/new/ to add `(UK)` at the end. 2.) In the global navigation for all /en-GB/ pages, set the logo link text to `Mozilla (UK)`. 3.) On /en-GB/firefox/new/, set both the logo `title` and `alt` attributes to `Firefox (UK)` and `Mozilla (UK)` respectively. Does this sound correct, Raphael?
Flags: needinfo?(rraue)
Assignee: nobody → agibson
Status: NEW → ASSIGNED
Looks good, I will sent the pages not in the index right now to the index again, once this change is live.
Flags: needinfo?(rraue)
Commit pushed to master at https://github.com/mozilla/bedrock https://github.com/mozilla/bedrock/commit/1d17493d8951ede1544515e3150444ad16876cd1 Bug 1438302: Add canonical link tag tests to headless integration tests (#5464)
https://technicalseo.com/seo-tools/hreflang/ gives a bunch of "Return tag(s) value(s) not matching self-referencing value(s)" errors when I run it against https://www.mozilla.org/en-US/firefox/new/. The reason(s) for those errors are not clear to me, but it seems possible that it could be related.
And this page suggests that the "x-default" tag should be for a language selector or auto-redirector URL (e.g. our urls without the locale) and not a page with the content: https://technicalseo.com/crawl-indexation/annotations/link-element-rel-attribute/alternate-hreflang/
Thank you Josh, I checked with this and some other tools as well but the hreflang setup for /firefox/new/ is right. The x-default can be any url in or outside the set, because its just for the languages and countrys not having a matching alternate in the set. This t can not be, our setup is here very comon. If we have our locale sitemap page (like the one apple has (https://www.apple.com/choose-country-region/), we could set this as the x-default. The not return mistake has to be a mistake in the tool. Check it yourself, the return tags are right, they match perfectly, I checked them even with differnet comparing tools and different encodings. I checked the setup as well with deepcrawl, audisto, sistrix and screamingfrog and they do not throw this error. We do have a lot no return mistakes in the whole website (around 800) and I will file a bug for this as well, but as /new/ is affected and the /new/ setup is right, I would like to first rule out all possible mistakes directly pointing to /new/. I hope the correction of scene2 will help as there are definetly errors which could affect /new/ as well.
Here is a good resumee on how different x-default can be used and when it is faulty. Our setup is like the one of dropbox: https://hrefbuilder.com/complete-guide-to-using-the-using-x-default-hreflang-element
Based on that info it still seems like our x-default should be set to the locale-free URL so that the default for when there is no known translation will be to the redirector that will give them the the best thing based on their browser settings. It might not be an error, but this seems like the best way for us to use the x-default tag, and is in fact how it was set for a long time. I'm looking for the bug where it changed now.
Looks like we changed it as part of some experimentation in the middle of last year in bug 1369738 and the PR below. https://github.com/mozilla/bedrock/pull/4907
(In reply to Raphael Raue from comment #16) > Here is a good resumee on how different x-default can be used and when it is > faulty. Our setup is like the one of dropbox: > https://hrefbuilder.com/complete-guide-to-using-the-using-x-default-hreflang-element Thanks Raphael, that page was very helpful for me. However, in the dropbox example on that page, the x-default href value does not include a locale. There is a different example on that page, under the section "What are the most common X-Default errors?", that (unfortunately) appears to more closely match our current implementation. I agree with Paul's comment #17, and I think that merging his PR https://github.com/mozilla/bedrock/pull/5466 could plausibly help resolve this bug. What do you think, Raphael?
Flags: needinfo?(rraue)
(In reply to Josh Mize [:jgmize] from comment #19) [...] > There is a different example on that page, under the section "What are the > most common X-Default errors?", that (unfortunately) appears to more closely > match our current implementation. [...] To clarify, we aren't making the exact mistake described in that section, as we are not setting multiple locales for the x-default. It "more closely" matches in that it's the only example I've seen that includes the locale in the href value in combination with hreflang="x-default".
I disagree. The x-default is just for not set locales and languages and our setup is comon. Something confuses the hreflang annotations, it might even not be the hreflang itself, but its not the x-default, that works well as we can see for example in tamil, a language we do not support (btw. maybe we should, its spoken by 70 Million people): https://screenshots.firefox.com/WuwLjSyG6sEybShP/adwords.google.com. There the x-default comes into play and its the one we set. I agree, that we should set it to a language select page, when we have it but would not add now a possible way of confusion. Google is weirdly speaking about auto-redirect pages as x-default, but in all other documents about hreflang to never ever redirect based on ip or language, what we do. And I am suspecting this redirects still as a possible cause for this bug. To find out, we need to do stuff step by step. Else we could solve one bug and just get the next one. The only thing we could do right now, is to take away the x-default. That would let Google decide which version it takes (it will take en-US) in regions and languages we dont target. Statement moz.com: "You can double-annotate a URL when using X-default, in case you were afraid to. Don't worry, it's cool." - https://moz.com/blog/hreflang-behaviour-insights And another sophisticated tool (competitor of deepcrawl) stating, that everything is fine with the x-default (the hint we can ignore, as google stated several times it ignores the htlm lang): https://audisto.com/accounts/3/crawls/83826/groups/hreflang/7/
Flags: needinfo?(rraue)
(In reply to Raphael Raue from comment #21) > as we can see for example in tamil, a language we do not support > (btw. maybe we should, its spoken by 70 Million people) We do officially support Tamil we just don't have the /firefox/new/ page translated into it and therefore it's not active for that language, but these things change often. That's why it would be better to point search results like these to our redirector page. Our l10n team relies largely on volunteer translators in their local communities and so not all languages are always up to date. So one day the redirector might not send someone to Tamil, but the next it might be ready. You can see the state of the translations of /firefox/new/ here: https://l10n.mozilla-community.org/langchecker/?locale=all&website=0&file=firefox/new/quantum.lang Tamil is nearly there and so it could become available at any time (translation updates don't require a deployment by us). Also our redirector is smart enough to pick the best available language from the list the user provides via their browser settings. So perhaps they had "Accept-Language: ta, hi, en". If the were browsing Google in Tamil they'd get our /en-US/ page because of our x-default. But if they just came to www.mozilla.org/firefox/new/ they'd get the page in their 2nd preferred language: Hindi, and more importantly then get to download Firefox in Hindi. I think this is a far superior experience and I'm not very comfortable waiting for google to index our pages for a new translation to be available via search.
See Also: → 1440469
Assignee: agibson → nobody
Status: ASSIGNED → NEW
Blocks: 1441206
I'm going to close this bug as FIXED. Follow up SEO-related fixes are being tracked in Bug 1441206. Thanks very much for reporting!
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Commit pushed to master at https://github.com/mozilla/bedrock https://github.com/mozilla/bedrock/commit/38a443d678bfa537473034d1b673ed17535917ce Bug 1438302: Move /firefox/new/?scene=2 to /firefox/download/thanks/
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: