URLs containing 'en' language don't redirect to 'en-US'

RESOLVED FIXED

Status

developer.mozilla.org
General
--
minor
RESOLVED FIXED
4 years ago
8 months ago

People

(Reporter: teoli, Unassigned)

Tracking

Details

(Whiteboard: [seo])

(Reporter)

Description

4 years ago
What did you do?
================
1. Go to https://developer.mozilla.org/en/events


What happened?
==============
The page shows the right content but didn't redirect to the KUMA-official redirect.

What should have happened?
==========================
The page shows the right content but didn't redirect to the KUMA-official redirect: https://developer.mozilla.org/en-US/events

Is there anything else we should know?
======================================
I tried this out on https://developer.mozilla.org/en/docs/Web, too.  It appears 
to be all URLs are affected by this.
Severity: normal → minor
Summary: event page with old URL doesn't redirect to the new one → URLs containing 'en' language don't redirect to 'en-US'

Updated

4 years ago
Component: General → Localization

Updated

4 years ago
Component: Localization → SEO
OS: Other → All
It's important that I point out that canonical address in wiki pages is correct.
Bulk update, please ignore.  You can search and delete on the word BASKINGSHARK
Component: SEO → General
Whiteboard: [specification][type:bug] → [seo]
Duplicate of this bug: 1356307

Comment 5

10 months ago
Anyone can reproduce it in local environment with docker?
I can reproduce in my local machine
https://developer.mozilla.org/en/events doesn't resolve.

https://developer.mozilla.org/en-US/events redirects to https://www.mozilla.org/en-US/contribute/events/. This was done in https://github.com/mozilla/kuma/commit/cdab893388, committed Apr 23, 2015.

https://developer.mozilla.org/en/docs/Web serves the same content as https://developer.mozilla.org/en-US/docs/Web, and includes the header link that en-US is the canonical version.

In the docker development environment, http://localhost/en/docs/Web is a 302 redirect to http://localhost/en-US/docs/Web. This is what we want. I can't duplicate production behavior locally.

Safwan, how did you reproduce it in your local machine?

Comment 7

10 months ago
> Safwan, how did you reproduce it in your local machine?

Sorry, I meant I can not reproduce it locally. Thats why I have asked if anyone could reproduce it locally.
Thank you for the clarification Safwan. No one can reproduce this locally, which means it is more likely to be a quirk of the SCL3 infrastructure.  I'm not planning on researching further, but instead concentrating on getting to AWS and seeing if it reproduces there. 

I got the URLs wrong for the docker development environment, which serves on port 8000. http://localhost:8000/en/docs/Web is a 302 redirect to  http://localhost:8000/en-US/docs/Web.

I tested last with the sample database, and today I tested with the anonymized production database. It works the same, with a 302 redirect to the proper locale, so it's not a database setting, or it could be one that is cleared by the anonymization script:

https://github.com/mozilla/kuma/blob/master/scripts/anonymize.sql

The most likely candidate would be the Constance settings, but I can't imagine any of those that would cause this.
I and jwhitlock has discussed over IRC and found that the apache configuration is the root cause of this.(1) Therefore none of us could reproduce it locally.

I have also found that, as per bug 754534, its expected behavior. Not only for en, we also do not redirect for cn, zh_cn, zh_tw locales for some historical reason that mentioned in bug 754534.

Now we need to go through bug 754534 and found the actual reason for implementing this feature and decide whether we should have this in AWS

1. https://github.com/mozilla/kuma/blob/master/configs/htaccess#L75-L78
See Also: → bug 754534
Good find Safwan. I read that long discussion, and I don't find any justification for serving those URLs without a redirect.  I think it was a temporary change during the transition from Mindtouch to Kuma that became permanent.

Desired:

http://localhost:8000/en/docs/Web is 301 to http://localhost:8000/en-US/docs/Web
http://localhost:8000/cn/docs/Web is 301 to http://localhost:8000/zh-CN/docs/Web
http://localhost:8000/zh_cn/docs/Web is 301 to http://localhost:8000/zh-CN/docs/Web
http://localhost:8000/zh_tw/docs/Web is 301 to http://localhost:8000/zh-TW/docs/Web

In the local environment:

http://localhost:8000/en/docs/Web is 302 to http://localhost:8000/en-US/docs/Web (good)
http://localhost:8000/cn/docs/Web is 302 to http://localhost:8000/en-US/cn/docs/Web (bad)
http://localhost:8000/zh_cn/docs/Web to http://localhost:8000/en-US/zh_cn/docs/Web (bad)
http://localhost:8000/zh_tw/docs/Web to http://localhost:8000/en-US/zh_tw/docs/Web (good)

Against prod:

https://developer.mozilla.org/en/docs/Web = 200, document (bad)
https://developer.mozilla.org/cn/docs/Web = 200, document (bad)
https://developer.mozilla.org/zh_cn/docs/Web = 200, document (bad)
https://developer.mozilla.org/zh_tw/docs/Web = 200, document (bad)

I think the fix it to add additional redirect rules to kuma/redirects/redirects.py, and to remove these lines from htaccess, which should remove the rules from production on the next push.

Dave mentioned these and other rules in https://github.com/mozilla/kuma/pull/4231. I think the broken and unimplemented rules deserve a look.
See Also: → bug 1362438
>I think the fix it to add additional redirect rules to kuma/redirects/redirects.py, and to remove these lines from htaccess, which should remove the rules from production on the next push.

I think as we already handle the locale redirects from LocaleMiddleware and the locale mapping lives in setting `LOCALE_ALIASES`, mapping the zh_cn, zh_tw, cn can be possible from there.
Interesting thing is that, for mindtouch redirect, 
https://developer.mozilla.org/cn/Web redirect to https://developer.mozilla.org/zh-CN/docs/Web

This is a weird behavior and make it complex to fix the bug!
I've got an alternate solution in:

https://github.com/mozilla/kuma/pull/4381

I made a mistake in #c10. The results in the local environment were correct, but not my annotations:

http://localhost:8000/en/docs/Web is 302 to http://localhost:8000/en-US/docs/Web (OK, but 301 would be better)
http://localhost:8000/cn/docs/Web is 302 to http://localhost:8000/en-US/cn/docs/Web (bad)
http://localhost:8000/zh_cn/docs/Web is 302 to http://localhost:8000/en-US/zh_cn/docs/Web (bad)
http://localhost:8000/zh_tw/docs/Web is 302 to http://localhost:8000/en-US/zh_tw/docs/Web (bad)

I've also only noted the first redirect, using curl:

curl -v http://localhost:8000/cn/docs/Web 2> /dev/null

The full redirect chain with *current master* is:

http://localhost:8000/cn/docs/Web 302 to
http://localhost:8000/en-US/cn/docs/Web 301 to
http://localhost:8000/zh-CN/docs/docs/Web is 404

I've been focusing on getting that first redirect right, but yes we'll need to consider the whole chain before calling this fixed.

Some of the current requests will be fixed:

GET /en/docs/Web/JavaScript/Reference/Global_Objects/Array/find
GET /en/docs/Web/JavaScript/Reference/Global_Objects/Object/constructor
GET /cn/docs/cn/Canvas_tutorial/Basic_animations
GET /cn/docs/cn/JS_GetImplementationVersion
GET /zh_cn/
GET /zh_cn/docs/Web/CSS/perspective
GET /zh_cn/Apps/Fundamentals/Audio_and_video_delivery
GET /zh_tw/docs/Web

Others seem to rely on the "add /docs/" behavior, and should work if the target doc exists:

GET /en/XUL/tabbrowser
GET /en/NsIPrefService
GET /cn/XUL_Reference
GET /cn/XUL/toolbarpalette
GET /zh_tw/Core_JavaScript_1.5_%E6%95%99%E5%AD%B8/%E5%BE%AA%E7%92%B0%E8%AA%9E%E6%B3%95/while_%E8%AA%9E%E6%B3%9

Other are weird, and may still be broken:

GET /cn/docs/cn/JavaXPCOM

The goal isn't to save every single one of these requests - some should be 404s and fall out of search indexes.  I added some analysis of how big the issue is to GitHub, and /en/ is the big one:

https://github.com/mozilla/kuma/pull/4231#issuecomment-324359728

Comment 15

9 months ago
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/c18bf5e8a1227614185d540296cc78267ed5857b
[Bug 962148] URL with 'en' locale don't redirect to 'en-US'

https://github.com/mozilla/kuma/commit/286948a9ba70454914eacf983015197500c05419
Merge pull request #4387 from safwanrahman/en_us

[Bug 962148] URL with 'en' locale don't redirect to 'en-US'
Fixed pushed to stage and production.
Status: NEW → RESOLVED
Last Resolved: 9 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.