505077 - Make AMJ (addons.mozilla.jp) the default add-on library for Japanese users AGAIN: embed (or redirect) and whitelist AMJ URLs in ja/ja-JP-mac builds

Status: RESOLVED WONTFIX

(Mozilla Localizations :: ja / Japanese, defect)

Description

[Kohei Yoshino [:kohei]]

*** Separated from Bug 454300 Comment 6 ***

Bug 454300 will replace %LOCALE%.add-ons.mozilla.com/%LOCALE% URLs with simple addons.mozilla.org/%LOCALE% URLs.

AMJ (addons.mozilla.jp) relies on the reditects from ja.add-ons.mozilla.com/ja so now we have to resolve the situation.

AMO (addons.mozilla.org/ja) itself does/should not have such redirects, and one possible solution is embedding the AMJ URLs directly into ja/ja-JP-mac builds.

Comment 1

[Kohei Yoshino [:kohei]]

Attachment: patch

Use firefox-l10n.js to override addons.mozilla.org/%LOCALE% URLs set by firefox.js.

Comment 2

[Kohei Yoshino [:kohei]]

Attachment: patch v2

I'd also like to add addons.mozilla.jp to xpinstall.whitelist.add.103. With the pref, people can install add-ons from AMJ without seeing an information bar.

Comment 3

[u217243]

Comment on attachment 399608 [details] [diff] [review]
patch v2

>+pref("xpinstall.whitelist.add.103", "addons.mozilla.org,addons.mozilla.jp");

A new pref "xpinstall.whitelist.add.36" has been added as a part of Bug 511771.
So we need this:

pref("xpinstall.whitelist.add.36", "getpersonas.com,addons.mozilla.jp");

Comment 4

[Reed Loden [:reed]]

Have you discussed this with l10n-drivers or the AMO team yet? I'm against this, personally.

Comment 5

[u217243]

Attachment: patch v2.1

Will bug 454300 be landed on Firefox 3.6? As I wrote in comment 0, this is absolutely essential to our continuous add-on site services. (or Japanese users will have to use useless AMO search.) See https://wiki.mozilla.org/Japan/AMJ for details of our serivices.

Adding addons.mozilla.jp to the whitelist is greatly helpful:

* AMJ is the default add-on library for Japanese build users. With the pref, we can significantly improve the experience of those users when they install add-ons.
* Also, we'll be able to showcase recommended Personas (lightweight themes) for Japanese users.

Dynamis, can you check the patch and get reviewed by an appropriate reviewer?

Comment 6

[u217243]

Attachment: patch v2.1

Comment 7

[Reed Loden [:reed]]

(In reply to comment #0)
> AMO (addons.mozilla.org/ja) itself does/should not have such redirects\

Why should it not?

Comment 8

[u217243]

(In reply to comment #7)
> (In reply to comment #0)
> > AMO (addons.mozilla.org/ja) itself does/should not have such redirects\
> 
> Why should it not?

AMJ is only a subset of AMO/ja; it doesn't have themes, Personas, developer tools, etc. If AMO/ja redirects to AMJ, Japanese people have to use AMO/en-US instead. That's not good user experience.

Comment 9

[Kohei Yoshino [:kohei]]

Bug 454300 is now fixed in 1.9.2 final. We should fix this too. Dynamis, can you take a look at the patch?

Comment 10

[dynamis (Tomoya ASAI)]

we should include default xpinstall.whitelist.add.36 value "getpersonas.com"
http://mxr.mozilla.org/comm-central/source/mozilla/browser/app/profile/firefox.js#176

Others looks fine.
Anyway, I'm not sure I have privilege to decide changing these prefs in l10n side and we should request approval to Axel or Stas etc.

Comment 11

[u217243]

Attachment: patch v2.2

xpinstall.whitelist.add.36 was wrong.
replaced addons.mozilla.org with getpersonas.com

Comment 12

[u217243]

Attachment: patch v2.3

hmm, one more thing: removed unnecessary %VERSION%s.

Comment 13

[Axel Hecht [:Pike]]

Technically, this patch is wrong, addons.mozilla.org should still be whitelisted.

Generally, though, I'd like Nick and Wil to comment here before we proceed.

Comment 14

[u217243]

(In reply to comment #13)
> Technically, this patch is wrong, addons.mozilla.org should still be
> whitelisted.

addons.mozilla.org is already whitelited by "xpinstall.whitelist.add".
"xpinstall.whitelist.add.36" is a new configuration (please see my comment 3.)

Comment 15

[Axel Hecht [:Pike]]

Still waiting for a comment from Nick and Wil.

And I don't think this is blocking 3.6.

Comment 16

[Wil Clouser [:clouserw]]

I know we're working on some things to make the experience better for ja, but I don't know how it affects this.  Nick should comment.

Comment 17

[Mike Beltzner [:beltzner, not reading bugmail]]

Not going to block on this. If we do another RC, I'll take a ridealong, and we can fix it in a 3.6.1, but otherwise we can deal with it server side.

Comment 18

[u217243]

Sorry, I should leave a comment. Because the patch in Bug 454300 (see comment 0 and 5) has been landed to Firefox 3.6 RC 1, the Japanese builds (ja/ja-JP-mac locales) now point AMO. We have used AMJ, so this is a regression. That's why this is jp-critical and blocking 3.6. Please reconsider this request.

As I wrote before, AMJ had been the default add-on library for the Japanese builds. This is nothing new. My patch just replaces server-side redirects with in-product preferences.

I know one new preference, xpinstall.whitelist.add.36, is different and should be treated carefully. I have described why the pref is required in my comment 5.

Comment 19

[:Gavin Sharp [email: [e-mail]]]

(In reply to comment #18)
> Sorry, I should leave a comment. Because the patch in Bug 454300 (see comment 0
> and 5) has been landed to Firefox 3.6 RC 1, the Japanese builds (ja/ja-JP-mac
> locales) now point AMO. We have used AMJ, so this is a regression

Surely we can update the mozilla.com redirects to also take effect even with the changes in bug 454300? We're still linking to www.mozilla.com/ja...

Comment 20

[Mike Beltzner [:beltzner, not reading bugmail]]

The expression of this bug is not a blocker for release; as Gavin indicates, we can file a different bug to have server-side redirects get the same net effect. Let's file that bug and make sure that we fix it on the website.

I still do not think we should do a second release candidate of the browser for something we can fix with a server-side redirect.

Comment 21

[u217243]

See my comment 8 why we cannot set redirects.

Comment 22

[u217243]

Once again, technically we can set up redirects.

The following URLs are in the default preferences:
(replaced %LOCALE% and %APP% with ja and firefox)

https://addons.mozilla.org/ja/firefox/search-engines/ 
https://addons.mozilla.org/ja/firefox
https://addons.mozilla.org/ja/firefox/recommended
https://addons.mozilla.org/ja/firefox/search?q=%TERMS%

And we have

https://addons.mozilla.jp/firefox/search-engines/
https://addons.mozilla.jp/firefox/
https://addons.mozilla.jp/firefox/extensions/recommended/
https://addons.mozilla.jp/firefox/search?q=%TERMS%

As you can see, if those redirects are set, people won't be able to visit the AMO/ja homepage. It's not good for the current AMO/ja users.

Previous redirects (Bug 427323) were set for ja.add-ons.mozilla.com, it was no problem.

Comment 23

[Axel Hecht [:Pike]]

Comment on attachment 418590 [details] [diff] [review]
patch v2.3

I don't like this patch, and I don't want it. I don't think that the issues raised about AMO vs AMJ are in any way special to Japanese, and I really don't think that this kind of pref-editing is something localizers should do in general.

I'll block this on an exit strategy for patching firefox-l10n.js, though I might take an exit strategy to make a case for "perfect is the enemy of wrong but good enough".

IMHO, Nick should have a strong word on how we want this to be in the long run, still waiting on his comment.

Technically, I'm the wrong person to review this patch, you'll need reviewers that are savvy in the preferences that you change. This is effectively a change at a mozilla-central level, so I'd apply review rules of the preferences in the original code to l10n-tweaks. Yes, I'm making this up as I go, sorry, but if you're tweaking something I don't understand, that's what I'd be falling back to.

Comment 24

[Nick Nguyen [:osunick]]

We're in the midst of improving localization on AMO so that the experience is better for non english speaking users.  Having said that, AMJ solves a very real user need considering the current state of localized Japanese content on AMO.  

So in the long run, my strong preference is for AMO to be the single point for hosting and browsing add-ons.  Having said that, given the sensitivity of the Japanese market to the level of localization, I think pointing links to AMJ are fine for now.

I'm not fine with updating the whitelist as the reason why we have the whitelist on AMO is because we have an established sandbox and review process on AMO which doesn't exist for AMJ, especially as it looks like AMJ will eventually host experimental add-ons.  If the concern is Personas, the answer is to make sure the translations for the Personas areas of AMO are good, and that we feature unique content for the JA locale, this functionality exists today.

Can we not keep the existing addons.mozilla.com domain up, solely as a redirector for AMJ?  AMJ links to AMO and I don't want to deny advanced users the ability to browse AMO.

Comment 25

[Axel Hecht [:Pike]]

(In reply to comment #24)
> Can we not keep the existing addons.mozilla.com domain up, solely as a
> redirector for AMJ?  AMJ links to AMO and I don't want to deny advanced users
> the ability to browse AMO.

From an l10n point of view, this would still require a patch to firefox-l10n.js. A smaller one, though, given the rest of the comments.

Not sure about the IT side of it.

Comment 26

[Nick Nguyen [:osunick]]

we have a dedicated redirector on AMO- outgoing.mozilla.org, i wonder if this could be used for this purpose.  Wil, what do you think?

Comment 27

[Wil Clouser [:clouserw]]

(In reply to comment #26)
> we have a dedicated redirector on AMO- outgoing.mozilla.org, i wonder if this
> could be used for this purpose.  Wil, what do you think?

Not the same thing.  If we're setting up redirects, we'd just do it in apache.

Comment 28

[Reed Loden [:reed]]

(In reply to comment #24)
> Can we not keep the existing addons.mozilla.com domain up, solely as a
> redirector for AMJ?  AMJ links to AMO and I don't want to deny advanced users
> the ability to browse AMO.

Let's use a query string like ?jp-redirect or something on AMO. That let's me continue to deprecate the horrible locale-based subdomains and AMC, yet allows jp builds to use AMJ still.

Comment 29

[Mike Beltzner [:beltzner, not reading bugmail]]

(removing [rc-ridealong] since we don't have a reviewed patch yet)

Comment 30

[Kohei Yoshino [:kohei]]

Firefox 3.6 is spreading and this becomes an actual problem now. On the Add-ons Manager, the recommended add-on list is still served by AMJ via services.AMO/ja, but links including "Browse All Add-ons", "See All Recommended Add-ons" and "See all results" point AMO/ja.

Which solution would be best here?

Comment 31

[Kohei Yoshino [:kohei]]

Some Japanese users making complaints about this.
Who can review the patch?

Please fix this in Firefox 3.0.2.

Comment 32

[Kohei Yoshino [:kohei]]

I meant to say 3.6.2...

Comment 33

[Kohei Yoshino [:kohei]]

Hi,

The rest of the redirects from services.addons.mozilla.org/ja/firefox/api (SAMO) to addons.mozilla.jp (AMJ) that were set by Bug 427323 have been removed. (Is it a side-effect of landing Zamboni?) Now all redirects are gone and AMJ is orphaned :(

We strongly hope that an appropriate person could review my patch ASAP and land it on Firefox 3.6.4 or 3.6.5. Thank you for your help.

Comment 34

[Kohei Yoshino [:kohei]]

(In reply to comment #33)
> The rest of the redirects from services.addons.mozilla.org/ja/firefox/api
> (SAMO) to addons.mozilla.jp (AMJ) that were set by Bug 427323 have been
> removed.

That means the recommended add-ons and search results on the Add-ons Manager are served from AMO. It's bad for user experience because the most of those add-ons are not localized into Japanese as I wrote in https://wiki.mozilla.org/Japan/AMJ.

Comment 35

[Kohei Yoshino [:kohei]]

I just blogged about the incident to our valued customers.
http://mozilla.jp/blog/entry/5613/

Comment 36

[Kohei Yoshino [:kohei]]

We'll prepare the add-on discovery page for Firefox 4. Updated patch to come.

Comment 37

[Kohei Yoshino [:kohei]]

Attachment: patch for 1.9.2 branch (Firefox 3.6)

Comment 38

[Kohei Yoshino [:kohei]]

Attachment: patch for trunk (Firefox 4)

Here are patches, but anyway, who can review these?

Comment 39

[Kohei Yoshino [:kohei]]

Attachment: patch for 1.9.2 branch (Firefox 3.6)

real diff files...

Comment 40

[Kohei Yoshino [:kohei]]

Attachment: patch for trunk (Firefox 4)

Comment 41

[Kohei Yoshino [:kohei]]

Attachment: patch for trunk (Firefox 4)

Close the bracket. Sorry for bugspam!

At AMJ, we are working to support those APIs/URLs introduced with Firefox4:

pref("extensions.getAddons.get.url"
     "https://addons.mozilla.jp/firefox/api/%API_VERSION%/search/guid:%IDS%")
pref("extensions.webservice.discoverURL",
     "https://addons.mozilla.jp/firefox/discovery/%VERSION%/%OS%");

Comment 42

[Axel Hecht [:Pike]]

Wasn't the idea to go for a server-side solution at some point in this bug?

Comment 43

[Kohei Yoshino [:kohei]]

Nothing has been decided yet. As I wrote in comment 8, we'd like to keep access to AMO as is, so the embedding solution looks better.

Comment 44

[Kohei Yoshino [:kohei]]

And, someone at AMO could remove redirects by accident or design.
As noted in comment 33, we are already getting into such situation :(

Comment 45

[Mike Beltzner [:beltzner, not reading bugmail]]

I don't really care if it's web-side redirects (with comments to prevent that from breaking) or client-side code, but it remains a hard blocker.

Comment 46

[Mike Beltzner [:beltzner, not reading bugmail]]

Comment on attachment 504427 [details] [diff] [review]
patch for trunk (Firefox 4)

Can we get this review expedited?

Comment 47

[:Gavin Sharp [email: [e-mail]]]

Comment on attachment 504427 [details] [diff] [review]
patch for trunk (Firefox 4)

I'm not really a suitable reviewer for this. Dave would know more about any code-level requirements for these pref values, and Axel should sign off on the policy aspects of having these overridden in firefox-l10n.

Comment 48

[Axel Hecht [:Pike]]

I'm not feeling good about adding a chunk of preferences to a locale.

Also, it seems like we're hacking around AMO bugs that I'm not sure are tracked as such.

If we intend to support forked instances of AMO around the globe, we should have a bug to support that.

If we don't, we should aggressively move towards ripping things like amj out.

Comment 49

[Dave Townsend [:mossop]]

Comment on attachment 504427 [details] [diff] [review]
patch for trunk (Firefox 4)

>diff -ruN a/firefox-l10n.js b/firefox-l10n.js
>--- a/firefox-l10n.js	2011-01-17 21:17:55.000000000 +0900
>+++ b/firefox-l10n.js	2011-01-17 21:56:01.000000000 +0900
>@@ -44,3 +44,13 @@
> pref("browser.search.order.6", "chrome://browser-region/locale/region.properties");
> pref("browser.search.order.7", "chrome://browser-region/locale/region.properties");
> pref("browser.search.order.8", "chrome://browser-region/locale/region.properties");
>+
>+// Add-on Preferences
>+pref("browser.dictionaries.download.url",           "https://addons.mozilla.jp/firefox/dictionaries/");
>+pref("browser.search.searchEnginesURL",             "https://addons.mozilla.jp/firefox/search-engines/");
>+pref("extensions.getAddons.get.url",                "https://addons.mozilla.jp/firefox/api/%API_VERSION%/search/guid:%IDS%");

You've not included some of the parameters that we send to AMO now, perhaps you just don't want/need them. Also unless AMJ is included in the same log gathering stuff that AMO is doing based on these pings then we won't be seeing the data from japanese builds, that seems unfortunate.

>+pref("extensions.getAddons.search.browseURL",       "https://addons.mozilla.jp/firefox/search?q=%TERMS%");
>+pref("extensions.getAddons.search.url",             "https://addons.mozilla.jp/firefox/api/%API_VERSION%/search/%TERMS%/all/10/%OS%/%VERSION%");

You need to replace the 10 with %MAX_RESULTS% here. I'm assuming you don't want the other parameters we normally pass here.

>+pref("extensions.getMoreThemesURL",                 "https://addons.mozilla.jp/firefox/themes/");
>+pref("extensions.webservice.discoverURL",           "https://addons.mozilla.jp/firefox/discovery/%VERSION%/%OS%");
>+pref("xpinstall.whitelist.add.36",                  "getpersonas.com,addons.mozilla.jp");

Instead please define a new pref xpinstall.whitelist.add.jp-40 with just "addons.mozilla.jp"

I'd agree with many of the previous comments that server side redirects may be the better solution here however the whitelist pref at least is going to need to be here.

Comment 50

[Axel Hecht [:Pike]]

Probably a question for fligtar, should we add amj to the whitelist in general?

Comment 51

[Reed Loden [:reed]]

(In reply to comment #50)
> Probably a question for fligtar, should we add amj to the whitelist in general?

Note that security should also be involved in any decisions concerning additions to the whitelist...

Comment 52

[Justin Scott [:fligtar]]

I have a number of thoughts on this, most of which would be ideally discussed with the Japan team in a better medium than Bugzilla, but in the interest of coming to a decision on this bug, I'll post a summary of them here and encourage further discussion elsewhere.

First, let me say that I think AMJ is valuable for Japanese users and I understand the reasons for wanting this. When the Japan-specific add-ons site was created, AMO wasn't localized yet, so it was necessary.

However, AMO has now been localized for nearly 4 years, and AMJ currently pulls add-ons from AMO's API and encourages users to go to AMO for more add-ons. A lot of effort has gone into replicating AMO's front-end on AMJ, and I'm not sure why that effort hasn't been put into making AMO localization better for all locales, not just Japan.

I strongly think the in-product experience should not go to locale-specific sites for several reasons:
* AMO undergoes constant security monitoring, frequent audits, bug bounty eligibility, code reviews, EV SSL, security best practices (like working to implement CSP), and on and on. Add-on security is critically important and checking to make sure that all other in-product distribution channels are similarly situated is overhead that's really not cheap or necessary. I'm not sure if Mozilla Japan's servers are controlled by our IT and Infra Security teams, but if they aren't, that's a separate issue that this point doesn't address.

* AMO's in-product touch points are tested by the Firefox QA team and they would presumably need to check all of those touch points again specifically for Japanese builds if a different API, URL, or discovery pane is used.

* We have daily reports based on how people use many aspects of AMO, the Add-ons Manager, add-on start-up performance, and other metrics that wouldn't be available for Japanese users.

* Locales will have feature inequality. For example, when we launch add-ons marketplace or personalized recommendations, we would have to say that it's available for all Firefox users except in Japan.

* Perhaps most importantly, the efforts spent on bringing this other site up to par with AMO would be better spent improving AMO's localization efforts for all locales.

AMO is increasingly becoming an integrated part of the Firefox experience and we (the add-ons team) work with the Firefox team on features, policies, security, etc. It doesn't seem right for us to change the default add-ons site for a specific locale to one developed in isolation without consulting or learning from the people who spend all of their time working with add-ons. I had no idea we were previously rewriting certain AMO URLs to go elsewhere and I think that should be stopped if there are any still in existence.

I think it's unnecessarily confusing to have multiple official add-on sites for a locale, and think addons.mozilla.org/ja needs to go to AMJ or AMJ needs to go to AMO. Given the issues above, I think that one official site needs to be AMO. Multiple sites only serve to further segment the add-on community in one locale from the global community.

As for this bug specifically, I understand the issue to be that we removed rewrites on our API that caused the add-ons featured in the Add-ons Manager to go from Japan-specific to the generic AMO featured list. I don't understand why this would block Firefox 4 as that API method is used by Firefox 3.*, not 4. I also don't see why we would redirect a specific locale's API requests when our API is localized. If there are add-ons you'd like to feature for Japanese users, we can give you permissions on AMO to feature those add-ons in your locale and offer translations to the authors. We have a Q1 goal of having locale-specific featured lists in several top-tier locales anyway.

Going forward, it'd be great for us to talk about things AMO needs to improve for you to feel like it's a good browsing experience for Japanese users. For example, if you want the Popularity sort to show popularity based on the current locale, please file a bug that says that, because I don't know of one currently filed and it's something we have the data to do. One of our 3 over-arching goals for add-ons this year is to increase the number of Firefox users who have an add-on installed, and improving our localized experience is definitely one of the key areas we need to improve on to do that.

Comment 53

[Kohei Yoshino [:kohei]]

We do know the AMO team have done very excellent work for years. However, l10n has always been low-priority and optional. (Yeah, this is true for Mozilla's all Web-related projects.)

I shared https://wiki.mozilla.org/Japan/AMJ with the team in 2009. How many of the issues have been resolved?

AMO's interface is localizable, but descriptions and screenshots of each add-on are not localizable. The search results of タブ (tab) shows clearly the current reality:

https://addons.mozilla.jp/firefox/search?q=%E3%82%BF%E3%83%96 (110 results)
https://addons.mozilla.org/ja/firefox/search/?q=%E3%82%BF%E3%83%96 (only 13 results)

Needless to say, those issues are not Japan-specific. The Mozilla China team have also launched their own add-on library with fully localized descriptions. Did you know that?

http://mozilla.com.cn/selections/

Some other communities have done the same. Did you find and research those local sites?

https://wiki.mozilla.org/Japan/AMJ#Other_local_add-on_libraries

The most important thing is: l10n is not just translation. (Again, this is true for all projects.)

AMO has "share this" widgets, but it only shows U.S. sites. And the contributions/marketplace. PayPal is not popular yet here in Japan and other countries. Most Japanese users cannot use those functionalities. Did you discuss such marketability while designing the product? If localizers have known such things in advance, not after deploying, we could help out.

(Personally I don't know both Python and GitHub so I won't submit patches anymore. Sorry. PHP/SVN were good!)

We have no intention of taking in everything at AMO. AMJ will switch to EV SSL in the near future, but won't have developer tools nor marketplace. Our main purpose of AMJ is to help Japanese users to find localized add-ons with ease. AMJ only shows public and preliminary reviewed add-ons retrieved from AMO. 

If users couldn't find what they need on AMO, they might use Google and then find some unreviewed/insecure add-ons. We hope AMJ can reduce such risk in the meantime.

We'd like to move to AMO if AMO would have enough localizability, marketability and findability, of course. Why not discuss the AMO roadmap (if exists), developer relations abroad, add-on marketing efforts with local communities?

Comment 54

[Kohei Yoshino [:kohei]]

One more *technical* thing: connection from Japan to U.S. servers is slow. Loading of the Add-on Discovery pane in the new Add-on Manager takes more than 5 seconds. The Firefox QA team might not know that. It's a bad user experience, and is a reason why we'd like to host the pane on AMJ. For the same reason, mozilla.jp has not used MoCo's CDN.

Comment 55

[matthew zeier [:mrz]]

Do me a favor - can you ping the following IP addresses and paste results:

63.245.208.1
63.245.216.1
63.245.213.1
180.92.184.19

If you have mtr installed, I'd like that output too.  

From an operational perspective, I'd rather pull up a caching proxy of AMO in APAC much like we've done for Europe.

Comment 56

[matthew zeier [:mrz]]

fyi, my initial tests show that Japan to Singapore is like New York to San Francisco.

Comment 57

[Justin Scott [:fligtar]]

(In reply to comment #53)
> I shared https://wiki.mozilla.org/Japan/AMJ with the team in 2009. How many of
> the issues have been resolved?

We added support for the first item (being able to filter out add-ons not in your locale) in bug 532016 last year, but removed it 6 months later because of a number of issues with it. We need to figure out a better way to surface and design the feature.

> Needless to say, those issues are not Japan-specific. The Mozilla China team
> have also launched their own add-on library with fully localized descriptions.
> Did you know that?
> 
> http://mozilla.com.cn/selections/

Yes, I found out about that site (formerly http://17huohu.cn/) only from our referrer logs, not because anyone informed us about it. This is exactly the type of thing I'm concerned about: instead of the China team talking to us about improving AMO for them, they saw that Japan made their own site and decided to do the same. Putting these individual sites in-product will only encourage others to make even more of these sites instead of helping us improve the localized experience on AMO. Having a different add-ons service in different locales means inconsistencies in security, functionality, etc.

> We'd like to move to AMO if AMO would have enough localizability, marketability
> and findability, of course. Why not discuss the AMO roadmap (if exists),
> developer relations abroad, add-on marketing efforts with local communities?

Our roadmap for 2011 has been circulating among team members the last month and I'm revising it based on that feedback before blogging about it for everyone to comment. I hope to do that soon. In the meantime, you can see it here: https://docs.google.com/Doc?id=dfntthnr_92gzm6zdgf&authkey=CNec3q4I

(In reply to comment #54)
> One more *technical* thing: connection from Japan to U.S. servers is slow.
> Loading of the Add-on Discovery pane in the new Add-on Manager takes more than
> 5 seconds. The Firefox QA team might not know that. It's a bad user experience,
> and is a reason why we'd like to host the pane on AMJ. For the same reason,
> mozilla.jp has not used MoCo's CDN.

5 seconds is indeed a really bad experience, and this is something we'd like to know much sooner so that we can fix it. Setting up a separate discovery pane on servers not controlled by Mozilla IT should really have not been the solution to this. I've talked with IT and they'll try to spin up some AMO caching proxies in Singapore in time for Firefox 4 (bug 631054).

Comment 58

[dynamis (Tomoya ASAI)]

(In reply to comment #55)
> Do me a favor - can you ping the following IP addresses and paste results:

I installed mtr and here is the report:

$ mtr -c 50 --report 63.245.208.1
HOST: macairlan.office.mozilla.or Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gw0.office.mozilla.or.jp   0.0%    50    0.6   0.4   0.3   1.1   0.2
  2.|-- tokyo10-b02.flets.2iij.ne  0.0%    50    3.0   3.4   2.5  10.4   1.4
  3.|-- tokyo10-ntteast0.flets.2i  0.0%    50   10.4   4.8   1.9 103.0  14.2
  4.|-- tky001lip20.iij.net        0.0%    50    2.2   6.5   1.9 201.2  28.1
  5.|-- tky001bb11.iij.net         0.0%    50   61.2   7.0   1.9  61.2  13.7
  6.|-- tky008bf02.iij.net         0.0%    50    3.4   3.4   2.6   6.5   0.8
  7.|-- lax002bb10.iij.net         0.0%    50  114.7 116.8 114.0 154.4   7.3
  8.|-- 208.178.58.77              0.0%    50  114.5 118.4 114.3 212.4  15.9
  9.|-- ip-208.49.147.106.gblx.ne  0.0%    50  152.9 131.1 126.2 234.8  16.3
 10.|-- te-8-2.core1.sjc.mozilla.  0.0%    50  126.8 127.3 126.4 138.3   2.3
 11.|-- v4.core3.sj.mozilla.com    2.0%    50  127.0 127.3 126.7 131.6   1.0

$ mtr -c 50 --report 63.245.216.1
HOST: macairlan.office.mozilla.or Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gw0.office.mozilla.or.jp   0.0%    50    0.3   0.4   0.3   2.1   0.3
  2.|-- tokyo10-b02.flets.2iij.ne  0.0%    50    3.2   3.4   2.5  11.7   1.4
  3.|-- tokyo10-ntteast0.flets.2i  0.0%    50    2.1   3.6   2.0  44.0   6.1
  4.|-- tky001lip20.iij.net        0.0%    50    2.4   7.2   2.0 193.5  27.0
  5.|-- tky001bb10.iij.net         0.0%    50    3.8   5.1   2.0  52.4   9.8
  6.|-- tky001bf00.iij.net         0.0%    50    2.0   5.5   1.9  53.9  10.3
  7.|-- sjc002bb00.iij.net         0.0%    50  127.8 128.5 127.5 135.4   1.4
  8.|-- sjc002ix02.iij.net         0.0%    50  112.6 128.5 112.2 179.1  15.2
    |  `|-- 216.98.96.90
    |   |-- 216.98.96.130
    |   |-- 216.98.96.242
  9.|-- 206.111.6.85.ptr.us.xo.ne  0.0%    50  112.5 113.3 112.0 128.5   3.0
 10.|-- vb2001.rar3.la-ca.us.xo.n  0.0%    50  121.8 122.0 120.9 129.8   1.6
 11.|-- ae0d0.mcr1.phoenix-az.us.  0.0%    50  134.1 134.6 133.6 146.2   2.4
 12.|-- ip65-47-254-10.z254-47-65  0.0%    50  137.1 138.3 136.5 160.5   4.1
 13.|-- xe-1-0-1.0.core1.phx.mozi  0.0%    50  130.7 131.3 130.5 136.9   1.0
 14.|-- lo0.border1.phx.mozilla.n  2.0%    50  136.6 137.9 133.9 166.9   7.5

$ mtr -c 50 --report 63.245.213.1
HOST: macairlan.office.mozilla.or Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gw0.office.mozilla.or.jp   0.0%    50    0.4   0.6   0.2   6.0   1.1
  2.|-- tokyo10-b02.flets.2iij.ne  0.0%    50    3.6   3.6   2.4  10.9   1.4
  3.|-- tokyo10-ntteast0.flets.2i  0.0%    50    3.7   2.7   1.9   7.7   1.2
  4.|-- tky001lip20.iij.net        0.0%    50    2.1   3.6   1.8  41.9   5.8
  5.|-- tky001bb10.iij.net         0.0%    50    2.3   5.6   1.9  47.1   9.6
  6.|-- tky009bf00.iij.net         0.0%    50    2.5   4.3   2.2  32.5   5.8
  7.|-- lax002bb10.iij.net         0.0%    50  111.3 115.5 111.2 200.7  15.2
  8.|-- 208.178.58.77              0.0%    50  111.8 113.8 111.2 172.2  10.0
  9.|-- mozilla.gigabitethernet9-  2.0%    50  257.6 272.1 254.0 445.8  30.0
 10.|-- gslb-v5.core.nl.mozilla.c  2.0%    50  252.9 275.5 252.4 477.7  51.6

$ mtr -c 50 --report 180.92.184.19
^[OPHOST: macairlan.office.mozilla.or Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gw0.office.mozilla.or.jp   0.0%    50    0.3   0.9   0.3  10.3   1.7
  2.|-- tokyo10-b02.flets.2iij.ne  0.0%    50    2.8   3.7   2.5  10.1   1.7
  3.|-- tokyo10-ntteast0.flets.2i  0.0%    50    2.2   6.6   1.9  97.3  16.3
  4.|-- tky001lip20.iij.net        0.0%    50    2.2   4.0   1.9  79.2  10.9
  5.|-- tky001bb11.iij.net         0.0%    50    2.1   4.9   1.9  52.2   8.8
  6.|-- tky009bf01.iij.net         0.0%    50    4.7   6.9   2.6  60.8  12.5
  7.|-- tky001ix01.iij.net         0.0%    50    3.2   8.6   2.7  61.0  13.6
  8.|-- 202.232.8.146              0.0%    50    3.6   4.3   3.2  18.9   2.5
  9.|-- gi0-0-0.cr1.nrt1.asianetc  0.0%    50    2.7   3.2   2.7   8.9   0.9
 10.|-- po3-1-0.gw2.sin3.asianetc  0.0%    50   82.5  82.6  82.1  85.0   0.6
 11.|-- vdi-0001.asianetcom.net    0.0%    50   81.8  82.6  81.6  89.2   1.4
 12.|-- 20.te1-1.csr1.sin1.sg.vox  0.0%    50   80.2  81.8  80.1 115.0   5.0
 13.|-- 180.92.184.19              2.0%    50   79.9  80.6  79.6  92.7   2.2

$ mtr -c 50 --report addons.mozilla.jp
HOST: macairlan.office.mozilla.or Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- gw0.office.mozilla.or.jp   0.0%    50    0.5   0.6   0.3   5.5   0.9
  2.|-- tokyo10-b02.flets.2iij.ne  0.0%    50    4.0   3.6   2.4  12.9   1.9
  3.|-- tokyo10-ntteast0.flets.2i  0.0%    50    2.4   6.5   2.0  64.3  12.3
  4.|-- tky001lip20.iij.net        0.0%    50    2.3   6.3   1.9 194.1  27.1
  5.|-- tky001bb11.iij.net         0.0%    50    2.2   5.8   1.9  59.8  10.1
  6.|-- tky001bf01.iij.net         0.0%    50    2.9   5.1   1.9  40.4   8.6
  7.|-- ngy003bb11.iij.net         0.0%    50    7.5  12.0   7.3  75.3  13.9
  8.|-- ngy003ipgw11.iij.net       0.0%    50    8.2  10.9   7.7 124.0  16.4
  9.|-- ngy003lip20.iij.net        0.0%    50    8.1  23.3   7.7 197.0  44.9
 10.|-- gifu-nttwest0.flets.2iij.  0.0%    50   18.3  18.7  17.2  32.5   3.0
 11.|-- gifu-b01.flets.2iij.net    0.0%    50   16.3  16.7  16.1  22.5   1.0
 12.|-- addons.mozilla.jp          0.0%    50   19.1  19.5  18.6  29.0   1.5

Comment 59

[Kohei Yoshino [:kohei]]

Note that the addons.mozilla.jp server will move to a much faster network sometime soon.

Comment 60

[matthew zeier [:mrz]]

> $ mtr -c 50 --report 180.92.184.19
>  13.|-- 180.92.184.19              2.0%    50   79.9  80.6  79.6  92.7   2.2

That's about what you'd get from New York to San Francisco.


> $ mtr -c 50 --report addons.mozilla.jp
>  12.|-- addons.mozilla.jp          0.0%    50   19.1  19.5  18.6  29.0   1.5

That's about what you'd get if you were on the western side of the US, San Jose to Phoenix, for example.

Both give acceptable page load times.  I'd like to solve this for the whole region, not just a specific locale.  We're working on spinning up proxies in bug 631054 and then you guys can retest and let me know what sort of page load times you see.

As an alternative (or even in addition too), if you know of a provider in .jp where I could host three machines I'd gladly spin up proxy hardware there too.

Comment 61

[Justin Scott [:fligtar]]

I'm trying to understand the exact hardblocker for Firefox 4 in this bug -- can someone state what that is? As I mentioned in comment #52, if it's regarding the featured add-ons API, that is not used in Firefox 4.

If we give someone from Mozilla Japan access to manage the featured add-ons in that locale, those add-ons will show in both Firefox 3.6's featured list and the discovery pane in Firefox 4. Is that the blocker?

Comment 62

[Kohei Yoshino [:kohei]]

If we can use redirect solution without breaking access to AMO, it's OK.
Then the remaining blocker is whitelisting AMJ. We'd also like to enable users to wear our own featured Personas without seeing security alert. (We'll embed iframes on mozilla.jp and host the content on addons.mozilla.jp, like http://www.mozilla.com/en-US/firefox/customize/)

Comment 63

[matthew zeier [:mrz]]

Do you even need to do that if AMO loads < 5 seconds for you in .jp?  Or are you solving a different problem?

Comment 64

[Kohei Yoshino [:kohei]]

The slow connection is one of the reasons why we need AMJ, as we have described in this bug and wiki. Especially the search results of AMO are too bad.

In terms of the Add-on Discovery pane:

* I understand we can manage the featured add-ons, as Justin said, but we cannot localize the linked page. As I have written many times, many localized add-ons don't have their localized description.

* Unfortunately the popular add-on list is unusable for Japanese users. Currently it shows Adblock Plus, DownThemAll!, Xmarks Sync, Video DownloadHelper and Download Statusbar, but three of those are not localized into Japanese. We have our own:
https://addons.mozilla.jp/firefox/extensions/?sort=downloads

* Last but not least, we'll also organize some marketing campaigns, promotions, etc. both on the in-product pane and the out-product site.

Comment 65

[matthew zeier [:mrz]]

(In reply to comment #64)
> The slow connection is one of the reasons why we need AMJ, as we have described
> in this bug and wiki. Especially the search results of AMO are too bad.

This can be solved with local proxy caches in Japan or Singapore.  That will significantly reduce page load times.

Comment 66

[Justin Scott [:fligtar]]

Okay, so for the Discovery Pane issues:

* The add-ons that appear in the pane are entirely predictable (featured / popular) so it's easy for us to offer Japanese name and description translations to the authors. I will also check with our Legal team to see if we could even do parts of this without needing the author's permission first.

* If I had known about this bug prior to yesterday we could have fixed the popular sort to be locale-specific, but given the short time frame, I suggest we just hard-code the most popular add-ons for Japan until we can solve that for real.

* The promotion module is already locale-specific, so it's easy for us to drop in HTML/CSS for Japan-specific promotions.

* As Matthew said, we should be able to greatly improve the page load time.

Does this address all the issues with the Discovery Pane? If so, next steps would be choosing the add-ons you'd want featured in the Japanese locale and giving us translations so we can start reaching out to the developers.

Based on bug 631147, it seems like the Japanese custom version of this page referenced earlier in the bug isn't built yet, so this is the perfect opportunity to make some progress towards a better AMO experience for Japan.

Comment 67

[matthew zeier [:mrz]]

> * As Matthew said, we should be able to greatly improve the page load time.

To that end, you can test addons.mozilla.org by adding the following to your hosts file:


72.26.221.77 addons.mozilla.org

Would be really interested in page load times.  Not that since no one's using this, the cache is empty.  I just hit the front page and loaded 59 objects with will expire in 3 hrs.  You should browse around to make the cache is warmed.

Comment 68

[Kohei Yoshino [:kohei]]

We're already working on Bug 631147. We don't always use Bugzilla for AMJ; it just a placeholder.

I absolutely agree that we could provide "a better AMO experience" for Japanese users, in the long term, but that is another bug. I know it's difficult to resolve the untranslated descriptions and the search quality issue.

> Created attachment 504427 [details] [diff] [review]
> patch for trunk (Firefox 4)

If the patch nor redirect won't be accepted, we'll just create an extension including the patch and seed it to our uses. We have already done "whitelisting AMJ" in our Add-on Pack for Students:
http://mozilla.jp/firefox/students/addons/

It's not a preferred solution, sure, but it's better for average Japanese users' experience. (Advanced users have en-US builds and are able to find/use unlocalized add-ons, from our perspective.)

Comment 69

[Kohei Yoshino [:kohei]]

Attachment: patch for trunk (Firefox 4) v2

(In reply to comment #49)
> >+pref("browser.dictionaries.download.url",           "https://addons.mozilla.jp/firefox/dictionaries/");
> >+pref("browser.search.searchEnginesURL",             "https://addons.mozilla.jp/firefox/search-engines/");
> >+pref("extensions.getAddons.get.url",                "https://addons.mozilla.jp/firefox/api/%API_VERSION%/search/guid:%IDS%");
> 
> You've not included some of the parameters that we send to AMO now, perhaps you
> just don't want/need them. Also unless AMJ is included in the same log
> gathering stuff that AMO is doing based on these pings then we won't be seeing
> the data from japanese builds, that seems unfortunate.

I have checked the latest pref value. We probably won't use those params but updated the patch along with it. 

> >+pref("extensions.getAddons.search.browseURL",       "https://addons.mozilla.jp/firefox/search?q=%TERMS%");
> >+pref("extensions.getAddons.search.url",             "https://addons.mozilla.jp/firefox/api/%API_VERSION%/search/%TERMS%/all/10/%OS%/%VERSION%");
> 
> You need to replace the 10 with %MAX_RESULTS% here. I'm assuming you don't want
> the other parameters we normally pass here.

Fixed.

> >+pref("extensions.getMoreThemesURL",                 "https://addons.mozilla.jp/firefox/themes/");
> >+pref("extensions.webservice.discoverURL",           "https://addons.mozilla.jp/firefox/discovery/%VERSION%/%OS%");
> >+pref("xpinstall.whitelist.add.36",                  "getpersonas.com,addons.mozilla.jp");
> 
> Instead please define a new pref xpinstall.whitelist.add.jp-40 with just
> "addons.mozilla.jp"

We'd like to whitelist AMJ also on Firefox 3.6 as I have attached before. A new pref requires us to make changes to the Firefox core code. I thought it was unacceptable to reviewers at this time, for one locale. It's tricky but a practical solution.

http://mxr.mozilla.org/mozilla-central/ident?i=XPINSTALL_WHITELIST_ADD_36

Comment 70

[matthew zeier [:mrz]]

You're moving forward with your own instance of AMO even with the proxy caching servers my group's getting setup?

Is infrasec even aware of the need to do penetration testing and security testing on addons.mozilla.jp>?

Comment 71

[Kohei Yoshino [:kohei]]

AMJ has launched in 2006 (see the wiki page) and *was* the default add-on library for Japanese users from 2008 (Bug 427323) to early 2010 (Comment 33), until someone broke AMO's redirects. It's nothing new. We'll continue to operate AMJ, serving both listing pages and APIs.

One new thing here is whitelisting AMJ.

Currently AMJ only provides browse and search. No user registration. No developer tools. No admin pages. Even no database.

We'll ask your security team or someone else for security testing if we'll add more complex features.

Comment 72

[Kohei Yoshino [:kohei]]

Off topic on security: why Firefox add-ons are not digital-signed? Some big companies like Google sign theirs but even Mozilla Labs doesn't sign.

Singing an add-on is pretty easy; here's my document:
https://developer.mozilla.org/en/Signing_an_extension

Add-ons are hosted on releases.mozilla.org (no SSL), so it can be potentially tampered with MITM attack, right?

I think extensions for Safari and Chrome are all signed.

Comment 73

[Kohei Yoshino [:kohei]]

(In reply to comment #72)
Ah I forgot the hash check mechanism of Firefox. But I believe Mozilla Labs should sign theirs anyway, and you should describe why most Firefox add-ons are not signed.

Comment 74

[Wil Clouser [:clouserw]]

Signing is bug 582347.  We're 73 comments into this one, let's keep it focused.

Comment 75

[Axel Hecht [:Pike]]

Comment on attachment 509391 [details] [diff] [review]
patch for trunk (Firefox 4) v2

Looking at http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/extensions/XPIProvider.jsm#2444, the whitelist pref is technically wrong.

I don't think that whitelisting sites for just some locales is a good idea in principle. Either they're good, or they're not.

Forwarding the feedback request to fligtar on the individual AMJ urls.

Comment 76

[Dave Townsend [:mossop]]

Comment on attachment 509391 [details] [diff] [review]
patch for trunk (Firefox 4) v2

>diff -u a/firefox-l10n.js b/firefox-l10n.js
>--- a/firefox-l10n.js	2011-01-17 21:17:55.000000000 +0900
>+++ b/firefox-l10n.js	2011-02-03 21:15:49.000000000 +0900
>@@ -44,3 +44,13 @@
> pref("browser.search.order.6", "chrome://browser-region/locale/region.properties");
> pref("browser.search.order.7", "chrome://browser-region/locale/region.properties");
> pref("browser.search.order.8", "chrome://browser-region/locale/region.properties");
>+
>+// Add-on Preferences
>+pref("browser.dictionaries.download.url",           "https://addons.mozilla.jp/firefox/dictionaries/");
>+pref("browser.search.searchEnginesURL",             "https://addons.mozilla.jp/firefox/search-engines/");
>+pref("extensions.getAddons.get.url",                "https://addons.mozilla.jp/firefox/api/%API_VERSION%/search/guid:%IDS%?src=firefox&appOS=%OS%&appVersion=%VERSION%&tMain=%TIME_MAIN%&tFirstPaint=%TIME_FIRST_PAINT%&tSessionRestored=%TIME_SESSION_RESTORED%");
>+pref("extensions.getAddons.search.browseURL",       "https://addons.mozilla.jp/firefox/search?q=%TERMS%");
>+pref("extensions.getAddons.search.url",             "https://addons.mozilla.jp/firefox/api/%API_VERSION%/search/%TERMS%/all/%MAX_RESULTS%/%OS%/%VERSION%?src=firefox");
>+pref("extensions.getMoreThemesURL",                 "https://addons.mozilla.jp/firefox/getpersonas");
>+pref("extensions.webservice.discoverURL",           "https://addons.mozilla.jp/firefox/discovery/%VERSION%/%OS%");
>+pref("xpinstall.whitelist.add.36",                  "getpersonas.com,addons.mozilla.jp");

The patch is advertised as for Firefox 4.0 so that's what I'm assuming it will be applied against, in which case you will need to make the change I suggested before. If you want this to work on 3.6 you'd unfortunately also have to make code changes or the whitelist changes wouldn't get applied to existing users's profiles.

Comment 77

[Kohei Yoshino [:kohei]]

Attachment: patch for trunk (Firefox 4) v3

(In reply to comment #75)
> http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/extensions/XPIProvider.jsm#2444,
> the whitelist pref is technically wrong.

Thank you for the reference. So xpinstall.whitelist.add.* should work for us. Fixed the patch by following Dave's comment 49.

> I don't think that whitelisting sites for just some locales is a good idea in
> principle. Either they're good, or they're not.

Obviously, AMJ is only for Japanese users. If AMJ is a local community-driven site, it may not be eligible for whitelisting, but Mozilla Japan is an affiliate of the Mozilla Foundation -- MoFo has paid AMJ's running cost since 2006.

We've got some feedback from users: "Why AMJ is not whitelisted though it's an official site?" Does this make sense?

Without the whitelist, 
* users have to see the yellow alert bar and click Allow button each time they install an add-on.
* thus, users begin to ignore the alert when they see it on non-Mozilla sites, and become accustomed to installing unreviewed, self-hosted add-ons. _We've been concerned about that._
* users cannot preview our own featured Personas with mouseover.

If this request wouldn't be absolutely approved, unfortunately we'll seed the patch directly to users as a last resort (see Comment 68).

(In reply to comment #76)
> If you want this to work on 3.6 you'd unfortunately also have to make code
> changes or the whitelist changes wouldn't get applied to existing users's
> profiles.

So it may be difficult to fix this reliably in Firefox 3.6. We're fine if the whitelist would be included at least in the upcoming Firefox 4.

Comment 78

[Justin Scott [:fligtar]]

Comment on attachment 509695 [details] [diff] [review]
patch for trunk (Firefox 4) v3

>+// Add-on Preferences
>+pref("browser.dictionaries.download.url",           "https://addons.mozilla.jp/firefox/dictionaries/");
This just redirects to AMO's dictionaries page. Why is it needed?

>+pref("browser.search.searchEnginesURL",             "https://addons.mozilla.jp/firefox/search-engines/");
>+pref("extensions.getAddons.get.url",                "https://addons.mozilla.jp/firefox/api/%API_VERSION%/search/guid:%IDS%?src=firefox&appOS=%OS%&appVersion=%VERSION%&tMain=%TIME_MAIN%&tFirstPaint=%TIME_FIRST_PAINT%&tSessionRestored=%TIME_SESSION_RESTORED%");

I sent the same sample API request to AMO and AMJ:
https://services.addons.mozilla.org/en-US/firefox/api/1.5/search/guid:%7Bd10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d%7D,[e-mail]
https://addons.mozilla.jp/firefox/api/1.5/search/guid:%7Bd10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d%7D,[e-mail]

This is the diff: http://pastebin.mozilla.org/1021595

Almost none of the features that Firefox 4 needs in the API call are supported, including:
* multiple GUID parameters
* screenshots/previews in the supported format
* Contributions support (this is a feature of Firefox 4 we've been touting to add-on developers)
* review and rating information
* downloads
* extended author information

(and a number of fields not required by Firefox 4, so I won't mention them specifically)

>+pref("extensions.getAddons.search.browseURL",       "https://addons.mozilla.jp/firefox/search?q=%TERMS%");
>+pref("extensions.getAddons.search.url",             "https://addons.mozilla.jp/firefox/api/%API_VERSION%/search/%TERMS%/all/%MAX_RESULTS%/%OS%/%VERSION%?src=firefox");

Just like the other API request, this is missing fields used by Firefox 4:
https://addons.mozilla.jp/firefox/api/1.5/search/adblock/all/30/Darwin/4.0b10?src=firefox

>+pref("extensions.getMoreThemesURL",                 "https://addons.mozilla.jp/firefox/getpersonas");
This just goes to getpersonas.com/ja. Why is it needed?

>+pref("extensions.webservice.discoverURL",           "https://addons.mozilla.jp/firefox/discovery/%VERSION%/%OS%");
This redirects to the homepage. When do you expect this page to be available?

Comment 79

[Kohei Yoshino [:kohei]]

(In reply to comment #78)
> >+pref("browser.dictionaries.download.url",           "https://addons.mozilla.jp/firefox/dictionaries/");
> This just redirects to AMO's dictionaries page. Why is it needed?

The redirect is temporary.

> >+pref("extensions.getAddons.get.url",                "https://addons.mozilla.jp/firefox/api/%API_VERSION%/search/guid:%IDS%?src=firefox&appOS=%OS%&appVersion=%VERSION%&tMain=%TIME_MAIN%&tFirstPaint=%TIME_FIRST_PAINT%&tSessionRestored=%TIME_SESSION_RESTORED%");

> Almost none of the features that Firefox 4 needs in the API call are supported,

> Just like the other API request, this is missing fields used by Firefox 4:

We're using the AMO API version 1.2. Will be upgraded to 1.5 in a few days.

> >+pref("extensions.getMoreThemesURL",                 "https://addons.mozilla.jp/firefox/getpersonas");
> This just goes to getpersonas.com/ja. Why is it needed?

The redirect is temporary. We'll have a landing page to have our own featured Personas.

> >+pref("extensions.webservice.discoverURL",           "https://addons.mozilla.jp/firefox/discovery/%VERSION%/%OS%");
> This redirects to the homepage. When do you expect this page to be available?

We have a copy of the homepage for now. Will be designed before the next beta release.

Comment 80

[Justin Scott [:fligtar]]

I recommend this bug be WONTFIXed for the following reasons:

* It's been made clear that the sole purpose of this bug is to make AMJ the official add-ons service in Japan, *not* to actually solve the Firefox 4 experience issue of Japanese users. comment #66 tries to address the experience issue in the Firefox 4 timeframe and is dismissed with a threat of what will happen if their request is not granted.

* As predicted in comment #52 and shown in comment #78, this change will remove Firefox 4 features in the Add-ons Manager available to all other locales and create extra work for Firefox QA, Infra Security, the Add-ons Manager team, and the Add-ons team to deal with this special case now and in future releases. It will also significantly complicate future efforts to more closely integrate AMO with Identity, Sync, and other services.

* The proposed service is not controlled or audited by MoCo IT or Infra Security.

* Firefox's interactions with the add-ons service are designed for and with AMO in mind, and the metrics and monitoring that feeds back into the product comes from AMO. The patch currently proposed does not collect start-up performance data with the other locales' data. To fix this and future issues will just require more and more effort and going out of the way to maintain a separate site rather than improve the main site.

* If this change is made, it will cement the Japan-specific add-ons site and encourage other locales to build their own sites instead of helping to improve AMO for non-English locales, which creates even more of the problems above.

Since the Japan team has made it clear they aren't interested in fixing the Discovery Pane issues and would rather seed a workaround add-on to their users than actually solve the problem at the source, I think another bug should be filed to determine what we can do so that users aren't the ones who have to pay for it.

Comment 81

[Kohei Yoshino [:kohei]]

This bug is a regression and NOT especially for the Add-on Manager in Firefox 4. What are you reading? Please don't get confused.

Such a big request is not allowed usually in a minor release, so the Mozilla Japan team have officially asked to make this a 4.0 (originally 3.6) blocker.

We are happy to improve the en-US-centric AMO site for users abroad (other local communities could also help your team) but I have already said it's a long-term goal. You cannot fix AMO's slow connection, unlocalized description, and bad search quality issues prior to the 4.0 release anyway.

AMJ is not the best, but better than AMO for Japanese users, at this time. We do know that our patch is just a workaround, not a persistent solution. If we could see the improved AMO by Firefox 4.1 (or 5.0, 6.0, 7.0...), we would remove the patch and completely shutdown AMJ.

Comment 82

[matthew zeier [:mrz]]

(In reply to comment #81)
> You cannot fix AMO's slow connection

I disagree.  I can fix it and spun off bug 631054 to work on that.

Have you performance tested with the proxies in Singapore?  Can I ship you proxy hardware to host in Japan?

I'd also throw support behind Justin's third bullet points - we haven't done any security review of this site.  Why is that okay?

How do metrics for AMJ get incorporated into the rest of the metrics work?

Comment 83

[Kohei Yoshino [:kohei]]

(In reply to comment #82)
> (In reply to comment #81)
> > You cannot fix AMO's slow connection
> 
> I disagree.  I can fix it and spun off bug 631054 to work on that.

Thanks for your work. Faster AMO is good for everyone. But we have a plan to move the AMJ server to a much faster network sometime soon. (Comment 59)

> Have you performance tested with the proxies in Singapore?  

Both IPs are not reachable.

> Can I ship you proxy hardware to host in Japan?

Maybe, but not now.

> I'd also throw support behind Justin's third bullet points - we haven't done
> any security review of this site.  Why is that okay?

Why now? AMJ has been running since 2006. Our main site http://mozilla.jp/ has also been running since 2004, separated from www.mozilla.com.

If security reviews from U.S. are required, we'll check with our legal counsel whether it's legal or not, according to the Japanese laws.

You cannot SQL injection, XSS, nor CSRF, at the very least. (Comment 71)

EV SSL is coming. (Comment 53)

> How do metrics for AMJ get incorporated into the rest of the metrics work?

If we could have full access the metrics, we can transfer our logs.

Comment 84

[Kohei Yoshino [:kohei]]

And I repeat my Comment 71:

> AMJ has launched in 2006 (see the wiki page) and *was* the default add-on
> library for Japanese users from 2008 (Bug 427323) to early 2010 (Comment 33),
> until someone broke AMO's redirects. It's nothing new. We'll continue to
> operate AMJ, serving both listing pages and APIs.
> 
> One new thing here is whitelisting AMJ.

Again, AMJ is NOTHING NEW. Our original request (Bug 427323) was readily accepted. PLEASE RESTORE Japanese user experience to the FORMER STATE. Doesn't it make sense??

Comment 85

[Mike Beltzner [:beltzner, not reading bugmail]]

(In reply to comment #61)
> I'm trying to understand the exact hardblocker for Firefox 4 in this bug -- can
> someone state what that is? As I mentioned in comment #52, if it's regarding
> the featured add-ons API, that is not used in Firefox 4.

When I triaged the bug, it looked like without this fix users in Japan would not be able to use the Get Add-Ons Discovery pane localized appropriately. If as per comment 52 that is not the case, then this is no longer a hard blocker.

I'm not ready to WONTFIX it yet, because as also mentioned in comment 52, it feels like there's a broader discussion to be had between the AMO team and the Mozilla Japan team about the right way to go forward on this. My personal feeling is that just because we have a Mozilla Japan doesn't mean that they should run Japanese versions of all Mozilla services. That isn't globalization. The right solution is to work with our teams to ensure we're providing a fantastic experience to Mozilla users, worldwide.

Comment 86

[Kohei Yoshino [:kohei]]

This bug is a obvious regression due to a mistake of someone at AMO, and has worsened user experience for Japanese (flagged "jp-critical"), so it can't be WONTFIXed. Again, AMJ is NOT a new service.

(In reply to comment #85)

I have got involved in Mozilla for 10 years and I do know MoCo is an excellent team ever, but unfortunately I *never* thought MoCo people understood real globalization, with rare exceptions. (I don't blame particular individuals.)

Do you know what this mean?
http://www.mozilla.com/en-US/firefox/features/#universal-customization
"Go beyond translation and experience the web in a way that makes sense to you."

I think the current Mozilla projects tend to be Americanized, not globalized. MoCo's services have always been en-US-centric and cannot be fully localized. AMO is one example among many; Do you know why MDC has lost many localizers? Do you know why localizers on the l10n-web list are always angry? L10n should be considered while designing services, not at the "phase 5", or such services would lack marketability in various regions. Localizers are not handy volunteers who just translate new strings in a week.

Mozilla Japan services are not alternatives but workarounds or complementary. AMJ is not "yet another AMO" but just a collection of localized add-ons. We have never planned to build a full-featured add-ons library. If the AMO team has put our feedback (https://wiki.mozilla.org/Japan/AMJ) in mind and fixed those bugs, we won't have to run AMJ. Bug 285850 is now 6 years old. Can we manage the featured add-ons for a specific locale? We didn't know that.

Did you forget this to-do list?
https://docs.google.com/Doc?docid=0Ad7mAOXgEBZyZGRzNnZ3YjRfOHRyOXptNGdz

The Chrome extension gallery has a locale filter:
https://chrome.google.com/extensions?hl=ja&itemlang=ja

(Comment 53)
> The search results of タブ (tab) shows clearly the current reality:
> 
> https://addons.mozilla.jp/firefox/search?q=%E3%82%BF%E3%83%96 (110 results)
> https://addons.mozilla.org/ja/firefox/search/?q=%E3%82%BF%E3%83%96 (only 13
> results)

We don't think those 13 results can satisfy Japanese users' needs. AMJ is just a simple, man-powered locale filter that AMO has not implemented yet.

Comment 87

[Dave Townsend [:mossop]]

Comment on attachment 509695 [details] [diff] [review]
patch for trunk (Firefox 4) v3

Dropping review request until it's clear what the path forward is here.

Comment 88

[Makoto Kato [:m_kato]]

addon.mozilla.jp is retired.