709140 - [km] Search engine setup for Firefox for Khmer.

Status: RESOLVED FIXED

(Mozilla Localizations :: km / Khmer, defect)

Description

[Axel Hecht [:Pike]]

We want to ship Firefox with a good list of search engines for Khmer. 

We're going to work towards this in three phases. First, the localization team and Milos (Milos Dinic) will look at the market for the language and come up with a good set of search providers. Vannak, we'll need your input on this, the guidelines for making recommendations are on <https://wiki.mozilla.org/L10n:Firefox/Productization>. 

For en-US, we currently use this set, to which we can fall back in the case of lacking localized alternatives:
Google, Yahoo!, Bing, Amazon, eBay, Wikipedia

After we decided on a particular search engine, Milos will contact the owner and ask for permission, and for specifics like proper search flags. Vannak, we'll likely need your help to at least get the contact information.

Once we have a specification on what the searchengines for Khmer should look like, the localization team and Milos will work on an implementation on that. This should happen by the localization team creating and attaching patches for the desired changes, and requesting review from Milos. After a successful review, the localization team will land them.

Please don't commit any modifications in browser/searchplugins to the hg repository without a patch reviewed by Milos, Stas, Axel or someone else appointed by Milos. The data here is sensitive to users and our search partners, thus the extra care here.

The Khmer team will be driving this bug to completion.

Comment 1

[Milos Dinic [:Milos]]

Hello Vannak,

I've been exploring a bit, and unfortunately, I couldn't find many options for you to pick from, so it would be nice if you could shed some light on Cambodian search engines here.

For the general search engines, I found http://www.khmerseek.com/khmer and http://www.cambodia.org/search , and khmerseek is showing up more results in Khmer for me, while cambodia.org is somewhat leaning toward English results, even though I've been searching for a term in Khmer (ភាសាខ្មែរ). Please note that if neither of those is good in your opinion, we could always stick to defaults(see comment 0).

As for the e-commerce searches, I didn't find literally anything. I've stumbled upon one website that seemed promising, but domain hasn't been renewed in a while, so it's not a option.

As for the specific interest search, I'd recommend Wikipedia in Khmer ( http://km.wikipedia.org/wiki/%E1%9E%91%E1%9F%86%E1%9E%96%E1%9F%90%E1%9E%9A%E1%9E%8A%E1%9E%BE%E1%9E%98 ) as it's probably one of the websites everyone has heard of.

Now, those were my proposals. I'm looking forward to hearing from you.

Comment 2

[Vannak Eng]

I may recommend http://www.Google.com.kh in stead of those two website as they are using the custom Google search.
and wikipedia Khmer is a good one that we should have.
thanks!

Comment 3

[Milos Dinic [:Milos]]

(In reply to Vannak Eng from comment #2)
> I may recommend http://www.Google.com.kh in stead of those two website as
> they are using the custom Google search.
> and wikipedia Khmer is a good one that we should have.
> thanks!

Ok, thanks for the info. Do you have any suggestions on e-commerce search engines, like Amazon/eBay, or there aren't any?

Comment 4

[Axel Hecht [:Pike]]

(In reply to Vannak Eng from comment #2)
> I may recommend http://www.Google.com.kh in stead of those two website as
> they are using the custom Google search.

Does google.com successfully choose url and language if you go there with no google cookies and coming from cambodia with km in accept-languages?
Then we can just use the stock google plugin.

Comment 5

[Milos Dinic [:Milos]]

(In reply to Axel Hecht [:Pike] from comment #4)
> (In reply to Vannak Eng from comment #2)
> > I may recommend http://www.Google.com.kh in stead of those two website as
> > they are using the custom Google search.
> 
> Does google.com successfully choose url and language if you go there with no
> google cookies and coming from cambodia with km in accept-languages?
> Then we can just use the stock google plugin.

Yes, I tested it already. Google.com shows google in Khmer with km in accept-lang.

Comment 6

[Vannak Eng]

for e-commerce search I dont think I have it.

Comment 7

[Milos Dinic [:Milos]]

Ok, then I'd say we should go with en-US defaults for general and e-commerce search, while adding wikipedia in Khmer in special interest. Do you agree with that?

Also, if you find Twitter somewhat popular in Cambodia, we can add it to specific interest search engines, too.

If you agree with twitter, list of search engines for Khmer would be:

google
yahoo
amazon
ebay
wikipedia-km
twitter

What do you think about it?

Comment 8

[Axel Hecht [:Pike]]

Can you do a quick check on whether the .co.uk versions of amazon or ebay would be more favorable than the US .com versions?

Comment 9

[Vannak Eng]

Milos I agree with that.

Comment 10

[Milos Dinic [:Milos]]

Vannak,

I have just tried to check whether .co.uk or .com is more favorable for Cambodian people for Amazon and eBay, but couldn't find any usable info. So, which one do you think will better suit your fellow Firefox users, .co.uk(British oriented) or .com(global, USA oriented) website?

One more thing:

I have tested google.com with `km` in intl.accept_languages, and I got google.com in Khmer. But. just to be safe, would you please start a new profile in Firefox(http://support.mozilla.com/en-US/kb/Managing-profiles#w_creating-a-profile), type about:config in url bar, press enter, click "I'll be careful", search for intl.accept, and you'll see intl.accept_languages in results list. Double click intl.accept_languages, and add `km` before anything that may be in there, so that it looks something like "km, en, en-US".

After that, open new tab, go to google.com. Google.com should, at this point, appear in your native language, Khmer.

If that's so, and with your opinion on the question above, we can proceed with implementation.

Thanks!

Comment 11

[Vannak Eng]

I follow what you told and I redirect www.google.com.kh when I type www.google.com.

For eBay and Amazon .com is our favorable.

Comment 12

[Milos Dinic [:Milos]]

OK, very good.

Note that I have
So we have the list of search engines:

google
yahoo
amazon
ebay
wikipedia-km
twitter


Google, yahoo, amazon, ebay and twitter plugins will be the default ones, as per comment 11, google is automagically redirecting you to www.google.com.kh, and other services aren't available in your language. This means that we just need to have them in list.txt, and those are going to automatically be picked up from en-US.

http://hg.mozilla.org/releases/l10n/mozilla-aurora/km/file/tip/browser/searchplugins/list.txt

Only change we need there is changing `wikipedia` to `wikipedia-kh`, and after that creation of wikipedia-kh.xml plugin, which should reside in the same folder as list.txt file.

Now, as for creating a plugin:

You can see more info on creating OpenSearch plugins at https://developer.mozilla.org/en/Creating_OpenSearch_plugins_for_Firefox . Note that we're using pretty much the same syntax for Firefox search plugins, and you can see one example at http://hg.mozilla.org/releases/l10n/mozilla-aurora/bs/file/tip/browser/searchplugins/wikipedia-bs.xml .

So, to round it all up: please create wikipeadia-kh.xml search plugin, change list.txt so that it contains new search plugin name without extension(ie. wikipedia-kh), create a patch, attach it to this bug, and flag me for review(set `r` flag to `?` and add `[e-mail]` besides).

Once you get a positive review, you can go forward by pushing the changes, along with appropriate description - adding r=milos and mentioning this bug.

Thanks!

Comment 13

[Vannak Eng]

Attachment: Wikipedia Km Opensearch

Wikipedia Km opensearch. Please help to review it.

Comment 14

[Milos Dinic [:Milos]]

Comment on attachment 582203 [details]
Wikipedia Km Opensearch

Pretty good, though a few things that we'd need to change here:

1. There are no changes to list.txt which is needed for this to work
2. Please add data representation of image instead of remote URL, because I'm not sure what are implications of using remote url for icons. You can use http://software.hixie.ch/utilities/cgi/data/data to get base64 encoded data representation of ico/png image.

, hence r-.

Also, when attaching a patch, it can be useful to check 'patch' checkbox, so that bugzilla knows it's a patch, which will enable some functionality for us when reviewing.

Other than those, plugin works fine on a new profile, so, when you implement suggestions above, we should be good to go.

Comment 15

[Vannak Eng]

Attachment: patch for wikipedia km

Comment 16

[Vannak Eng]

Attachment: patch for wikipedia km

Comment 17

[Milos Dinic [:Milos]]

Vannak, do you need any help creating a patch? If so, please do let me know, so that I can help.

Comment 18

[Vannak Eng]

Yes I need your help.

Comment 19

[arky [:arky]]

Comment on attachment 582484 [details] [diff] [review]
patch for wikipedia km

Please ignore this if you already reviewed this patch.

Comment 20

[arky [:arky]]

Milos, Please help Vannak out with this. TIA

Comment 21

[Vannak Eng]

that is the correct patch or do i need to submit a new patch?

Comment 22

[arky [:arky]]

Vannak you need to do this. 

Clone a repo (you already have that) 
Make the changes 
Generate a diff (hg diff > foo.patch) 

Attach the patch to this bug so that we can apply it.

Comment 23

[Milos Dinic [:Milos]]

Vannak,

If you're on Windows, and using a GUI hg tool, like TortoiseHG, save a diff to a file, and then attach it here, the same way you attached previous.

Comment 24

[Vannak Eng]

i just installed mozilla-build so i open the start-l10n.bat and cd to km then using hg diff > km.patch is it correct?

Comment 25

[Vannak Eng]

Attachment: patch for search plugin

Comment 26

[Milos Dinic [:Milos]]

Comment on attachment 593830 [details] [diff] [review]
patch for search plugin

Review of attachment 593830 [details] [diff] [review]:
-----------------------------------------------------------------

So,

This is a valid patch, but r- for the following reasons:

Narro seems to be adding files for search engines in list.txt if there aren't any .xml files with the same name in /browser/searchplugins/ directory. Anthow, that's a wrong way to do stuff, as we only need files for the search engines we don't already have in en-US. So, what you want to do is, create search plugin in /browser/searchplugins/ named wikipedia-km.xml and change the `wikipedia` to `wikipedia-km` in list.txt file

Also, I don't understand wht wikipedia-hy.xml is here, at all.

Comment 27

[Vannak Eng]

Attachment: New patch for searchplugins khmer

Comment 28

[Milos Dinic [:Milos]]

Comment on attachment 594092 [details] [diff] [review]
New patch for searchplugins khmer

Review of attachment 594092 [details] [diff] [review]:
-----------------------------------------------------------------

Vannak,

As I said, Narro is adding all these .xml files in /browser/searchplugins/ and we don't want them in there. So, what you'd want to do now is:

1) go to your clone, and type: hg revert /browser/searchplugins/
2) cd /browser/searchplugins/
3) remove all the files except list.txt and wikipedia-km.xml(if you have it)
4) if you don't have it already, create wikipedia-km.xml
5) type: hg add /browser/searchplugins/wikipedia-km.xml
6) change the list.txt file so that you change `wikipedia` to `wikipedia-km`

After that, create a diff using the same command again, and attach it here.

Comment 29

[Vannak Eng]

Attachment: patch for search plugin

Comment 30

[arky [:arky]]

The last diff looks good. Please consider marking this as FIXED.

Comment 31

[Vannak Eng]

Please review this bug.

Comment 32

[Axel Hecht [:Pike]]

I filed a doc request in bug 724487 to describe the life-cycle of bugs like this, but in short:

- This needs Milos' review
- If the review succeeds, it needs landing.

Comment 33

[Milos Dinic [:Milos]]

Comment on attachment 594099 [details] [diff] [review]
patch for search plugin

Review of attachment 594099 [details] [diff] [review]:
-----------------------------------------------------------------

I checked this plugin in new profile, works well, blank for search terms shows the wikipedia search page, search for some words I found on wikipedia works.

Only one nit: please change `Wikipedia(Km)` to `Wikipedia(km)` after getting r+ from Axel and before you commit.

Looks good, r+. Asking Axel for additional review.

Comment 34

[Axel Hecht [:Pike]]

Comment on attachment 594099 [details] [diff] [review]
patch for search plugin

r- for two technical nits:

The ShortName should match the title attribute of the search <link> on km.wikipedia.org. I'll try to paste that, but given that I don't have  km font, that might come across as garbage: វិគីភីឌា (km)

Also, both files have windows line endings, but should have unix line endings. That should be an option in your editor, hopefully, so change that, and save again. A good indication is when the patch to list.txt doesn't look like it's rewriting the whole file no more, but only shows the change to wikipedia, plus a few lines of unchanged context.

Comment 35

[Axel Hecht [:Pike]]

PS: The broken searchplugin already landed without review, sadly.

Comment 36

[Vannak Eng]

hi Axel,
I'm using the dreamweaver/notepad to edit this file. So please advice me with editor i should use.

Comment 37

[Axel Hecht [:Pike]]

I have little knowledge on windows editors, Pascal does recommend geany, though. And that has preferences at http://www.geany.org/manual/current/index.html#files-preferences to set line endings etc.

Comment 38

[Vannak Eng]

Attachment: these two files have unix line ending.

Comment 39

[Axel Hecht [:Pike]]

Comment on attachment 597636 [details] [diff] [review]
these two files have unix line ending.

Thanks, this looks good.

There are some best-practices for landing patches like this one:

Land just these changed files in the changeset. Use a commit message that references the bug, describes the change, and reviews.

In this example, that'd be

hg commit -m"bug 709140, change wikipedia to km, r=Pike, Milos" browser/searchplugins/list.txt browser/searchplugins/wikipedia-km.xml

Comment 40

[Milos Dinic [:Milos]]

Once landed, please close this bug and paste the revision number here. If you can't close it, just comment here, and we'll do it for you.

Thanks Vannak!

Comment 41

[Vannak Eng]

hi Milos I don't know how to close this bug. ;) please help me.

Comment 42

[Milos Dinic [:Milos]]

Will do. Note to self:

http://hg.mozilla.org/releases/l10n/mozilla-aurora/km/rev/775bed2b2d3f

Thanks, Vannak!

Comment 43

[Axel Hecht [:Pike]]

Just updated Vannak's bugzilla priviledges, you should now be able to set the "Status" field beneath the comment area to RESOLVED, and leave the resolution to the FIXED, that shows up then. This bug is already dealt with, though.

Comment 44

[Vannak Eng]

thank Milos