989964 - [proposal][feature] Duckduckgo as a search provider on Homescreen search/Everything.me

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::Search, defect)

Description

[C. sanchiavedraZ]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0 (Beta/Release)
Build ID: 20140218140359

Steps to reproduce:

[proposal][feature]
Add a fixed or optional shortcut to search on Duckduckgo.

Comment 1

[C. sanchiavedraZ]

For the time being, Google shortcut is always there, but only sometimes Duckduckgo appears as an option.
--
B2G 1.1.1.0hd-GP

Comment 2

[Kevin Grandon :kgrandon]

I think the proper fix is allowing duck duck go to be a default search provider, here: https://github.com/mozilla-b2g/gaia/blob/master/apps/settings/resources/search/providers.json

Peter - can you let us know if there are any concerns from a product point-of-view for adding duck duck go to our default search provider list?

Comment 3

[Abhiram Chintangal [:maitreya]]

Thanks! 
I tried adding DuckDuckGo entry in the providers.json, but I don't see it in the search-engines list. Here are the changes I made (http://pastebin.com/FwVaB6Jf). To update the changes the did a make install-gaia but it didn't work.

(In reply to Kevin Grandon :kgrandon from comment #2)
> I think the proper fix is allowing duck duck go to be a default search
> provider, here:
> https://github.com/mozilla-b2g/gaia/blob/master/apps/settings/resources/
> search/providers.json
> 
> Peter - can you let us know if there are any concerns from a product
> point-of-view for adding duck duck go to our default search provider list?

Comment 4

[Joshua Smith [:joshua-s]]

Assigning to :maitreya!

Comment 5

[Kevin Grandon :kgrandon]

(In reply to Abhiram Chintangal [:maitreya] from comment #3)
> Thanks! 
> I tried adding DuckDuckGo entry in the providers.json, but I don't see it in
> the search-engines list. Here are the changes I made
> (http://pastebin.com/FwVaB6Jf). To update the changes the did a make
> install-gaia but it didn't work.

Hmm, I'd need to look into it, there's also two JSON files, not sure which one (or if both), need to be modified. You also might need to perform a reset-gaia (instead of install-gaia). Ni? On benfrancis to see if we're on the right track.

Comment 6

[Peter Dolanjski [:pdol]]

(In reply to Kevin Grandon :kgrandon from comment #2)
> Peter - can you let us know if there are any concerns from a product
> point-of-view for adding duck duck go to our default search provider list?

Let me look into it.  Keeping NI on me to come back with a reply.

Comment 7

[Ben Francis [:benfrancis]]

Adding it to providers.json should be enough, but you will need to do "make reset-gaia" instead of "make install-gaia" to see it appear in the list of providers in settings, as Kevin suggested.

This setting can get overwritten on a per-SIM basis (e.g. https://github.com/mozilla-b2g/gaia/blob/master/customization/mobizilla/mobizilla_search.json) but that doesn't happen unless you specify GAIA_DISTRIBUTION_DIR at build time.

Also note that you probably want your URL template to be "https://duckduckgo.com/?q={searchTerms}"

I'm not sure whether the DuckDuckGo's proxy to Google suggestions at ddgg.nfriedly.com actually works (DuckDuckGo don't provide suggestions themselves), but we don't support suggestions yet anyway.

Comment 8

[Abhiram Chintangal [:maitreya]]

(In reply to Ben Francis [:benfrancis] from comment #7)
> Adding it to providers.json should be enough, but you will need to do "make
> reset-gaia" instead of "make install-gaia" to see it appear in the list of
> providers in settings, as Kevin suggested.
> 
> This setting can get overwritten on a per-SIM basis (e.g.
> https://github.com/mozilla-b2g/gaia/blob/master/customization/mobizilla/
> mobizilla_search.json) but that doesn't happen unless you specify
> GAIA_DISTRIBUTION_DIR at build time.
> 
> Also note that you probably want your URL template to be
> "https://duckduckgo.com/?q={searchTerms}"
> 
> I'm not sure whether the DuckDuckGo's proxy to Google suggestions at
> ddgg.nfriedly.com actually works (DuckDuckGo don't provide suggestions
> themselves), but we don't support suggestions yet anyway.

thank you, I reply to your suggestions in order:

With regard to installing modified gaia on the device, I tried both make install-gaia/reset-gaia without
any success. Btw, my test device is a tablet and here are the make-options I use:
 
DEVICE_DEBUG=1 GAIA_DEVICE_TYPE=tablet NOFTU=1 make reset-gaia/install-gaia

Maybe, it is whats causing the settings to be overwritten? 
  
And thanks for pointing out the typo in the url-template, fixed it. As for the suggestions url, I just tested it and it doesn't work. I guess I won't worry about it now as it yet to implemented.

Comment 9

[Abhiram Chintangal [:maitreya]]

I got to work! I was confused between the search engine settings of the browser and the searchbar (Completely different things)
Should I file a bug for adding DuckDuckGo to the browser search engine options as well?

Comment 10

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

The browser is controlled via browser.json : 
https://github.com/mozilla-b2g/gaia/blob/master/customization/browser.json

If you add the search engine in there, you should be able to use duckduckgo for the browser.

Comment 11

[Abhiram Chintangal [:maitreya]]

(In reply to Naoki Hirata :nhirata (please use needinfo instead of cc) from comment #10)
> The browser is controlled via browser.json : 
> https://github.com/mozilla-b2g/gaia/blob/master/customization/browser.json
> 
> If you add the search engine in there, you should be able to use duckduckgo
> for the browser.

Thanks for the help. I guess, I will file a bug for the search engine fix.

Comment 12

[Abhiram Chintangal [:maitreya]]

Attachment: patch for the bug

Comment 13

[Théo Chevalier]

Looks like we use .ico files with 16x16 and 32x32 images for other providers, I think we should stick to it.
It also have the advantage of taking less memory (~5KB vs. 16KB)

Comment 14

[Abhiram Chintangal [:maitreya]]

(In reply to Théo Chevalier [:tchevalier] from comment #13)
> Looks like we use .ico files with 16x16 and 32x32 images for other
> providers, I think we should stick to it.
> It also have the advantage of taking less memory (~5KB vs. 16KB)

thanks for pointing it out, I didn't even notice it. Anyways, I added a new 32x32 icon and pushed the changes.

Comment 15

[Ben Francis [:benfrancis]]

Comment on attachment 8457941 [details] [review]
patch for the bug

Clearing review pending input from Product, as per conversation in IRC. Please flag me for review again once we get that. Thank you!

Comment 16

[Matěj Cepl]

Isn't this effectively a duplicate of bug 1040865 (which actually has a pull request to it at https://github.com/mozilla-b2g/gaia/pull/23381)?

Comment 17

[Kevin Grandon :kgrandon]

(In reply to Matěj Cepl from comment #16)
> Isn't this effectively a duplicate of bug 1040865 (which actually has a pull
> request to it at https://github.com/mozilla-b2g/gaia/pull/23381)?

Unfortunately the patch in that bug is wrong, and this bug is older. Since we will probably go with the patch here, let's dupe against this one.

Comment 19

[randomenduser]

Happy to test this! I remember reading somewhere that they didn't have a 3rd party autocompletion URL but were working on it (can't find the URL at the moment). I think you could ask if you could use theirs (a little inspector magic can find this), but otherwise some of the info needed for this fix is in that PR (even if it's not all in the right spots).

Comment 20

[Peter Dolanjski [:pdol]]

All, I finally have the go ahead to add DuckDuckGo as a non-default search provider in the standard Firefox OS build.

Mike, I'm told you should review the url that is used to make sure it aligns with our other products.  Can you take a look at the patch?

Comment 21

[Kevin Grandon :kgrandon]

Comment on attachment 8457941 [details] [review]
patch for the bug

Awesome. Asking Ben and Arthur (Settings Peer) to review this as well.

Comment 22

[Arthur Chen [:arthurcc][inactive after 6/30]]

Comment on attachment 8457941 [details] [review]
patch for the bug

The patch needs rebasing. And I'm not sure whether we should also update `customization/search/providers.json`.

Comment 23

[Ben Francis [:benfrancis]]

Comment on attachment 8457941 [details] [review]
patch for the bug

Peter, do we have a contact at DuckDuckGo we can ask about which URL we should use for search suggestions, if any? As I understand it, DuckDuckGo do not provide their own search suggestions.

ddgg.nfriedly.com looks like it is owned by DreamHost but I get a certificate error when I navigate to that origin.

Comment 24

[Frederik Braun [:freddy]]

They seem offer suggestions according to <https://duckduckgo.com/opensearch.xml>

Comment 25

[Ben Francis [:benfrancis]]

(In reply to Frederik Braun [:freddyb] from comment #24)
> They seem offer suggestions according to
> <https://duckduckgo.com/opensearch.xml>

Awesome, well spotted! I swear that wasn't there last time I looked :)

Abhiram, would you like to rebase and change the suggestions URL template to https://ac.duckduckgo.com/ac/?q={searchTerms}&type=list

(In reply to Arthur Chen [:arthurcc] from comment #22)
> I'm not sure whether we should also update
> `customization/search/providers.json`.

Yes, the one in settings is just a default default! I think it will be overwritten by the one in the /customization/ directory if Gaia is built with GAIA_DISTRIBUTION_DIR=customization, although that is just an example. We should probably change both.

Abhiram, where did you get the "default" property from? I can't see that documented anywhere.

Comment 26

[Mike Connor [:mconnor]]

Confirmed that the agreement covers OS, will follow up for an appropriate code.

Comment 27

[Frederik Braun [:freddy]]

Attachment: Github Pull Request (add duckduckgo to search providers and customization defaults)

Taking this for the sake of progress :-)

Comment 28

[Mike Connor [:mconnor]]

"urlTemplate": "https://duckduckgo.com/?q={searchTerms}",

should be

"urlTemplate": "https://duckduckgo.com/?t=ffos&q={searchTerms}",

Unless partner tagging gets handled elsewhere?  I'm not sure how OS handles search params.

Comment 29

[Frederik Braun [:freddy]]

Do we need this for suggestions as well?

Comment 30

[Ben Francis [:benfrancis]]

Comment on attachment 8534305 [details] [review]
Github Pull Request (add duckduckgo to search providers and customization defaults)

r+ from me if we can confirm the correct branding for the search string and icon.

Where did "ffos" come from? The official abbreviation is "fxos".

Where did the icon come from? It looks different to https://duckduckgo.com/favicon.ico

Comment 31

[Mike Connor [:mconnor]]

ffos came from DDG. It is not needed for suggestions.

Comment 32

[Frederik Braun [:freddy]]

Their favicon is super crappy on the flame, I think I took one of the other icons ;)
I personally would prefer taking something that looks nice over something that is idealistic, but I know this is not my code, my module or my call to make.

Someone please advise, which icon to take and I'll be happy to rebase the pull request accordingly.

Comment 33

[Abhiram Chintangal [:maitreya]]

Ah sorry about not responding guys. Its finals week at school :(

Anyways glad you got the patch fixed.

Comment 34

[Lori]

I think we should use the same kind of icon as in Comment 30. It's more popular, so it should be more familiar to people. Either way, we can always change it once we have more info. Demand for this feature is pretty high (myself included). Below are some more DDG icons, but not many:

Red with Transparency PNG
https://github.com/duckduckgo/android/tree/master/res
:: Select any of the 'drawable-*' folders and then select 'icon.png'.

Red full PNGs
https://github.com/duckduckgo/ios/tree/master/DuckDuckGo/Images.xcassets/AppIcon.appiconset
:: Select any file that ends in '.png'

SVG with title included
https://duck.co/help/company/press
:: Select 'Logos' and then select the first option at the left.

Comment 35

[Frederik Braun [:freddy]]

My last comment (comment 32) should have contained an needinfo, sorry.

Ben: Which icon should I take?

Comment 36

[Ben Francis [:benfrancis]]

We should probably use the icon they specify in their opensearch.xml, which is actually a data URL in this case https://duckduckgo.com/opensearch.xml

I say this because ideally in future all search engines will be added this way, and this is something DuckDuckGo can update themselves if they change their logo.

It is only 16x16px but I don't think we even display it anywhere at the moment! (we used to in the browser app and we may in future in the search app)

Comment 37

[Ben Francis [:benfrancis]]

(note: this currently looks the same as their favicon)

Comment 38

[Mike Connor [:mconnor]]

I think, but I'm not sure, that the OpenSearch icon is actually multiple sizes.  If I recall correctly it's something like four sizes.  That's probably ideal (most of the favicons in Firefox Desktop plugins are actually 16 + 32 to accommodate high DPI screens).  I'd suggest thinking about the future use-cases when adding the icons.

Comment 39

[Frederik Braun [:freddy]]

I only see one data URI and will amend my pull request to use it.

Comment 40

[Frederik Braun [:freddy]]

Comment on attachment 8534305 [details] [review]
Github Pull Request (add duckduckgo to search providers and customization defaults)

Asking for another review, will squash commits after r+.
(I am a bit saddened by this ugly icon.)

Comment 41

[Ben Francis [:benfrancis]]

Comment on attachment 8534305 [details] [review]
Github Pull Request (add duckduckgo to search providers and customization defaults)

I think maybe you only updated one of the icons, but other than that r+ from me. Thank you.

I hope in future we will fully support OpenSearch and DuckDuckGo will be able to update their icon whenever they wish.

Comment 42

[Frederik Braun [:freddy]]

rebased. Asking for check-in once Gaia Try is through.
(Not expecting any additional fallout since this is just a rebase of a previous green run)

Comment 43

[Frederik Braun [:freddy]]

try was green (after some re-runs), as expected. Waiting for check-in.

Comment 44

[Ryan VanderMeulen [:RyanVM]]

Master: https://github.com/mozilla-b2g/gaia/commit/1069daf8abc7c2098771897d6793832d093f0651

Comment 45

[Théo Chevalier]

Looks like DuckDuckGo does not appear in Settings > Search in current master. Am I missing something?

Comment 46

[Dale Harvey (:daleharvey)]

The way the current configuration works means upgrades have no effect on the current selection, you would need to do a reset-gaia / wipe your current config to see ddg. This will be fixed by https://bugzilla.mozilla.org/show_bug.cgi?id=1119820

Comment 47

[Théo Chevalier]

Ah, thanks for the info Dale. I was guessing something like that