Clean-up search engines in TB: Add Duck Duck Go and Google to the default list of search engines, remove disfunctional AOL search

RESOLVED FIXED in Thunderbird 66.0

Status

enhancement
RESOLVED FIXED
a year ago
4 months ago

People

(Reporter: unicorn.consulting, Assigned: jorgk)

Tracking

Trunk
Thunderbird 66.0

Thunderbird Tracking Flags

(thunderbird_esr6065+ fixed, thunderbird65 fixed, thunderbird66 fixed)

Details

()

Attachments

(2 attachments, 5 obsolete attachments)

Reporter

Description

a year ago
No description provided.
Reporter

Comment 1

a year ago
Google was removed from the list of opensearch providers some years ago due to a contractual problem between Mozilla and Google.  I do not think we do our users any service by simply not having Google in the available search list.

With the rise of folk being concerned about tracking we should also include Duck Duck Go as one of the more popular non tracking search engines. This particular provider is included in Firefox,  but Not Thunderbird.
Severity: normal → enhancement
OS: Unspecified → All
Hardware: Unspecified → All
See Also: → 937809, 1104922
Summary: Add Duck → Add Duck Duck Go and Google to the default list of search engines.
Assignee

Comment 2

a year ago
Hmm, how would we do this? Google seems to be included in some locales, see:
https://dxr.mozilla.org/comm-central/source/mail/locales/search/list.json
So is it just a matter of adding google to "default" or "en-US"?

There are also four mentions of "duckduckgo" in that file, but don't we need an XML here?
https://dxr.mozilla.org/comm-central/source/mail/locales/en-US/searchplugins

I have zero idea, so please help!
Flags: needinfo?(mozilla)
Flags: needinfo?(florian)
I'm honestly confused by what happened here because none of the needed engines are present in the build. I think it must be picking them up elsewhere? I'll try to take a look.
Flags: needinfo?(mozilla)
Locale specific thunderbird engines are still stored in the locale directories:

https://dxr.mozilla.org/l10n-central/search?q=file%3A*.xml&redirect=false

So the files are referenced in list.json, but they are packaged via the locale files.

So if you wanted to add google and ddg for English, you should take the files from browser/locales/searchplugins, remove the codes and place them in

comm-central/mail/locales/en-US/searchplugins

Then update list.json (google is actually already there, so you just need to add it to list.json)

IF you want to put them in any locales other than English, they would need to go into the specific locale.

We should probably spin off a larger bug to unify the locales as we have for Firefox. It's made life a lot easier.
Flags: needinfo?(florian)
Reporter

Updated

a year ago
Duplicate of this bug: 1465543

Updated

9 months ago
Duplicate of this bug: 1490511

Updated

9 months ago
Duplicate of this bug: 1454259
Magnus is the only currently active developer for https://addons.thunderbird.net/en-US/thunderbird/addon/google-search-for-thunderbi/
Flags: needinfo?(mkmelin+mozilla)

Comment 9

8 months ago
Thus more reason to add them. Cannot rely on an incompatible add-on to get decent search in Thunderbird. At least please add DDG in a following release.
Assignee

Comment 10

6 months ago
The fix doesn't look too hard according to comment #4.
Assignee

Comment 11

6 months ago
Note: We're working on bug 1427317 that will allow to add search engines manually. Of course we should still clean-up the search engine area.
Summary: Add Duck Duck Go and Google to the default list of search engines. → Clean-up search engines in TB: Add Duck Duck Go and Google to the default list of search engines, remove disfunctional AOL search

Updated

6 months ago
Duplicate of this bug: 1511808
Assignee

Comment 13

5 months ago
Posted patch 1427133-search.patch (obsolete) — Splinter Review
After a while we've come back to this. What I've done here is this:
Copied browser/components/search/searchplugins to mail/components/search/searchplugins absolutely 1:1.

From mail/components/search/searchplugins I removed the defunct AOL search and also Yahoo which appears to have fallen out of favour. I don't really want to spend the time to fiddle Yahoo back into list.json.

Mike, does this sound right to you?

In a local build it all looks good: There's Google, DDG, etc.

We're a bit under pressure to fix search engines in TB, so I hope this can be backported to TB 60.
Assignee: nobody → jorgk
Status: NEW → ASSIGNED
Flags: needinfo?(mkmelin+mozilla)
Attachment #9033886 - Flags: review?(mozilla)
Assignee

Comment 14

5 months ago
Posted patch 1427133-search-esr60.patch (obsolete) — Splinter Review
OK, the mechanism in ESR 60 is different. At least we can add Google and remove the defunct AOL engine. That's better already.
Attachment #9033889 - Flags: review?(mozilla)
Assignee

Comment 15

5 months ago
Added Google to en-US as well.
Attachment #9033889 - Attachment is obsolete: true
Attachment #9033889 - Flags: review?(mozilla)
Attachment #9033890 - Flags: review?(mozilla)
Reporter

Comment 16

5 months ago
(In reply to Jorg K (GMT+1) (urgent reviews and bustage fix only, Dec 22nd to Jan 1st) from comment #15)
> Created attachment 9033890 [details] [diff] [review]
> 1427133-search-esr60.patch (v1b)
> 
> Added Google to en-US as well.

Thanks Jorg,  could you include duck duck go as well the two things that originally started this bug.
Assignee

Comment 17

5 months ago
Here also removing aol-web-search.xml. Is that a good idea or can locales still refer to it. Also adding ddg.xml from trunk and adding it to defaults and en-US/en-GB. Mike will tell us whether it's correct like this.
Attachment #9033912 - Flags: review?(mozilla)
(In reply to Jorg K (GMT+1) from comment #13)
> What I've done here is this:
> Copied browser/components/search/searchplugins to
> mail/components/search/searchplugins absolutely 1:1.

This means including in Thunderbird all the partner codes that Firefox uses. I don't think this will be OK, but someone who knows how the Firefox search deals work will have the final word on this. Hopefully mkaply can comment on this, or point you to someone who can.
Assignee

Comment 19

5 months ago
Well, Mike said in comment #4: "remove the codes". I didn't understand what he meant. Looking at a particular file:

+++ b/mail/components/search/searchplugins/amazon-en-GB.xml
@@ -0,0 +1,18 @@
+<!-- This Source Code Form is subject to the terms of the Mozilla Public
+   - License, v. 2.0. If a copy of the MPL was not distributed with this
+   - file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
+
+<SearchPlugin xmlns="http://www.mozilla.org/2006/browser/search/">
+<ShortName>Amazon.co.uk</ShortName>
+<Description>Amazon.co.uk Search</Description>
+<InputEncoding>UTF-8</InputEncoding>
+<Image width="16" height="16">resource://search-plugins/images/amazon.ico</Image>
+<Url type="text/html" method="GET" template="https://www.amazon.co.uk/exec/obidos/external-search/" resultdomain="amazon.co.uk">
+  <Param name="field-keywords" value="{searchTerms}"/>
+  <Param name="ie" value="{inputEncoding}"/>
+  <Param name="mode" value="blended"/>
+  <Param name="tag" value="firefox-uk-21"/>
+  <Param name="sourceid" value="Mozilla-search"/>
+</Url>
+<SearchForm>https://www.amazon.co.uk/</SearchForm>
+</SearchPlugin>

it's hard to tell where the "code" is. Maybe tag and sourceid. I can of course remove those whole-sale. The somewhat more pressing issue is the ESR60 patch which should be straight forward.
Comment on attachment 9033886 [details] [diff] [review]
1427133-search.patch

Review of attachment 9033886 [details] [diff] [review]:
-----------------------------------------------------------------

::: mail/components/search/searchplugins/amazondotcom.xml
@@ +14,2 @@
>    <Param name="mode" value="blended"/>
> +  <Param name="tag" value="mozilla-20"/>

<Param name="tag" in amazon plugins is definitely a partner code.

@@ +14,3 @@
>    <Param name="mode" value="blended"/>
> +  <Param name="tag" value="mozilla-20"/>
> +  <Param name="sourceid" value="Mozilla-search"/>

I'm not sure about the sourceid param. Looks like you didn't have it before in Thunderbird.

::: mail/components/search/searchplugins/baidu.xml
@@ +14,5 @@
> +  <Param name="action" value="opensearch"/>
> +</Url>
> +<Url type="text/html" method="GET" template="https://www.baidu.com/baidu" resultdomain="baidu.com">
> +  <Param name="wd" value="{searchTerms}"/>
> +  <Param name="tn" value="monline_dg"/>

The 'tn' parameter on baidu is a referral tag.

::: mail/components/search/searchplugins/bing.xml
@@ +16,2 @@
>          <Param name="q" value="{searchTerms}"/>
> +        <Param name="pc" value="MOZI"/>

This looks like a partner code.

You are removing <Param name="form" value="MOZESB"/> which was definitely a partner code. IIRC Thunderbird had a search deal with Bing 8 years ago. Does this deal still exist?

::: mail/components/search/searchplugins/ddg.xml
@@ +13,5 @@
> +    <MozParam name="t" condition="purpose" purpose="contextmenu" value="ffcm"/>
> +    <MozParam name="t" condition="purpose" purpose="keyword"     value="ffab"/>
> +    <MozParam name="t" condition="purpose" purpose="searchbar"   value="ffsb"/>
> +    <MozParam name="t" condition="purpose" purpose="homepage"    value="ffhp"/>
> +    <MozParam name="t" condition="purpose" purpose="newtab"      value="ffnt"/>

This whole block is partner codes.

::: mail/components/search/searchplugins/ebay.xml
@@ +8,5 @@
> +<Image width="16" height="16">resource://search-plugins/images/ebay.ico</Image>
> +<Url type="text/html" method="GET" template="https://rover.ebay.com/rover/1/711-53200-19255-0/1" resultdomain="ebay.com">
> +  <Param name="ff3" value="4"/>
> +  <Param name="toolid" value="20004"/>
> +  <Param name="campid" value="5338192028"/>

campid contain a compaign id used for affiliation tracking.

::: mail/components/search/searchplugins/ecosia.xml
@@ +11,5 @@
> +  <Param name="type" value="list"/>
> +  <Param name="q" value="{searchTerms}"/>
> +</Url>
> +<Url type="text/html" method="GET" template="https://www.ecosia.org/search">
> +  <Param name="tt" value="mzl"/>

I think this is a partner code.

::: mail/components/search/searchplugins/google-2018.xml
@@ +1,1 @@
> +<!-- This Source Code Form is subject to the terms of the Mozilla Public

I don't think you need more than one google search plugin in Thunderbird, and you definitely don't need the google-2018 one which is specifically used by Firefox only in the US.

::: mail/components/search/searchplugins/google.xml
@@ +8,5 @@
>  <InputEncoding>UTF-8</InputEncoding>
> +<Image width="16" height="16">resource://search-plugins/images/google.ico</Image>
> +<Url type="application/x-suggestions+json" method="GET" template="https://www.google.com/complete/search?client=firefox&amp;q={searchTerms}"/>
> +<Url type="text/html" method="GET" template="https://www.google.com/search" rel="searchform">
> +  <MozParam name="client" condition="purpose" purpose="keyword" value="firefox-b-ab"/>

The client parameter is the partner code on the Google plugins.

::: mail/components/search/searchplugins/twitter.xml
@@ +11,2 @@
>    <Param name="q" value="{searchTerms}"/>
> +  <Param name="partner" value="Firefox"/>

This looks a lot like a partner code ;).

::: mail/components/search/searchplugins/yahoo-jp-auctions.xml
@@ +9,5 @@
> +<Image width="16" height="16">data:image/x-icon;base64,AAABAAIAEBAAAAEAIAAEAwAAJgAAACAgAAABACAARgcAACoDAACJUE5HDQoaCgAAAA1JSERSAAAAEAAAABAIBgAAAB/z/2EAAALLSURBVDiNfZNbSFRRFIb/dc4Zz2gz3iK1oFToJSPCpECoqBAiCPPFy0PFoBUlEWRmiFDjg4VmkYHdCDQIIiswM30IL5g6mQ1qeQuzcVQ0YWq00XScc87qQSfHAVsv+2Hv/1v/umxgjWBmWj4N9vdvzSNNr079u6uqEtfSrQD6qgKWAVGtN3Pmuouz2d5ef2YlwX8gbDYLAAgA2ooyTC9SjTMvD4EtRWd5tK0+28elIPnbbm4uFOmgWQXAlpJMk8s2UGHQheF3WLB7vOWBrDGXjzTXIOZA8j0i0iRfMREBgAIAlpJM06xtoMLj+AJN0HuCZKM8p210OxsfyiERm8pdo/29xi1xLZKPmL2wjzdMpmnbQIXH8RmaYPCwuE7Hs985InKXrO1Iap+ZcVTOOZ2DAEDMLBCRxsyBYx0Nub9GhvZMvLufRK5veg+FKhACJF4YcRui9snGuMTa3eeL04ho3ptYgtUqAtAAbPg11HvN3VImTk/YPOtCYzUBisTqgiouQp4XAhGTePTLslhcdq0ISEhQABARjUbGJ2Zp29NVvQLdgkdVFbcDsjQvRqRcH3YFx2rdzx7l27ss1wCoRKQ0NTVJq5YGAJy2AVNn+VWu2gt+c3InfyjLa2Tm4LE+6/nagtNcffkEj/Z0FKyauxkQ6uruygB0APBzeDDTUnqJe4rPsWPQetz7bry/+2J1fha/zjvJtm7LlaUmAkQAwy+mej9lT9u/xgfNTV3YnJYz7wvpfFJ2O8RoQPTh1IsEAE+PhAe71EW9qs6q4bHJkv1xzWI+4PRbMpGIVAAY6mjOsXW23orYui2XAKB0f+AdEoQMMCYAZT2k8K5LDZMpwNLHobQ01R/CzNEAxqgmPSpbJJTrRUDRAFEAmBl/1IBbx56P5QKAb5nevfE6k0DkAqhyEfjBIB0zKxojHOyZ9O8LluatMTPBapUKa2vVv/Q3b+fAt5TJAAAAAElFTkSuQmCCiVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAHDUlEQVRYha2Xa3AT1xXH/2f3rixZfie2eQ9JKK+EJinm0RRcPCkm00cSIKb1wDTNtIlJO3TSyfRLv7BpM518aHAmHejkVTJtEicycXAInYSXFGqoS+WkmJcfWEBpjI0B+SFZ2t27e/pBspGNBAZ6ZjQare4957f/87i7wG0aM6vJb0/0Qtci49LZeSn/0Y32K7cZnIjIZmZPtK97+3DHF0fO/6vppNHfvRYAAoEX1Nvxf0MLBoMaALA5+BPuaOY6wPy0uoBDn9WxTEL4/bqYiBK3ZKzrCgAwc9bh2uc/2LO+hBsfR/xvVUhC9K4FAPb51EwQ4nYAAggoAJyT23/9QP/5Y0v7oyay3bNcVuy0bHujWgB1O2R/9xNUMOVDXwLCISL+vwD4dV1U6Lpsr/vt8kunOw5khc8LZrYHYwNqXvYsYcVOy7Y3qwXw/g451LtW5JY2pIO4pdwEX3tGK6t53Tr5rl5+JdS5L3LKrxnGgCRRICKmhHRs5GXng2JdUgiIeU+/jxkPVawVuaUNzD6VaJ094uumuyA1eDjUuS96yq8ZRkSSVigAB15NgVBUDA4Pgt33CCkh2974Ef5z2P8hD/WuIVpn+3y+0e64KQVY1wXpukwG3xtpC7ji8SFJWp4A21fXMRC1HEjHRr63EBzrtFRAm1dTh5nLVm0QOUXvMrNCRM6EFfCPDb4v0uZ3xYxBS3HlC7Acs5YI8GqAJrIRDnfaInu2JnsB5UIXzMjAWgBAfT0B1ynCZNuoaGmhlpYWlNXUWFeDB7S4MWSpWoHGjnWNkMwOVOGFOxaSntIyIYru6HffeRc7xZO/8JR6fwwAqKpycM3Oq8EVInJSr3X6fv/ti+0n9kRO+V1xIyIVLV8wXxsc7ICEFzISsnKL5mqllU9HJs2dW1H8je8FmU+4iO41kxOU0wIwszoyXu34wAbV7RnuOXJAnN617dX4uZN5w8MXJWkjso/fziDFDTtyVuYW3ydKVz4VmfbgkoqCucuCmW5uTApSZrs72tf9tnPu+LpI3EJf51Fc+nIXlJwpTiK4nV48ErDNbsdb8oAoXv7EhekLFj6WP3dZMPjaM9onF6bYmzdv5vHKjvEy0qPG5XPzw63NJwKrf2grd4PjOSD1julKjiaJ0icNIAHVVYT4f1vljNW/Efeu3/RxdtHkx6oA1Ze4MyfdzjFdUF+f+HYVzYhw4bTI1J8+rGpTwYUz71eJoxQxHTjMYz2wA1LccKzLTuxMqxUWUA821qL90OeP2kb0d/WATUQOM6ftuHQ1oCQ3lPcc++fejjefc11ua7a0kvnaUKQXDAGvi6AQgdmBIryQQyHpLV4gJn2rGoHG7Rg+0Sm9ORCrtuzEPeWrdFXzvJDqO6MCAEBEjq7rChEdnLRgycrZP3vFLJqzVLMunrRyvSUgOIiaDIcdqCIbMhKS+SVfF6XLq7rnrHnqrys2/YELp0JYWbB2b3wcZ5r26rYR2zzie7wSGSdhMBjUysrKLGYu72lt3tfx1q+0K+3NUtw5RwxFrwCqG27rvJ1XNFudVFkTmb5w0cP5c8qPsG1uCv0j8Oqnz1fCJFi4BO37rzfi7uWV11UirfHIAwdzec/RZjOwaQl/tArWrg3z+J0fwN6x/j4++peXeqNdTWUA4NOrXMn1G0OH9/LWxeDapTBr54A79u9k24jpqam+IUByoRiBuNDabAQ2LeGPvgur4VHIv29+kge7zxwfUZOZFT2ZVmbeGDq0h7cuBW9ZDPOV+eDO/Y23COH3j0L0HG02P3+unN8Dhltf/iWHu47tGV2XgCAkU8vMz3Y17eFtD4FfXpiA6Ni/k20rURM3ZakQ3f8+FOve8Ufu3P3n/XzG705eH60nZqYUJZ7tOpSA2LI4kY7OA41sGddCkA4oDFC6j67rSkpNPGKGe9/hvr7cTFJeA9H0GW/9JnjLIhhb7gJ3BT5mM/zVgzevRPI9IOV3xjyOhwgllaj92lWA1LOA9Bs8oOgAJ88KBYGAghUr7Ou1ExExMwNEChH9iZnxnZd2b4O0MGPJSl3zeL4k/wqIigDk1sqcn8OhFwFcZEAQgZlHgWwCF4Bw8Bd7I1WceObgTIHTKDHa9xwfmGWYpurOK24HAFERgA0Ag8O2Z3KOWthvOIXKuBPHYSDHRbgSd2YCAAHsq4K6rh42JmDJCUhExOTOP50KJhpWTy3yZDvlPQPWIsu2keWCyeNGNAGOTXAVZJP3k+rCJ01Htqz5oPe4rkPRddx4omE0HYT6egVVAFDlEJEj1CyekaepDa5CBYZkBuDK5ISBedNylbfPDiovAjg+uRsqMDGAEQhgrGoCJO0BQ4DBYEb64z5pkoEBw4HiYBgAZndMvA4ymcjLyu4YNOT9ACBsi6XI/CLJNnHYIGFbzlcAsCIwsRq4nv0Phi4X5d/okEsAAAAASUVORK5CYII=</Image>
> +<Url type="text/html" method="GET" template="https://auctions.yahoo.co.jp/search/search" resultdomain="yahoo.co.jp">
> +  <Param name="p" value="{searchTerms}"/>
> +  <Param name="ei" value="EUC-JP"/>
> +  <Param name="fr" value="mozff" />

This looks like a partner code.

::: mail/components/search/searchplugins/yahoo-jp.xml
@@ +9,5 @@
> +<Image width="16" height="16">data:image/x-icon;base64,AAABAAIAEBAAAAEAIABoBAAAJgAAACAgAAABACAAqBAAAI4EAAAoAAAAEAAAACAAAAABACAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADMA/24zAP//MwD//zMA//8zAP//MwD//wAAAAAAAAAAMwD/7TMA/+MAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADkI//8zAP//MwD/bwAAAAAAAAAAAAAAADEA/zQyAP8pNwD/FwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA5CP//MwD//zIA/3AAAAAAAAAAAAAAAAAAAAAAMgD/wDIA/6cAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0AP8sMwD/9zMA//8zAP//NwD/FwAAAAAAAAAAAAAAADQA/3EzAP//MQD/PgAAAAAAAAAAAAAAAAAAAAA3AP8OMwD/2DMA//80AP9PMwD/WjMA//8xAP8qAAAAAAAAAAAyAP8kMwD//zMA/9EAAAAIAAAAAAAAAAAAAAADMwD/tDMA//8yAP+JAAAAAAAAAAAxAP85MwD//zQA/1gAAAAAAAAAADMA/9UzAP//MwD/qQAAAAAAAAAAMgD/kzMA//8zAP/EAAAABAAAAAAAAAAAMwD//zMA//8zAP//MwD//zMA/30xAP8+MwD/+TEA/z4yAP+sMwD//zMA//8zAP//MwD//zMA/5oAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP//AAD//wAA//8AAP//AADgZwAA+OMAAPjzAADwcQAA4DAAAMMYAADDAAAAA/8AAP//AAD//wAA//8AAP//AAAoAAAAIAAAAEAAAAABACAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMwD//zMA//8zAP//MwD//zMA//8zAP//MwD//zMA//8zAP//MwD//zMA//8zAP//AAAAAAAAAAAAAAAAMQD/HzQA/2MyAP+iAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAyAP+dMwD/9jMA//8zAP//MwD//zMA//8zAP//MwD//zMA//8zAP//MwD//zMA/3gAAAAAAAAAAAAAAAAzAP+/MwD//zMA//80AP9UAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANAD/JzQA/4ozAP//MwD//zMA//8zAP/nMwD/NzQA/ycxAP8VAAAABAAAAAAAAAAAAAAAADMA/1AzAP//MwD/zjMA/18AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADMA//8zAP//MwD//zMA/98AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAjAA/yUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMwD//zMA//8zAP//MwD/3wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADUA/yIzAP9zMwD/UAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAzAP//MwD//zMA//8zAP/fAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANAD/njMA//8zAP/pMwD/DwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANAD/TzMA//8zAP//MwD//zMA/+AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAzAP9QMwD//zMA//80AP+PAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAg0AP+UMwD//zMA//8zAP//MwD//jIA/1IAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC4A/wszAP/2MwD//zMA//sxAP8qAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMwD/lzMA//8zAP//MwD//zMA/84zAP/5MwD/+TIA/1EAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADMA/7MzAP//MwD//zQA/7wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADIA/1szAP//MwD//zMA//8zAP/fOQD/EjMA/0EzAP/tMwD//DQA/2wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMgD/ZTMA//8zAP//MwD//zIA/1IAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAxAP8vMwD/9TMA//8zAP//MwD/+DEA/zQAAAAAAAAAADUA/yIzAP/UMwD//zMA/5YAAAAGAAAAAAAAAAAAAAAAAAAAAAAAAAAzAP8ZMwD//TMA//8zAP//MwD/3wAAAAkAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAALgD/FjMA/98zAP//MwD//zMA//8yAP9mAAAAAAAAAAAAAAAAAAAAADMA/w8zAP+4MwD//zMA/8MzAP8ZAAAAAAAAAAAAAAAAAAAAAAAAAAAzAP/IMwD//zMA//8zAP//MgD/fwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAYzAP/DMwD//zMA//8zAP//MgD/ogAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAYzAP+zMwD//zMA/+k0AP9JAAAAAAAAAAAAAAAAAAAAADMA/3kzAP//MwD//zMA//8zAP/2MAD/IAAAAAAAAAAAAAAAAAAAAAAAAAABMwD/pDMA//8zAP//MwD//zMA/9YuAP8LAAAAAAAAAAAAAAAAAAAAAAAAAAA1AP8/NAD/YjMA/5ozAP//MwD//zMA//8zAP+/MwD/gzIA/00wAP8QNQD/KzMA//8zAP//MwD//zMA//8zAP+zAAAAAAAAAAAAAAAAAAAAADMA/40zAP//MwD//zMA//8zAP/3MwD/LQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADMA/98zAP/fMwD/3zMA/98zAP/fMwD/3zMA/98zAP/fMwD/3zMA/58AAAAAMwD/3TMA//8zAP/fMwD/kTEA/0MzAP88MgD/YDMA/4IzAP+6MwD//zMA//8zAP//MwD//zMA/7oyAP9HNAD/NjAA/yUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0AP9FMgD/LgAAAAAAAAAAAAAAADMA//YzAP//MwD//zMA//8zAP//MwD//zMA//8zAP//MwD//zMA//8zAP//MwD//wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/////////////////////////////////////+ABw//gAcP/+AHD//4fz//+H/H//h/w//wf8P/4D/B/+Af4f/AD+D/gYPgfwPB8H4H4PA8B8AAPA/ACAAD//nAA//////////////////////////////////////////////</Image>
> +<Url type="text/html" method="GET" template="https://search.yahoo.co.jp/search" resultdomain="yahoo.co.jp">
> +  <Param name="p" value="{searchTerms}"/>
> +  <Param name="ei" value="UTF-8"/>
> +  <Param name="fr" value="mozff" />

Same here.

::: mail/components/search/searchplugins/yandex-az.xml
@@ +10,5 @@
> +<Url type="application/x-suggestions+json" method="GET" template="https://yandex.az/suggest/suggest-ff.cgi">
> +  <Param name="part" value="{searchTerms}"/>
> +</Url>
> +<Url type="text/html" method="GET" template="https://yandex.az/search" resultdomain="yandex.az">
> +  <MozParam name="clid" condition="purpose" purpose="searchbar"   value="2186618"/>

The clid parameter looks like partner codes.
Assignee

Comment 21

5 months ago
(In reply to Florian Quèze [:florian] (PTO until Jan 2) from comment #20)
> IIRC Thunderbird had a search deal with Bing 8 years ago. Does
> this deal still exist?
Good question. Same goes for whether there's a deal with Box.com (FileLink provider).
For Thunderbird, you should use just a generic google.xml with no codes at all (not our renames of google-b-d, etc.)

And to Florian's point, yes those are all partner codes that should be removed unless Thunderbird has separate agreements in place (I'll confirm this).
Comment on attachment 9033912 [details] [diff] [review]
1427133-search-esr60.patch (v2)

r+ with this section removed:

+    <MozParam name="t" condition="purpose" purpose="contextmenu" value="ffcm"/>
+    <MozParam name="t" condition="purpose" purpose="keyword"     value="ffab"/>
+    <MozParam name="t" condition="purpose" purpose="searchbar"   value="ffsb"/>
+    <MozParam name="t" condition="purpose" purpose="homepage"    value="ffhp"/>
+    <MozParam name="t" condition="purpose" purpose="newtab"      value="ffnt"/>

These are the Firefox specific codes.
Attachment #9033912 - Flags: review?(mozilla) → review+
Assignee

Updated

5 months ago
Attachment #9033890 - Attachment is obsolete: true
Attachment #9033890 - Flags: review?(mozilla)
Assignee

Comment 24

5 months ago
(In reply to Mike Kaply [:mkaply] from comment #22)
> For Thunderbird, you should use just a generic google.xml with no codes at
> all (not our renames of google-b-d, etc.)
> 
> And to Florian's point, yes those are all partner codes that should be
> removed unless Thunderbird has separate agreements in place (I'll confirm
> this).
OK, I''l revise that. I thought, we're still called "Mozilla Thunderbird", so if TB creates some search traffic, no one should be upset ;-)
Assignee

Comment 25

5 months ago
Great, we've got TB 60 covered :-) - I've removed the partner codes and some trailing spaces.
Attachment #9033912 - Attachment is obsolete: true
Attachment #9034010 - Flags: review+
Assignee

Comment 26

5 months ago
Notes for TB 60 ESR: Try build here:
https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=c692499bf776276db5ac7f1743e7856f02b9aa5b
Working nicely, we'll follow-up with getting the icons right in bug 1512067 (uplift required) and bug 1517352.
Assignee

Updated

5 months ago
Attachment #9034010 - Flags: approval-comm-esr60+
Assignee

Comment 27

5 months ago
TB 60.5 ESR:
https://hg.mozilla.org/releases/comm-esr60/rev/ebeea939a6e421d36d047a222b9d968cdd885bac

Mike, I've done the unorthodox thing to land the ESR 60 patch first. I hope we can make progress on the trunk patch soon. Well, I guess you want me to remove all the partner codes from the patch first, right? I'll get to this later today.
Assignee

Comment 28

5 months ago
Posted patch 1427133-search.patch (v2) (obsolete) — Splinter Review
As per Florian's comments from comment #20:

- Only using one "google" now, removed MozParam name="client".
- removed partner codes as well as I could:
  + sed -i '/<Param name="tag"/d' *.xml - Amazon
  + sed -i '/<Param name="tn"/d' baidu.xml
  + sed -i '/<Param name="pc"/d' bing.xml
  + sed -i '/<Param name="campid"/d' ebay*.xml
  + sed -i '/<Param name="partner"/d' twitter*.xml
  + sed -i '/<Param name="fr"/d' yahoo*.xml
  + sed -i '/<MozParam name="clid"/d' yandex*.xml
  + sed -i '/<Param name="client"/d' marktplaats*.xml
  + sed -i '/<Param name="client"/d' qwant.xml, or should <Param name="client" value="opensearch"/> have stayed?

Left these:
<Param name="sourceid" value="Mozilla-search"/> which is in Amazon, Wikipedia, etc.
bing.xml:        <MozParam name="form" condition="purpose" purpose="contextmenu" value="MOZCON"/>
bing.xml:        <MozParam name="form" condition="purpose" purpose="searchbar" value="MOZSBR"/>
bing.xml:        <MozParam name="form" condition="purpose" purpose="homepage" value="MOZSPG"/>
bing.xml:        <MozParam name="form" condition="purpose" purpose="keyword" value="MOZLBR"/>
bing.xml:        <MozParam name="form" condition="purpose" purpose="newtab" value="MOZTSB"/>

Please give further instructions.
Attachment #9033886 - Attachment is obsolete: true
Attachment #9033886 - Flags: review?(mozilla)
Attachment #9034615 - Flags: review?(mozilla)

Comment 29

5 months ago

Are these patches going to land in TB any time soon?

Assignee

Comment 30

5 months ago

Yes, already landed on the TB 60 ESR branch and will be released in TB 60.5 at the end of the month. Do you want an unofficial TB 60.4.1? For which platform?

Comment 31

5 months ago

I am ready to wait - but Great news Jorg. Thanks!

Comment 32

4 months ago

I just downloaded Thunderbird to see how it's doing these days after reading that there have been new devs hired. I want to experience it now, so I can watch it improve. :-) That said, I was also trying to get DDG search working. I eventually got to this ticket, which is great to see this is being worked on.

However, I'm curious: is it even necessary to specify a search engine in Thunderbird? It would be awesome if there were an option to just "Use my browser's default search engine", and then I only have to set the search engine in one place (Firefox, in my case). Perhaps there are technical constraints that don't make this feasible, though.

Thanks for all the great, hard work!

Comment on attachment 9034615 [details] [diff] [review]
1427133-search.patch (v2)

Review of attachment 9034615 [details] [diff] [review]:
-----------------------------------------------------------------

Remove this section from Bing:

        <MozParam name="form" condition="purpose" purpose="contextmenu" value="MOZCON"/>
        <MozParam name="form" condition="purpose" purpose="searchbar" value="MOZSBR"/>
        <MozParam name="form" condition="purpose" purpose="homepage" value="MOZSPG"/>
        <MozParam name="form" condition="purpose" purpose="keyword" value="MOZLBR"/>
        <MozParam name="form" condition="purpose" purpose="newtab" value="MOZTSB"/>

You should remove the eBay engines completely. Those URLs only work as affiliate URLs, so they won't work at all with any changes.

Otherwise, this looks good
Attachment #9034615 - Flags: review?(mozilla) → review+
Assignee

Comment 34

4 months ago

Thanks, Mike. I've removed the said lines from Bing and I removed all eBay engines.

Attachment #9034615 - Attachment is obsolete: true
Attachment #9035761 - Flags: review+

Comment 35

4 months ago

Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/comm-central/rev/1337b98ae249
Fix search engines by copying browser/components/search/searchplugins and removing partner codes. r=mkaply

Status: ASSIGNED → RESOLVED
Last Resolved: 4 months ago
Resolution: --- → FIXED
Assignee

Updated

4 months ago
Target Milestone: --- → Thunderbird 66.0
Assignee

Updated

4 months ago
Attachment #9035761 - Flags: approval-comm-beta+

Comment 37

4 months ago

And confirmed live in 60.5.0 - thank you very much for this!

You need to log in before you can comment on or make changes to this bug.