1263313 - (new-amo-search) Change the search in add-ons manager to use just AMO search

Status: VERIFIED FIXED

(Toolkit :: Add-ons Manager, defect)

Description

[Andy McKay]

https://www.dropbox.com/s/n5665x8ksuxajx5/Screenshot%202016-04-08%2015.55.04.png?dl=0

The search in the add-ons manager does a couple of things:
* it allows you to search your local add-ons
* search available add-ons from amo (and filters out local add-ons)
* clicking more takes you to amo
* install an add-on without using doorhangers (the flow is quite different)

I'd like to propose that:
* searching in your local add-ons really isnt that useful
* having a page search an api and parse the results isn't all that useful, when search more just takes you to amo

I think we can remove the functionality that isn't much value and point search add-ons straight at AMO. We'd replace the entire search page with an AMO page. With the improvements in bug 1245571, we'll now be able to show those search results and highlight the add-ons you've already got installed and show upgrades.

The end result should be the removal of a lot of code (quite a few bugs) and an improved experience with minimal functional loss.

Comment 1

[Markus Jaritz [:designakt] (UX)]

Attachment: addon-search-result-comparison.gif

I think this is a brilliant idea!
The difference in those 2 searches so far is that the results on about:addons does not return experimental add-on, or add-ons that do not work on your Firefox, but a search on AMO does, which makes it more difficult to pick the right add-on.

So I think that AMO search needs to be improved before we can do that. This should include ranking experimental add-ons lower (or not showing them by default, if you arrive via an about:addons search), and not showing add-ons that do not work with ones version of Firefox.
And we need to improve the layout of the AMO search to view at least 5-6 add-ons on an area the size of the attachment (instead of currently 3.5), and give them the same space, same hight.

Comment 2

[Andy McKay]

I agree with you on the filtering, AMO should be trying to do the same kind of filtering that the add-ons manager does.

Comment 3

[Andy McKay]

Let's just do this. When entering something into the search box it will bounce over to AMO. Doing so in a new tab is perfectly acceptable.

Comment 4

[Andrew Swan [:aswan]]

So what should the URL be?  Something like?
https://addons.mozilla.org/en-US/firefox/search/?q=${TERM}&appver=${VERSION}&platform=${PLATFORM} ?

Comment 5

[[e-mail]]

I'm checking on this since I think we might have slightly different params between the old and new frontend.

Comment 6

[[e-mail]]

(In reply to Andrew Swan [:aswan] from comment #4)
> So what should the URL be?  Something like?
> https://addons.mozilla.org/en-US/firefox/search/
> ?q=${TERM}&appver=${VERSION}&platform=${PLATFORM} ?

I think it should be:

https://addons.mozilla.org/%LOCALE%/%CLIENTAPP%/search/?q=${TERM}&appver=${VERSION}&platform=${PLATFORM}

Note: On the new frontend we only support firefox or android for CLIENTAPP.

Also the locale is a bit of a tricky one - should that be the UI locale? That might not give the same results as leaving out the locale in the url since what's supported for languages in web requests isn't necessarily the same as the UI locale.

If you leave out the %LOCALE% then the app will redirect based on accept headers.
If you leave out the %CLIENTAPP% the app will redirect based on the user-agent request header.

Comment 7

[[e-mail]]

Is any of this going to require the new find as you type API to show suggestions before just sending the users to AMO? We're going to need a new one for the new frontend based on the current mocks.

Comment 8

[[e-mail]]

(In reply to Stuart Colville [:scolville] [:muffinresearch] from comment #7)
> Is any of this going to require the new find as you type API to show
> suggestions before just sending the users to AMO? We're going to need a new
> one for the new frontend based on the current mocks.

My rationale here would be search suggestions ahead of sending the user to amo would almost certainly be a better UX.

Comment 9

[Markus Jaritz [:designakt] (UX)]

As a first step it is OK without search suggestions, as it would keep the status quo.
But I agree, suggestions while typing would be a huge improvement which we should integrate in the new addons manager.

Comment 10

[Markus Jaritz [:designakt] (UX)]

Adding pwalm, so that he is aware that there will be traffic from addons manager to AMO search resuts pages soon.

Comment 11

[Andrew Swan [:aswan]]

(In reply to Markus Jaritz [:designakt] (UX) from comment #10)
> Adding pwalm, so that he is aware that there will be traffic from addons
> manager to AMO search resuts pages soon.

To be clear, I wouldn't expect this any time "soon" with all the other higher priority work going on right now.

Comment 12

[Kev Needham [:kev]]

Concur that redirecting search to AMO makes sense. 

Only things I'd add are:

- We should notify the user that search entries will go to AMO. Nothing fancy required, text near the search bar is perfect
- Results should open in a newtab vs. in about:addons to make it clear that the results are coming from a web server

Comment 13

[Andy McKay]

Attachment: Screenshot 2017-07-21 07.39.03.png

For notification I suggest we change the placeholder in the input from "Search all add-ons" to "Search on addons.mozilla.org" or something similar.

Comment 14

[YF (Yang)]

(In reply to Andy McKay [:andym] from comment #13)
> Created attachment 8888802 [details]
> Screenshot 2017-07-21 07.39.03.png
> 
> For notification I suggest we change the placeholder in the input from
> "Search all add-ons" to "Search on addons.mozilla.org" or something similar.

I vote for "Search Mozilla Add-ons".

Comment 15

[YF (Yang)]

A question, after this change, sorting (name, last updated or best matching) will be implemented on AMO or abandoned?

Comment 16

[Kev Needham [:kev]]

Would prefer https://addons.mozilla.org to make it clearer that requests are sent to a website. "Search Mozilla Add-ons" doesn't allow for that.

(In reply to Andy McKay [:andym] from comment #13)
> Created attachment 8888802 [details]
> Screenshot 2017-07-21 07.39.03.png
> 
> For notification I suggest we change the placeholder in the input from
> "Search all add-ons" to "Search on addons.mozilla.org" or something similar.

Comment 17

[tofumatt [:tofumatt]]

Also we call the site "Firefox Add-ons". "Search Firefox Add-ons site" would work if you don't wanna use a URL (which is good because it's kind of ugly).

Comment 18

[YF (Yang)]

(In reply to Kev Needham [:kev] from comment #16)
> Would prefer https://addons.mozilla.org to make it clearer that requests are
> sent to a website. "Search Mozilla Add-ons" doesn't allow for that.

I don't think that highlighting it will open a website is a requirement.

If necessary, maybe can use the "..." or an icon or a additional icon-button (also available to open the AMO home page) on search box side.

(In reply to tofumatt [:tofumatt] from comment #17)
> Also we call the site "Firefox Add-ons". "Search Firefox Add-ons site" would
> work if you don't wanna use a URL (which is good because it's kind of ugly).

Hard-code the "Search Firefox Add-ons site" seems to good.

Comment 19

[Kev Needham [:kev]]

(In reply to YF (Yang) from comment #18)
> (In reply to Kev Needham [:kev] from comment #16)
> > Would prefer https://addons.mozilla.org to make it clearer that requests are
> > sent to a website. "Search Mozilla Add-ons" doesn't allow for that.
> 
> I don't think that highlighting it will open a website is a requirement.

I do. The page is presented in the Add-ons Manager section, and the search box is in what a user could reasonably expect to think is pasrt of the Firefox UI. I wish to make it clear before the user hits enter that information will be sent. "Search Firefox Add-ons (https://addons.mozilla.org) is an acceptable compromise.

Comment 20

[emanuela [ux]]

Hey all.

My suggestion here is

>>> Find extensions and themes at addons.mozilla.com

Find: sounds more as promise, you search to find something. Plus, it aligns us with preferences.

Then, the what: extensions and themes. 
Finally, where:addons.mozilla.com

Comment 21

[YF (Yang)]

(In reply to emanuela [ux team] from comment #20)
> Hey all.
> 
> My suggestion here is
> 
> >>> Find extensions and themes at addons.mozilla.com
> 
> Find: sounds more as promise, you search to find something. Plus, it aligns
> us with preferences.
> 
> Then, the what: extensions and themes. 
> Finally, where:addons.mozilla.com

You forgot the dictionary and search plugins.


I now think the "Search add-ons on addons.mozilla.com" is good.
#header-search witdh should be customizable from locales, like 23em or 30em.

Comment 22

[Markus Jaritz [:designakt] (UX)]

(In reply to emanuela [ux team] from comment #20)
> Hey all.
> 
> My suggestion here is
> 
> >>> Find extensions and themes at addons.mozilla.com
> 
> Find: sounds more as promise, you search to find something. Plus, it aligns
> us with preferences.
> 
> Then, the what: extensions and themes. 
> Finally, where:addons.mozilla.com

Can you please visualize how that should look in product. I worry that this might not fit. Certainly in other languages.

Comment 23

[emanuela [ux]]

Attachment: new-amo-search

Hey Markus, you're right, it doesn't fit (see attachment). 

Ok, splitting extensions and themes (as we're going to do in the new AMO) is not an option.

But, as we can see, even a more generic

> Find add-ons at addons.mozilla.org

fails with the current width, meaning we're going to change the width in any way since we're going to add the URL in the placeholder text.

So, my new suggestion for the string is

> Find new add-ons at addons.mozilla.org

(Yes, I will stick with the Find for the language where make sense for having a cohesive way to talk to the users in in-content pages).

Comment 24

[Andrew Swan [:aswan]]

I started on this and realized that the existing UX becomes clumsy with search redirected to AMO.  Right now, we don't show "Search" as an option on the left side of about:addons until you have done a search.  Even then, if you go to another pane, we take "Search" back out of the list so you can't easily get get back to your search results (pressing Enter in the search box without changing the query is the simplest way but its not particularly obvious).

I've also attached two screenshots, one from the moment right before submitting a search, and then one after we get the search results.  For starters, the new AMO page isn't really designed to be embedded in about:addons and I am not a visual designer but I think the two clash.  On a more functional level, the search box from the header is replaced with the one in content which is still roughly in the upper right, but not actually in the same place and with a different appearance.

I'll take off my amateur UX designer hat now and pass this over to Markus, what do you think?

Comment 25

[Andrew Swan [:aswan]]

Attachment: Screen Shot right before search

Comment 26

[Andrew Swan [:aswan]]

Attachment: Screen Shot right after search

Comment 27

[Markus Jaritz [:designakt] (UX)]

I think we intended for this search to open an new tab with the search result on AMO.
That is why Emanuela proposed a tweak to the search field placeholder text.
Emanuela can you specify the flow for Andrew. Thanks.

Comment 28

[Andrew Swan [:aswan]]

(In reply to Markus Jaritz [:designakt] (UX) from comment #27)
> I think we intended for this search to open an new tab with the search
> result on AMO.

I had understood that to be a temporary solution.  If that's meant to be permanent, that would be good to know, it will definitely affect how I approach this.

Comment 29

[emanuela [ux]]

Attachment: search-example.png

This solution is meant to be temporary. The search box should open a new tab a bring people to AMO. 
*IF* we're looking for a different solution, the only part we need to embed is the 'SearchResults' div (see attachment).

With the new AoM into preferences will have an all-new series of problems with the Search that we're going to need an entirely new flow.

Comment 30

[Andrew Swan [:aswan]]

(In reply to emanuela [ux team] from comment #29)
> This solution is meant to be temporary. The search box should open a new tab
> a bring people to AMO. 
> *IF* we're looking for a different solution, the only part we need to embed
> is the 'SearchResults' div (see attachment).

I don't understand, what is the long-term desired behavior?  Embedding just a subset of the AMO page (the SearchResults div) is not practical.

> With the new AoM into preferences will have an all-new series of problems
> with the Search that we're going to need an entirely new flow.

What is "the new AoM into preferences"?

Comment 31

[Andy McKay]

I didn't realise we were planning on keeping search anywhere in about:addons as some kind of embedded frame. I think the story around discovery for 2018 is in enough doubt at this point, that we probably shouldn't worry about blocking on future plans. What search will look like in the new flows I don't know.

Comment 32

[Philip [:pwalm]]

> What is "the new AoM into preferences"?

We're exploring integrating add-ons management (AoM) into browser prefs. So adding search to "Get Add-ons" at this point seems not really worth it, since that page might not even exist in a few months time.

Comment 33

[Andrew Swan [:aswan]]

(In reply to Andy McKay [:andym] from comment #31)
> I didn't realise we were planning on keeping search anywhere in about:addons
> as some kind of embedded frame. I think the story around discovery for 2018
> is in enough doubt at this point, that we probably shouldn't worry about
> blocking on future plans. What search will look like in the new flows I
> don't know.

To be clear, I was trying to figure out if the patch for this bug should just rip out the Search pane entirely.  I don't want to do that if there's some reasonable chance that we'll get a new design at some point with embedded search results.
Opening a new tab feels like really wonky UX to me but its easy enough to do...

Comment 34

[emanuela [ux]]

Let's focus on this part:

(In reply to Andy McKay [:andym] from comment #3)
> When entering something into the search box it will
> bounce over to AMO. Doing so in a new tab is perfectly acceptable.

This behaviour is entirely acceptable.   

In the search box, the placeholder text should be:
> Find new add-ons at addons.mozilla.org

If the user searches something and hit the press button, Firefox will open a new tab, which it will point to the AMO search results page.

Comment 35

[Andrew Swan [:aswan]]

Attachment: Bug 1263313 Remove tests of old about:addons search

Also consolidated the two .ini files in toolkit/mozapps/extension/test/browser
since we only run these tests in tabs now.

Review commit: https://reviewboard.mozilla.org/r/205954/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/205954/

Comment 36

[Andrew Swan [:aswan]]

Attachment: Bug 1263313 Switch search to AMO

Review commit: https://reviewboard.mozilla.org/r/205956/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/205956/

Comment 37

[Andrew Swan [:aswan]]

Attachment: Bug 1263313 Remove search pane from about:addons

Also remove related code that was only used from here including
stuff related to marketplace purchases, a few XBL bindings, etc.

Review commit: https://reviewboard.mozilla.org/r/205958/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/205958/

Comment 38

[Andrew Swan [:aswan]]

Linux try run is here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=5297cd03d7b11b278e3439bf631b29962e7b0f25

I should probably also do a try run on Android, I'll launch that from reviewboard

Comment 39

[Andy McKay]

Letting marshall know for privacy concerns around comment 13.

Comment 40

[Merwin]

This seems appropriate to me.  The concern here is that this will bounce you to an AMO page with tracking. But we have other functionality that might bounce you to pages with tracking. For example, we now have pocket recommendations that might land you on a new york times article that has GA.  In this case, the page you land on is Mozilla's, but categorically we are ok with users landing on pages with trackers.

Comment 41

[Robert Helmer [:rhelmer]]

Comment on attachment 8935111 [details]
Bug 1263313 Remove tests of old about:addons search

https://reviewboard.mozilla.org/r/205952/#review212094

::: toolkit/mozapps/extensions/test/browser/head.js
(Diff revision 1)
>                       {name: "extensions.update.background.url"},
>                       {name: "extensions.update.enabled"},
>                       {name: "extensions.update.autoUpdateDefault"},
>                       {name: "extensions.getAddons.get.url"},
>                       {name: "extensions.getAddons.getWithPerformance.url"},
> -                     {name: "extensions.getAddons.search.browseURL"},

Hm, this pref is still used no?

Comment 42

[Robert Helmer [:rhelmer]]

Comment on attachment 8935113 [details]
Bug 1263313 Remove search pane from about:addons

https://reviewboard.mozilla.org/r/205958/#review212072

::: browser/app/profile/firefox.js
(Diff revision 1)
>  pref("extensions.webextPermissionPrompts", true);
>  pref("extensions.webextOptionalPermissionPrompts", true);
>  
>  // Preferences for AMO integration
>  pref("extensions.getAddons.cache.enabled", true);
> -pref("extensions.getAddons.maxResults", 15);

This pref is still present in the a few places in the tree, for instance https://searchfox.org/mozilla-central/source/testing/profiles/prefs_general.js and also talos config.

::: browser/app/profile/firefox.js
(Diff revision 1)
>  pref("extensions.getAddons.cache.enabled", true);
> -pref("extensions.getAddons.maxResults", 15);
>  pref("extensions.getAddons.get.url", "https://services.addons.mozilla.org/%LOCALE%/firefox/api/%API_VERSION%/search/guid:%IDS%?src=firefox&appOS=%OS%&appVersion=%VERSION%");
>  pref("extensions.getAddons.getWithPerformance.url", "https://services.addons.mozilla.org/%LOCALE%/firefox/api/%API_VERSION%/search/guid:%IDS%?src=firefox&appOS=%OS%&appVersion=%VERSION%&tMain=%TIME_MAIN%&tFirstPaint=%TIME_FIRST_PAINT%&tSessionRestored=%TIME_SESSION_RESTORED%");
>  pref("extensions.getAddons.search.browseURL", "https://addons.mozilla.org/%LOCALE%/firefox/search?q=%TERMS%&platform=%OS%&appver=%VERSION%");
> -pref("extensions.getAddons.search.url", "https://services.addons.mozilla.org/%LOCALE%/firefox/api/%API_VERSION%/search/%TERMS%/all/%MAX_RESULTS%/%OS%/%VERSION%/%COMPATIBILITY_MODE%?src=firefox");

Same as above, appears in talos config.

::: browser/app/profile/firefox.js
(Diff revision 1)
>  pref("extensions.getAddons.get.url", "https://services.addons.mozilla.org/%LOCALE%/firefox/api/%API_VERSION%/search/guid:%IDS%?src=firefox&appOS=%OS%&appVersion=%VERSION%");
>  pref("extensions.getAddons.getWithPerformance.url", "https://services.addons.mozilla.org/%LOCALE%/firefox/api/%API_VERSION%/search/guid:%IDS%?src=firefox&appOS=%OS%&appVersion=%VERSION%&tMain=%TIME_MAIN%&tFirstPaint=%TIME_FIRST_PAINT%&tSessionRestored=%TIME_SESSION_RESTORED%");
>  pref("extensions.getAddons.search.browseURL", "https://addons.mozilla.org/%LOCALE%/firefox/search?q=%TERMS%&platform=%OS%&appver=%VERSION%");
> -pref("extensions.getAddons.search.url", "https://services.addons.mozilla.org/%LOCALE%/firefox/api/%API_VERSION%/search/%TERMS%/all/%MAX_RESULTS%/%OS%/%VERSION%/%COMPATIBILITY_MODE%?src=firefox");
>  pref("extensions.webservice.discoverURL", "https://discovery.addons.mozilla.org/%LOCALE%/firefox/discovery/pane/%VERSION%/%OS%/%COMPATIBILITY_MODE%");
> -pref("extensions.getAddons.recommended.url", "https://services.addons.mozilla.org/%LOCALE%/%APP%/api/%API_VERSION%/list/recommended/all/%MAX_RESULTS%/%OS%/%VERSION%?src=firefox");

This one doesn't seem to be used elsewhere at least :)

Comment 43

[Andrew Swan [:aswan]]

Comment on attachment 8935113 [details]
Bug 1263313 Remove search pane from about:addons

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/205958/diff/1-2/

Comment 44

[Andrew Swan [:aswan]]

Comment on attachment 8935112 [details]
Bug 1263313 Switch search to AMO

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/205956/diff/1-2/

Comment 45

[Andrew Swan [:aswan]]

Comment on attachment 8935113 [details]
Bug 1263313 Remove search pane from about:addons

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/205958/diff/2-3/

Comment 47

[Andrew Swan [:aswan]]

From https://bugzilla.mozilla.org/show_bug.cgi?id=1424048#c0, there was a request to add a src attribute to the search URL.  What should the value be?

Comment 48

[Robert Helmer [:rhelmer]]

Comment on attachment 8935111 [details]
Bug 1263313 Remove tests of old about:addons search

https://reviewboard.mozilla.org/r/205952/#review212122

lgtm! tried it out locally, pretty great.

One minor thing I noticed: `toolkit/mozapps/extensions//test/browser/addons/browser_searching/` appears to contain `install.rdf` and `bootstrap.js` that can be removed as well.

Comment 50

[Robert Helmer [:rhelmer]]

Comment on attachment 8935113 [details]
Bug 1263313 Remove search pane from about:addons

https://reviewboard.mozilla.org/r/205958/#review212146

Comment 51

[Robert Helmer [:rhelmer]]

Comment on attachment 8935112 [details]
Bug 1263313 Switch search to AMO

https://reviewboard.mozilla.org/r/205956/#review212144

Comment 52

[Robert Helmer [:rhelmer]]

Comment on attachment 8935111 [details]
Bug 1263313 Remove tests of old about:addons search

https://reviewboard.mozilla.org/r/205954/#review212142

Comment 53

[Andrew Swan [:aswan]]

Comment on attachment 8935111 [details]
Bug 1263313 Remove tests of old about:addons search

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/205954/diff/1-2/

Comment 54

[Andrew Swan [:aswan]]

Comment on attachment 8935112 [details]
Bug 1263313 Switch search to AMO

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/205956/diff/2-3/

Comment 55

[Andrew Swan [:aswan]]

Comment on attachment 8935113 [details]
Bug 1263313 Remove search pane from about:addons

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/205958/diff/3-4/

Comment 56

[Mozilla Autoland]

We're sorry, Autoland could not rebase your commits for you automatically. Please manually rebase your commits and try again.

hg error in cmd: hg rebase -s 98621d95291d -d e6cd64922e36: rebasing 438366:98621d95291d "Bug 1263313 Remove tests of old about:addons search r=rhelmer"
merging toolkit/mozapps/extensions/test/browser/browser_bug562797.js
warning: conflicts while merging toolkit/mozapps/extensions/test/browser/browser_bug562797.js! (edit, then use 'hg resolve --mark')
unresolved conflicts (see hg resolve, then hg rebase --continue)

Comment 57

[Andrew Swan [:aswan]]

Comment on attachment 8935111 [details]
Bug 1263313 Remove tests of old about:addons search

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/205954/diff/2-3/

Comment 58

[Andrew Swan [:aswan]]

Comment on attachment 8935112 [details]
Bug 1263313 Switch search to AMO

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/205956/diff/3-4/

Comment 59

[Andrew Swan [:aswan]]

Comment on attachment 8935113 [details]
Bug 1263313 Remove search pane from about:addons

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/205958/diff/4-5/

Comment 60

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2ce52e33f6d5
Remove tests of old about:addons search r=rhelmer
https://hg.mozilla.org/integration/autoland/rev/713844b922a7
Switch search to AMO r=rhelmer
https://hg.mozilla.org/integration/autoland/rev/2297e770f16c
Remove search pane from about:addons r=rhelmer

Comment 61

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/2ce52e33f6d5
https://hg.mozilla.org/mozilla-central/rev/713844b922a7
https://hg.mozilla.org/mozilla-central/rev/2297e770f16c

Comment 62

[marius.santa]

Attachment: search.gif

I was able to reproduce the issue on Windows 10 64Bit Firefox  	53.0a1(20170101030204).
Tested and verified on Windows 10 64Bit, Ubuntu 16.04 LTS and Mac OS  X 10.13.2 in Firefox 60.0a1 (20180221220150).

Comment 63

[marius.santa]

Attachment: search.gif

I was able to reproduce the issue on Windows 10 64Bit Firefox  	53.0a1(20170101030204).
Tested and verified on Windows 10 64Bit, Ubuntu 16.04 LTS and Mac OS  X 10.13.2 in Firefox 60.0a1 (20180221220150).

Comment 64

[lilydjwg]

Hi there, searching in my local add-ons is really quite useful because I have 44 of them, and who can remember all full names of them (and some of them may have localized names)? Last time I wanted to change options of my recently installed RSS addon, it takes a lot of time to scan the list because I can't just search "RSS". (I had more than 70 addons before Firerfox 57. The number of installed addons will keep increasing unless there would be another large imcompatile change.)

I reported bug 1504396 requesting a way to search my installed addons but it was unhopefully closed as duplicate.