Closed Bug 978736 Opened 6 years ago Closed 6 years ago

Implement a lightweight suggestion API for FxOS Rocketbar

Categories

(Marketplace Graveyard :: API, defect, P3)

Avenir
defect

Tracking

(Not tracked)

RESOLVED FIXED
2014-03-11

People

(Reporter: kgrandon, Assigned: mat)

References

Details

(Whiteboard: [qa-])

We are currently working on Rocketbar for FirefoxOS. Rocketbar provides fast, autocomplete results as users type into the search box. As this is an autocomplete, we need the response to be as light as possible, and only returning what the user needs to see. We should be able to return a much lighter response to rocketbar, giving the user a much better experience. We can further optimize this by tuning the icons and localized name to only return a single result based on the device/query params.

We were also doing some work on this here in the past: https://bugzilla.mozilla.org/show_bug.cgi?id=899779#c1

We currently display up to 4 marketplace results, using an endpoint such as: https://marketplace.firefox.com/api/v1/apps/search/?q=query&limit=4
The implementation from bug 899779 is still here:

https://marketplace.firefox.com/api/v1/apps/suggest/?q=query&limit=4&lang=en-US

If lang is not passed, an object containing all the translations is returned - also, the icon size is always 64x64 for the moment but we could tweak that.
See also bug 915701 to improve quality and performance of the results.

Is there anything else needed ?
(In reply to Mathieu Pillard [:mat] from comment #1)
> The implementation from bug 899779 is still here:
> 
> https://marketplace.firefox.com/api/v1/apps/suggest/?q=query&limit=4&lang=en-
> US

Hi Mat, that's awesome - I didn't realize that landed in production! I was just about to ping you here. Our requirements have changed slightly since the initial implementation - so this no longer fills our needs exactly. We could use this if we did the following:

- Remove the two fields: description and marketplace URLs.
- Add two fields: manifestURL, and slug.

We need the slug for opening the marketplace, and the manifestURL for de-duplicating results. As this is fairly specific to our needs, I'm not sure if it makes sense to propose this as an open search extension any more - but this could be really nice for Marketplace/Rocketbar.

Also if we are no longer proposing this as an open search standard, I wonder if it would be easier to simply make the response an array-of-arrays:

[
  [name1, icon1, slug1, manifestUrl1],
  [name2, icon2, slug2, manifestUrl2]
]

What do you think - do you think it would be feasible to make these changes? (Our path forward here is determined this time, so I'm sorry about the churn on this before)
Also for reference, here is where we fetch marketplace results today: https://github.com/mozilla-b2g/gaia/blob/master/apps/search/js/providers/marketplace.js
If we are no longer going the opensearch route, then I'd prefer more similar to what we have elsewhere in the API. So something like:

{
  "meta": {
    "next": null,
    "previous": null,
    "total_count": 2,
    "offset": 0,
    "limit": 25
  },
  "objects": [
    {
      "name": "Test App (kinkajou3969)",
      "slug": "test-app-kinkajou3969",
      "manifest_url": "http://kinkajou3969.testmanifest.com/manifest.webapp",
      "icons": "http://avatar/img/uploads/addon_icons/0/3-128.png?modified=1390229060"
    },
    {
      "name": "Clastsordbenver Rincal Britonzontack",
      "slug": "clastsordbenver-rincal-brit",
      "manifest_url": "http://localhost:8001/app/67c32073-f910-4ddd-b5f6-cc76582b81d1/manifest.webapp",
      "icons": "http://avatar/img/uploads/addon_icons/0/5-128.png?modified=1392075817"
    }
  ]
}

Slightly more verbose, but easier to maintain for us, and easier modify in the future. I fear going the array of arrays route is going to be too limiting.

This would be trivial to implement for me, I could have that ready for next week's push.
(In reply to Mathieu Pillard [:mat] from comment #5)
> If we are no longer going the opensearch route, then I'd prefer more similar
> to what we have elsewhere in the API. So something like:

This could work for us too. Really we just want to save as *many* bytes as possible. My preference would be to remove the "meta" container as we won't be using it the app. We could also potentially remove the "objects" key and just return an array of results. If this makes things harder, then we don't have to do this - the much slimmed response is still a huge win.

Also is it easier for you to use "icons" instead of "icon" as there is just one with this response?

(In reply to Mathieu Pillard [:mat] from comment #5)
> This would be trivial to implement for me, I could have that ready for next
> week's push.

That would be awesome - thanks for being so responsive! If you want to use this bug to track the work, feel free to do so. Thanks!
I opened https://github.com/mozilla/zamboni/pull/1827 ; I'll try to get it reviewed this week.

Details:
- The URL will be /api/v1/apps/search/rocketbar/
- The Content-Type will be application/x-rocketbar+json
- The results will be in the following format:
[{"name": "name1", "slug": "slug1", "manifest_url": "manifest_url1", "icon": "icon_url1"}, ...]

Notes:
- The icon size is 64 for now, let me know if you'd prefer 128.
- The manifest url will point to the Marketplace mini-manifest for packaged apps, like in the rest of our API responses.
- I also moved the old opensearch suggest API to /api/v1/apps/search/suggest/ in that commit for coherency.
Status: NEW → ASSIGNED
Priority: -- → P3
Summary: Consider implementing a lightweight API for FxOS Rocketbar → Implement a lightweight suggestion API for FxOS Rocketbar
Target Milestone: --- → 2014-03-11
You rock - the search results should be blazing fast after this. Thanks for quick response! :)
Fixed in https://github.com/mozilla/zamboni/commit/0179f65023b188a11c1c592cc25033b73d66d1de

I'll land in production next week (Tuesday 11 around noon PST). Meanwhile you can test on dev:
https://marketplace-dev.allizom.org/api/v1/apps/search/rocketbar/?q=puzzle&lang=en-US&limit=4

Be sure to include lang and limit parameters to get only one translation and as few results as you need.
Assignee: nobody → mpillard
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
I am not sure how to verify this one, can you please add some STRs or mark it as [qa-]?
Whiteboard: [qa-]
Depends on: 983568
:mat - just so you are aware, I tested this on a device and commented in the other bug, but here are the results we're seeing: 

In master today, each marketplace fetch and parse takes between 400 and 600ms. After some quick testing, the new api takes between 300 and 500ms, a definite improvement.

I'm looking forward to seeing what else we can do here, and will follow bug 915701. I know it's no easy feat, but I would love to see sub 100ms response times.
Kevin : bug 915701 might speed it up a little, yeah. My guess is that we can gain ~ 200ms there, by using ElasticSearch specific autocomplete API and having it return only what we need (currently I'm querying the main ES index, even though I'm only returning to you the minimum that you need, ES is sending me the whole result with all fields)

A bigger improvement would come from bug 957387 but I don't know when we'll get to that.

One thing you can do right now is specifying the region in the querystring, or "region=None" to disable region filtering completely. I'm on IRC if you want to discuss this because it's complex. Quick summary: Right now, without that parameter, we are going through geoip to try to find your region and filter accordingly.

If you want to pass the region, here is what we are doing on the Marketplace front-end: https://github.com/mozilla/fireplace/blob/master/hearth/media/js/mobilenetwork.js (at the moment you can ignore the carrier parameter - we don't currently filter with it)

If you want to disable it, passing region=None, be aware that the apps you'll get might not be available for the user (either because it requires payments - which can be impossible for their region - or simply because the developer has chosen to exclude the app from their region)
Awesome. Thanks for the info mat. If you don't mind I'm going to create a meta bug for Rocketbar performance, and block on that so we can track these issues. Maybe we can even help out with some resources if we have the bandwidth.
Blocks: 983615
You need to log in before you can comment on or make changes to this bug.