417424 - AMO Manager: Multiple requests for recommended list on first opening

Status: VERIFIED FIXED

(Toolkit :: Add-ons Manager, defect, P2)

Description

[Michael Morgan [:morgamic]]

Steps to reproduce:
1) open add-ons manager for the first time
2) go to get add-ons tab
3) watch http traffic

The service URL seems to be requested multiple times (often 6, sometimes more):
https://services.addons.mozilla.org/en-US/firefox/api/list/featured/all/10

Image here:
http://people.mozilla.com/~morgamic/samo_requests.png

Had 3 people report same thing, so marking NEW -- could someone take a look?  This could be important for AMO scalability if it is indeed sending multiple requests.

Less of a problem if bug 415277 could be fixed, but a separate issue nonetheless.

Comment 1

[Wil Clouser [:clouserw]]

I'm not totally convinced this is happening, but I agree that I see the same results as you using the tamper data extension (which is not marked compatible with ff3...).

To reproduce my reason for doubting:

1) open add-ons manager, get recommended list
2) close add-ons manager
3) re-open add-ons manager.  list is loaded from cache.

On step 3, tamper data shows me X amount of requests (where X is apparently random but less than 10), but tcpdump shows no packets going to AMO.  On initial startup and when I do a search tcpdump does show the packets.  It seems like tamperdata is malfunctioning to me.

Comment 2

[Dave Townsend [:mossop]]

It is happening because I wrote it to happen. The goal is to find 5 displayable items. So the feature requests 10 in the hope that 5 in there will be suitable (finger in the air generated number). If it doesn't find 5 items it requests another 10 from the recommended list. It keeps doing this until it stops finding new items to display.

This was all written when I was of the impression that the recommended list would spit out random results for each call. In actual fact it is cached so we should be seeing the same list each time. Because of this we could just drop the additional calls and give up after the first request. The hope was that once extensions were updated to Firefox 3 no more than one request would be needed anyway.

Comment 3

[Michael Morgan [:morgamic]]

What if you passed client and version info to the API so the API could only return compatible add-ons?  It would make a more complex query but you'd only have to do it once.  Other services pass more info for that reason (extension updates, AUS, etc.).  Is that plausible?

Comment 4

[Dave Townsend [:mossop]]

I already am passing the client name but version is certainly possible, it should just be a pref change. The other things that we filter on are OS (which we could pass as well) and then we filter out sandboxed items and items that the user already has installed (the last of these we would always have to do client side).

Comment 5

[Michael Morgan [:morgamic]]

In cases like this with in-client URLs we've always erred on the side of passing more info than less, so I would be for adding additional info if it helps reduce the # of requests and the ambiguity of API results.  Laura -- what do you think?

Comment 6

[Mike Beltzner [:beltzner, not reading bugmail]]

Blocking until we can determine if this is accidentally overloading our servers and ironically preventing us from serving any add-ons.

Comment 7

[Dave Townsend [:mossop]]

Regardless of whether the repeated calls are overloading the server they are pointless as long as the result is cached so we may as well remove it.

I'd be tempted to up the number of results requested for the one off call to maybe 20, don't know what you think of that morgamic?

Lets do the adjustment to the API taking the app version into account separately.

Comment 8

[Dave Townsend [:mossop]]

Attachment: patch rev 1

Dump the repeated requests, just go with whatever we get first time around.

Comment 9

[Dave Townsend [:mossop]]

Attachment: patch rev 2

This is as the previous patch however it drops the now useless global uri field.

Comment 10

[Dave Townsend [:mossop]]

Attachment: patch rev 3

Sorry, don't need to track oldcount anymore either.

Comment 11

[Dave Townsend [:mossop]]

Checking in toolkit/mozapps/extensions/src/nsAddonRepository.js;
/cvsroot/mozilla/toolkit/mozapps/extensions/src/nsAddonRepository.js,v  <--  nsAddonRepository.js
new revision: 1.2; previous revision: 1.1
done

Comment 12

[Tony Chung [:tchung]]

hi mike, wil,

can you verify this has been fixed in your service URL tool?  Thanks.

Comment 13

[Michael Morgan [:morgamic]]

I don't see this issue any more.

Comment 14

[Michael Morgan [:morgamic]]

Wil - reopen if you see otherwise.