630287 - periodic AMO API call errors in PHX web servers

Status: RESOLVED INVALID

(Socorro :: General, task)

Description

[Robert Helmer [:rhelmer]]

We periodically cannot connect to services.addons.mozilla.org from the web heads:

2011-01-31 11:46:11 -08:00 --- error: `soc.web extensions.getExtensionsFromAMO` 
- AMO API Call https://services.addons.mozilla.org(...) only returns 1 results.  Expected 24 results.

This does not look like a new problem since I see this in SJC kohana logs too.

stephend noticed that sometimes the "correlation" column on the topcrasher report page of crash-stats times out, I think that this is why.

Comment 1

[Stephen Donner [:stephend] Not actively reading bugmail]

The JSON calls are initiated from, e.g. https://crash-stats.mozilla.com/topcrasher/byversion/Firefox/3.6.3, and are POSTS that expect a list of signatures:

https://crash-stats.mozilla.com/correlation/bulk_ajax/addon/Firefox/3.6.3/

Comment 2

[Laura Thomson :laura]

Is this a bug for us, or for AMO?

Comment 3

[Robert Helmer [:rhelmer]]

(In reply to comment #2)
> Is this a bug for us, or for AMO?

I think it's probably AMO, but wanted to check on our side first.

Comment 4

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Per getExtensionsFromAMO() that is an error that we log when a report has X extensions and when we make the request to AMO it only returns Y results.  The problem is that people have installed addons that have not been registered with AMO.

This is logging as a Kohana error, it maybe should be logging as info instead of error to reduce log severity.

Comment 5

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

If the http status code does not return a 200, then it should log as an error.

Comment 6

[Laura Thomson :laura]

It returns 200, so I'm ok with INFO.

Comment 7

[Stephen Donner [:stephend] Not actively reading bugmail]

I'm seeing this 100% of the time, right now, on https://crash-stats-dev.allizom.org/topcrasher/plot_signature/.

Comment 8

[Peter Bengtsson [:peterbe]]

We're using Hive these days.