Compatibility Reporter API

RESOLVED FIXED in 5.2

Status

RESOLVED FIXED
10 years ago
3 years ago

People

(Reporter: fligtar, Assigned: lorchard)

Tracking

Details

(URL)

Attachments

(1 attachment)

(Reporter)

Description

10 years ago
An API is needed to accept and store compatibility reports for add-ons. This is the only function of the API.

Each report should have the following fields:
* add-on GUID (does not have to be hosted on AMO)
* application version
* application build ID
* operating system
* list of installed add-ons and their versions (optional)
* user's comments (optional)
* timestamp
* IP address (to prevent abuse)

The API should take these parameters and store them in the database.
(Assignee)

Comment 1

10 years ago
So, thinking that the parameters would be:

* guid - GUID for the addon in question
* worksProperly={0,1} - whether or not the addon worked properly
* appVersion - Release version of the browser
* appBuild - Build number for the browser
* clientOS - Name of the OS on which the browser runs
* comments - User-supplied comments
* otherGuid[0..n] - Indexed GUIDs of other installed addons
* otherVersion[0..n] - Indexex versions of other installed addons

Note that the indexes of otherGuid[0..n] and otherVersion[0..n] must be in sync to pair addon GUIDs with respective versions, but that shouldn't be too hard to assemble.

Probably the easiest way to implement this is just to accept plain old form POSTs at a URL (eg. application/x-www-form-urlencoded type request body). So, a POST request body something like (spaces added for wrapping):

guid={8675309} &worksProperly=0 &appVersion=3.5.3 &appBuild=20090824 &clientOS=Intel+Mac+OS+X+10.5 &comments=I+wish+this+addon+worked &otherGuid[0]={5551212} &otherVersion[0]=1.2.3 &otherGuid[1]={abcdef} &otherVersion[1]=5.6.7 &otherGuid[2]={wxyzhijk} &otherVersion[2]=9.9.9

The API will take care of coming up with timestamp and IP internally.

Do we have someone working on the Firefox end of things to CC on this and get feedback on the above?

Since it's mentioned that IP is collected to prevent abuse... how should that be done, exactly? Should I replace successive reports for a GUID from a given IP? Limit reports by IP?

Also, how will the lists of additional addons be used? I could store them as JSON blobs in a column with the rest of the report details, and stick with just a single table.  But, if you need to do reports / queries involving the additional addons, I'll need at least two tables to relate additional addons with primary reports.
(In reply to comment #1) 
> Probably the easiest way to implement this is just to accept plain old form
> POSTs at a URL (eg. application/x-www-form-urlencoded type request body).

Would it not be easier to accept JSON?  Particularly for sending other addons, in json that could be a list of (guid,version) pairs.
(Assignee)

Comment 3

10 years ago
(In reply to comment #2)
> (In reply to comment #1) 
> > Probably the easiest way to implement this is just to accept plain old form
> > POSTs at a URL (eg. application/x-www-form-urlencoded type request body).
> 
> Would it not be easier to accept JSON?  Particularly for sending other addons,
> in json that could be a list of (guid,version) pairs.

I'm not sure actually what would be easier for the Firefox side, so I could switch to JSON if someone on that side finds it easy.  Thus, I need feedback
(Assignee)

Comment 4

10 years ago
(In reply to comment #3)
> (In reply to comment #2)
> > (In reply to comment #1) 
> > > Probably the easiest way to implement this is just to accept plain old form
> > > POSTs at a URL (eg. application/x-www-form-urlencoded type request body).
> > 
> > Would it not be easier to accept JSON?  Particularly for sending other addons,
> > in json that could be a list of (guid,version) pairs.
> 
> I'm not sure actually what would be easier for the Firefox side, so I could
> switch to JSON if someone on that side finds it easy.  Thus, I need feedback

Though, FWIW, doing it as POST is the dead simplest in PHP since it's done automatically with every request - even turning the indexed params into arrays.  With JSON, you need to get the POST request body with php://input, run it through json_decode, and hope it's valid.

That said, JSON would be the better option in terms of data structures and standards.  And I think Fx has some native JSON encoding available.  In that case, I'd expect a POST request body something like:

{
    "guid": "{8675309}",
    "worksProperly": true,
    "appVersion": "3.5.3",
    "appBuild": "20090824",
    "clientOS": "Intel Mac OS X 10.5",
    "comments": "I wish this addon worked",
    "otherAddons": [
        ["{5551212}", "1.2.3"],
        ["{abcdef}", "5.6.7"],
        ["{wxyzhijk}", "9.9.9"]
    ]
}

I could go either way
(Reporter)

Comment 5

10 years ago
Brian and Dave will be cranking out the extension next week. Any comments on the  proposal?

As for what to do for multiple IPs, I don't think we need to do any pro-active stuff -- it's mainly for if we discover an IP that is obviously submitting tons of bogus reports we can exclude it in our queries.

And I would be fine with storing additional extensions in a non-indexable way.
(Reporter)

Comment 6

10 years ago
Sorry, one more field we probably need is the application GUID. I'm guessing other apps like Thunderbird will want to use this as well, so we should store that along with the app version and build.
(Assignee)

Comment 7

10 years ago
Haven't seen any feedback on the above yet, but will try to get some time in on this today / tomorrow.  I'm going to go with accepting a JSON request body, per comment 4, since I think that should actually be easiest on the Fx side using native JSON support.
(Assignee)

Comment 8

10 years ago
Okay, here's a first stab as the submission API.  This adds /compatibility/incoming, which accepts POSTs in JSON format that look like:

{
    "guid": "{8675309}",
    "worksProperly": false,
    "appGUID": "{555-555-1212}",
    "appVersion": "3.5.3",
    "appBuild": "20090824",
    "clientOS": "Intel Mac OS X 10.5",
    "comments": "I wish this addon worked",
    "otherAddons": [
        ["{5551212}", "1.2.3"],
        ["{abcdef}", "5.6.7"],
        ["{wxyzhijk}", "9.9.9"]
    ]
}

Adding jbalogh for r? since he piped up earlier.
Assignee: nobody → lorchard
Status: NEW → ASSIGNED
Attachment #405964 - Flags: review?(jbalogh)
Attachment #405964 - Flags: review?(jbalogh) → review+
Comment on attachment 405964 [details] [diff] [review]
Initial impl of report collection API

Looks good, tests are nice.  Make sure you add a migration to config/migrations.
Is this live anywhere yet for testing?

What's the full URL form? I presume something like this:

https://preview.addons.mozilla.org/en-US/firefox/api/1.3/compatibility/incoming
(Assignee)

Comment 11

10 years ago
This should be checked in as r53352, with the migration in r53353

The URL should be:

https://preview.addons.mozilla.org/en-US/firefox/compatibility/incoming

But, it doesn't look like it's working yet.  Hmm.  Probably needs the update to the DB table on preview.
The compatibility_reports table is on preview and I don't see any other sql.  Please make a migration script too.
(Assignee)

Comment 13

10 years ago
(In reply to comment #12)
> The compatibility_reports table is on preview and I don't see any other sql. 
> Please make a migration script too.

Is there something other than r53353 I need to check in for a migration script?

If the table's there, and it's still not working, is there a way to peek at the logs?  Getting a 404 that I assume is a squelched error.
(Assignee)

Comment 14

10 years ago
Scratch that, it worked for me:

$  curl -sD - -H'Content-Type: application/json' -X POST -d'{ "guid": "{8675309}", "worksProperly": false, "appGUID": "{555-555-1212}","appVersion": "3.5.3","appBuild": "20090824","clientOS": "Intel Mac OS X 10.5","comments": "I wish this addon worked", "otherAddons": [ ["{5551212}", "1.2.3"], ["{abcdef}", "5.6.7"], ["{wxyzhijk}", "9.9.9"] ]  }' https://preview.addons.mozilla.org/en-US/firefox/compatibility/incoming

HTTP/1.1 204 No Content
X-AMO-ServedBy: pm-app-amo24.mozilla.org
Content-Length: 0
Date: Thu, 15 Oct 2009 20:57:22 GMT
X-Cache-Info: not cacheable; request had a content length
Server: Apache/2.2.3 (Red Hat)
X-Powered-By: PHP/5.2.9
Content-Type: text/html; charset=UTF-8
Via: Moz-Cache-zlb02
(Assignee)

Updated

10 years ago
Status: ASSIGNED → RESOLVED
Last Resolved: 10 years ago
Resolution: --- → FIXED
(In reply to comment #13)
> (In reply to comment #12)
> > The compatibility_reports table is on preview and I don't see any other sql. 
> > Please make a migration script too.
> 
> Is there something other than r53353 I need to check in for a migration script?
> 

Nope, thanks
(Reporter)

Updated

10 years ago
Component: API → Compatibility Tools
QA Contact: api → compatibility
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.