In order to power the various recommended/similar/"others like this" sections of add-ons, we need to have a cron job that updates add-on relationships with other add-ons. This formula can evolve into something pretty complicated, but for this first phase, we're only going to use collections and a fairly simple ranking sort. For each add-on, we can identify its related add-ons based on the number of collections they are in together. So, for example, Adblock Plus is in 8622 collections, and the following add-ons are also in some of those same collections: NoScript - 2047 overlapping collections Adblock Plus: Element Hiding Helper - 1292 Flashblock - 1109 Password Exporter - 296 For each add-on, we should store the top 10 related add-ons in order of descending overlapping collections.
(In reply to comment #0) > In order to power the various recommended/similar/"others like this" sections > of add-ons, we need to have a cron job that updates add-on relationships with > other add-ons. This sounds like fun! And potentially very cpu intensive. > This formula can evolve into something pretty complicated, but for this first > phase, we're only going to use collections and a fairly simple ranking sort. Can we consider doing this in Not PHP? It's not exactly a good...platform for doing algorithmic things like this.
It's going to be a cron job in /bin/. If you take the bug you can write it in anything that's already installed on the boxes. ;) AMO boxes have python 2.6! (please no more config files, thx)
Jeff did a prototype of this and I want to make some changes based on the results. I will post the new formula in the next couple of days.
Assignee: nobody → fligtar
I have a better formula now with lots of generated data; I'll add that onto the jbalogh.khan pages tomorrow so we can see if we like it.
Ok, check out the right sidebar for recommendations on add-on detail pages, e.g. http://jbalogh.khan.mozilla.org/amo/site/en-US/firefox/addon/12025 I ran the ranking formula against the cross product of all the add-ons, and for each add-on kept the top 20 (or less) correlated add-ons with a score > 0.1. The correlation score runs between 0 and 1, and 0.1 is an arbitrary limit. This is my ridiculously awesome ranking formula: def simple(ratings, one, two): a, b = ratings[one], ratings[two] matches = len(a.intersection(b)) return (2 * matches) / float(len(a) + len(b)) And now for some mathematical notation: (2 × ⎢A ∩ B⎥) ∕ (⎢A⎥ + ⎢B⎥), where A and B are sets of collection ids containing add-on A and B. (Hope bugzilla doesn't choke on that.)
Bugzilla loves symbols
This is way better than the first one. For add-ons in a specific area, the matches are quite good. Super-popular add-ons that don't have a great category seem to have more generic matches, but I don't know that we'll ever be able to avoid that. I like it!
Assignee: fligtar → jbalogh
This formula can also be applied to collections pretty easily. I think that's a more interesting space since comparing 12 or so add-ons together is more meaningful than comparing 1 add-on at a time. Are we interested in doing that now?
Created attachment 393036 [details] [diff] [review] add-on recommendations on display pages I threw fixes for bug 506800 and bug 506158 together in this patch so it's easier for you see if the recommendations are working. Highlight of the patch: a shell script that calls a php script to get mysql connection strings so it can write query data to a file then call a python script to munge that data into a sql command that is sent back with another mysql connection string.
Attachment #393036 - Flags: review?(clouserw)
So you're saying we already have the data for similar collections, and we could just put a box there with them?
I'm saying that I can flip the input data from addon -> [collections] to collection -> [addons] and apply the same algorithm to get "other collections like this one" recommendations. With a little more thinking we could have "other addons that would be nice in this collection".
Comment on attachment 393036 [details] [diff] [review] add-on recommendations on display pages in connection.php read_config and write_config aren't class variables so this file doesn't work at all. That said, echoing out a user/pass on the command line is not good. Can you use the same code you wrote for migrations to grab the user/pass from the config?
Attachment #393036 - Flags: review?(clouserw) → review-
Created attachment 393623 [details] [diff] [review] v2, with mysql-python You'll have to install mysql-python to get the script working. I'm not sure where to put a README, so here's I would do. Let me know if you need more instructions. 1. Get http://pypi.python.org/pypi/virtualenv and http://www.doughellmann.com/projects/virtualenvwrapper/ 2. Follow the instructions on http://www.doughellmann.com/docs/virtualenvwrapper/, create a virtualenv: mkvirtualenv amo workon amo 3. Install the installer easy_install pip 4. Install what we need pip install -r python-requirements.txt
Comment on attachment 393623 [details] [diff] [review] v2, with mysql-python - Fix L10n to use the normal .po style in display.thtml - remove recommendations.sh - python doesn't run, talked on IRC. :(
Attachment #393623 - Flags: review?(clouserw) → review-
Created attachment 394921 [details] [diff] [review] v3 Fixes .po, py26, recs.sh.
Comment on attachment 394921 [details] [diff] [review] v3 r+ after things we talked about on IRC: - build-*.py fix - CSS fix squishing our collection box
Attachment #394921 - Flags: review?(clouserw) → review+
Also, this is pretty sweet.
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
I don't see the related add-ons in preview for the add-on "Firebug".(https://preview.addons.mozilla.org/en-US/firefox/addon/1843). Am I missing something?
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
You won't until bug 511321 is fixed
Status: REOPENED → RESOLVED
Last Resolved: 9 years ago → 9 years ago
Resolution: --- → FIXED
(In reply to comment #20) > You won't until bug 511321 is fixed So, like, not until after we push to prod :-)
The recommendations have been showing up since yesterday.
So they have; Verified FIXED.
Status: RESOLVED → VERIFIED
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.