608813 - Migrate blocklist to zamboni

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Public Pages, defect, P3)

Description

[Wil Clouser [:clouserw]]

Blocklist URLs follow the format:

https://addons.mozilla.org/blocklist/3/%APP_ID%/%APP_VERSION%/%PRODUCT%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/

Our blocklist is a single standalone script in PHP at:

http://viewvc.svn.mozilla.org/vc/addons/trunk/site/app/webroot/services/blocklist.php?view=markup

About half the code is test stuff - that should be moved into our new test suite w/ fixtures and such.  This is a very high traffic script so don't do anything that takes much computation or that can't be cached.

This is also our blocklist for every bad add-on we know about - it's pretty important we don't mess it up.  When in doubt, go overkill on tests.  

The only UI is the built in django admin stuff at blocklistapp and blocklistitem.

More info on blocklist: https://wiki.mozilla.org/Blocklisting
How to test current code: https://wiki.mozilla.org/Extension_Blocklisting:Testing
Current unit tests: http://svn.mozilla.org/addons/trunk/site/app/tests/services/blocklist.test.php

Comment 1

[Justin Scott [:fligtar]]

I've been using the django admin stuff to blocklist things and it works pretty well. Note that the new gfx driver block stuff that Les added to PHP does not have a django admin tool yet.

Comment 2

[Wil Clouser [:clouserw]]

(In reply to comment #1)
> I've been using the django admin stuff to blocklist things and it works pretty
> well. Note that the new gfx driver block stuff that Les added to PHP does not
> have a django admin tool yet.

Should be easy to flip on.  It's just a new bl table

Comment 3

[Jeff Balogh (:jbalogh)]

https://wiki.mozilla.org/Extension_Blocklisting:Code_Design

Comment 4

[Jeff Balogh (:jbalogh)]

My fx4 about:config has this:

https://addons.mozilla.org/blocklist/3/%APP_ID%/%APP_VERSION%/%PRODUCT%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/%PING_COUNT%/%TOTAL_PING_COUNT%/%DAYS_SINCE_LAST_PING%/

Comment 5

[Jeff Balogh (:jbalogh)]

Attachment: blocklist.xml

My current blocklist: https://addons.mozilla.org/blocklist/3/%7Bec8030f7-c20a-464f-9b0e-13a3a9e97384%7D/4.0b12pre/Firefox/20110217/Darwin/en-US/release/Darwin/default/default

Comment 6

[Jeff Balogh (:jbalogh)]

Primary: https://github.com/jbalogh/zamboni/commit/b59ef9a 
Cachingly: https://github.com/jbalogh/zamboni/commit/ae4bf67 
Mergingly: https://github.com/jbalogh/zamboni/commit/e917639

Comment 7

[krupa raj[:krupa]]

Moving this to 5.12.12 as there is not enough time for QA

Comment 8

[Wil Clouser [:clouserw]]

The loadtest says this is pretty good.  Once it slowed down for around 1 minute, and every request (total, probably 30) took 30-45s to complete.  It also generated a couple "Can't connect to mysql" errors - I assume they are related.

I think on the average it's fine though:

Remora:
> Good: 156673
> Max/Avg/Min/StdDev: 169ms/36ms/6ms/43ms
> Bad: 46
> Total Time: 434.149641037
> Req/s: 342.828798947

Zamboni:
> Good: 193290
> Max/Avg/Min/StdDev: 1655ms/310ms/22ms/330ms
> Bad: 60
> Total Time: 440.036703825
> Req/s: 417.294735652

Comment 9

[krupa raj[:krupa]]

This needs more QA. Bumping to the next milestone.

Comment 10

[krupa raj[:krupa]]

Bug 640230 tracks the QA efforts for this feature. Removing [QA+]