Closed Bug 624095 Opened 14 years ago Closed 14 years ago

Harder. Better. Faster. Stronger. /update.

Categories

(addons.mozilla.org Graveyard :: Code Quality, defect, P3)

Product:
addons.mozilla.org Graveyard
Component:
Code Quality
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
5.12.10

People

(Reporter: clouserw, Assigned: andy+bugzilla)

Details

I'm sorry to report that /update is a slug compared to its crazy optimized PHP twin. PHP is first: Good: 70543 Max/Avg/Min/StdDev: 825ms/197ms/48ms/186ms Bad: 0 Total Time: 121.608709812 Req/s: 551.070725966 Good: 17151 Max/Avg/Min/StdDev: 4336ms/1936ms/557ms/786ms Bad: 0 Total Time: 144.322224855 Req/s: 112.893215278
I have horrible feeling that jbalogh's changes didn't get merged on to trunk that sped this up dramatically. We can also speed things up by not doing an session lookup on this URL, so will add that in.
Status: NEW → ASSIGNED
(In reply to comment #0) > I'm sorry to report that /update is a slug compared to its crazy optimized PHP > twin. How did you test this? What's the baseline speed of /pages/faq (which should be pretty fast without any lookups)?
I spoke to wil about this Jeff and have the details. Results are at: https://wiki.mozilla.org/User:Clouserw/AMO/loadtest But looking at the code some of those changes you made on 2010-11-30 aren't in. I can't find the commit either. That's the first big win.
Let me know if you want to land some code and fire off another test. @jeff: I replayed 100k lines of logs for this. I expected it to start slow and get super fast but, as you can see, the average was very high. For the record, this was with 252 concurrent connections (what I've run all the tests at).
The distribution was quite varied a few thousand rows gave this distribution in terms of number of queries. http://dpaste.com/304268/ I've got some checkins that speed a few general things up, but nothing major yet. The biggest single difference is stopping the transforms on getting the Addon.
Good: 15562 Max/Avg/Min/StdDev: 3785ms/1644ms/457ms/632ms Bad: 0 Total Time: 117.27315402 Req/s: 126.056130437
We need to figure out a solution for this.
Target Milestone: 5.12.6 → Q1 2011
fwiw https://github.com/jbalogh/zamboni/commit/9c5b3716640c1b41948895c1687c94620c1bdc24 ...comes in quite a lot faster (19 req/sec vs 50 req/sec) locally. Doubt that we'll get that much boost of on amo, but it wouldn't take long just to grab and fix that code and have a quick peek at the result. Any complaints if I did that?
That's the original commit so I'm not sure what you're proposing, but do what you need to get this going (as long as you don't break tests!).
The original commit was never load tested, would just like to compare. There a few failing unit tests I'm commenting out or fixing since this isn't going to go live, just get my code onto the preview environment.
Let me know when you want me to retest.
Priority: -- → P3
So I don't forget, after removing the max-request=200 from WSGIDaemonProcess today we got to: Req/s: 210.209181788 But still saw pauses in the traffic.
Moving back to .10 to keep it on the radar. What are the next steps?
Target Milestone: Q1 2011 → 5.12.10
The lightest possible python script that can run under mod_wsgi so that oremj can run it under a different mod_wsgi settings from the rest of a.m.o without breaking all of a.m.o.
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.