492352 - Available Graphs data should be static, not dynamic, to improve initial load responsiveness

Status: RESOLVED FIXED

(Webtools Graveyard :: Graph Server, defect)

Description

[Johnathan Nightingale [:johnath]]

When the graph server first loads, a significant chunk of time is spent fetching the test data to populate the Branch, Test, Platform and Machine combo boxes in the "Available Graphs" section. Firebug suggests that the majority of this time is spent without network traffic, waiting for the DB to respond to the query, rather than in actually sending the data across.

I suspect it would be a significant responsiveness improvement if, instead, this data was just generated statically once-per-minute/5-minutes/hour and pulled intact by graph server, instead of being regenerated each page load.

Comment 1

[Ryan Doherty (:rdoherty)]

Attachment: v1

Here's a patch that does just this.

Steps:
* Make a directory called 'tmp' in server, chmod 777
* Run cron.py (from inside the scripts dir)
* Profit!

Most of the diff is moving some code around.

Comment 2

[Chris AtLee [:catlee]]

Comment on attachment 393267 [details] [diff] [review]
v1

Can I ask that the script not be called cron.py?

Comment 3

[Ryan Doherty (:rdoherty)]

(In reply to comment #2)
> (From update of attachment 393267 [details] [diff] [review])
> Can I ask that the script not be called cron.py?

Sure, we use maintenance.php for AMO, so maintenance.py?

Comment 4

[alice nodelman [:alice] [:anode]]

Comment on attachment 393267 [details] [diff] [review]
v1

I'm excited to see what sort of a speedup we get out of this.

Comment 5

[Ryan Doherty (:rdoherty)]

changeset a94f0d17b14f

Comment 6

[Chris AtLee [:catlee]]

What's blocking pushing this into production?

Comment 7

[Ryan Doherty (:rdoherty)]

(In reply to comment #6)
> What's blocking pushing this into production?

Nothing that I know of. I can file a bug to push this Thurs. night.

Comment 8

[Chris AtLee [:catlee]]

I don't think this is working properly in production.

Comment 9

[Ryan Doherty (:rdoherty)]

(In reply to comment #8)
> I don't think this is working properly in production.

I've reopened bug 515461

Comment 10

[Chris AtLee [:catlee]]

Ok, looking better.  Fetching /api/test now takes 6 seconds for me.  I notice there are no headers set on response that would allow the browser to cache the result, nor is the results compressed.  I think both of these would make the response on /api/test super snappy.

Comment 11

[Ryan Doherty (:rdoherty)]

(In reply to comment #10)
> Ok, looking better.  Fetching /api/test now takes 6 seconds for me.  I notice
> there are no headers set on response that would allow the browser to cache the
> result, nor is the results compressed.  I think both of these would make the
> response on /api/test super snappy.

I agree, and don't know why they response isn't gzipped as I thought it was previously.

Marking this as fixed and will open another bug for the 2 issues listed.

Comment 12

[Chris AtLee [:catlee]]

(In reply to comment #11)
> (In reply to comment #10)
> > Ok, looking better.  Fetching /api/test now takes 6 seconds for me.  I notice
> > there are no headers set on response that would allow the browser to cache the
> > result, nor is the results compressed.  I think both of these would make the
> > response on /api/test super snappy.
> 
> I agree, and don't know why they response isn't gzipped as I thought it was
> previously.
> 
> Marking this as fixed and will open another bug for the 2 issues listed.

This is bug 518015