489735 - Graphs take way too long to load

Status: RESOLVED FIXED

(Webtools Graveyard :: Graph Server, defect, P1)

Description

[Jesse Ruderman]

Loading the page in the URL field (4 mozilla-central Vista Ts graphs) takes 12 seconds.  Seems like 6s with an in-page spinner spinning, then 4s with Firefox unresponsive, then 2s drawing.

Tested with Firefox trunk on Mac (self opt build).

This may have some of the same causes as bug 466094 and bug 457273.

Comment 1

[Dietrich Ayala (:dietrich)]

Same problem on Windows and Linux -> OS = all.

Comment 2

[Dietrich Ayala (:dietrich)]

Anyone know who owns this code, and should work on it?

Comment 3

[Dietrich Ayala (:dietrich)]

Actually, the graphs seem to load quickly. It's the population of the dropdown fields in the top-left form that are blocking the load, from what I can tell.

Comment 4

[Ryan Doherty (:rdoherty)]

(In reply to comment #2)
> Anyone know who owns this code, and should work on it?

I've improved it in the past, but currently don't have the time to spend significant effort in improving it.

Comment 5

[andershol]

Attachment: Change data format

There seems to be two distinct problems:
- Page takes long to load, which seems to be what this bug, bug 552100 and somewhat also bug 547061 are about
- Long list of tests take long to populate and tear down, which seems to be what bug 466094 and bug 457273 are about

The load problems seems to be caused by a 17.9MB json file (gzip'ed to 457k). Worse this file gets that big because of a lot duplication. It is a join of 6 different tables, which takes a while to put together on the server, and on the client it is pulled apart again to populate the dropdowns.

In this patch format have been change to a somewhat more "normalized" format which takes up less space 581k (gzip'ed to 26k) and which can be used directly on the client. The format could be changed so the size drops to 241k (gzip'ed to 20k) if only unique "tests" arrays are transmitted, but that would probably cut down on readability. Also,
- The "Include non-recent"-checkbox was not implemented and therefore did nothing, have been removed (see also bug 446056)
- If all filters are manually set to "All", a text is displayed rather than trying to display all elements.
- ExtraDataTinderboxData have been removed since it was unused
- The sampledata.js have been removed (should be deleted)

I have run through the BFT from the wiki, and it seemed to work.

Comment 6

[andershol]

Attachment: Add foreign keys

I noticed that the schema.sql do not include foreign keys and there seems to be some data type inconsistencies between columns that presumably refer to each other. This probably doesn't help the optimizer create the best plans, allows for inconsistent data, and makes it harder to understand the model. This patch changes the data type of the referring column and adds some foreign keys.

Comment 7

[andershol]

Attachment: Schema upgrade script

This is a sql-script for changing the referring columns data-types to match that of the referenced columns and for adding foreign keys. Same as the preceding patch, but expressed as a sql-script.

Comment 8

[Robert Helmer [:rhelmer]]

http://graphs.mozilla.org is much faster now (new UI + caching)