753686 - 15,000 DOM elements and 45 seconds worth of GET requests on a single page for Firefox topcrashers

Status: VERIFIED FIXED

(Socorro :: Webapp, task)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

There are over 15,000 DOM elements on https://crash-stats.mozilla.com/topcrasher/byversion/Firefox/12.0/14 and its ilk -- do we really need to display 300 signature at once?

Additionally, it takes 45 seconds to retrieve the accompanying dataset:

[02:03:45.764] GET https://crash-stats.mozilla.com/buginfo/bug?id=633253,633253,723894,705159,640904,723894,705159,640904,743221,616117,722083,610551,711568,743221,616117,722083,610551,711568,633253,694874,633253,694874,572011,572011,728193,744727,720956,716900,595351,707038,728193,744727,720956,716900,595351,707038,729368,716232,655561,655562,654903,729368,716232,655561,655562,654903,722101,702531,722101,702531,740315,671468,740315,671468,574996,597260,574996,597260,630197,630197,675260,711847,675260,711847,730703,717909,700176,730703,717909,700176,723894,640904,723894,640904,627238,547588,530074,627238,547588,530074,725503,725503,719114,719114,720991,714187,720991,714187,500105,569688,500105,569688,673543,631008,673543,631008,558757,679656,551163,500103,483480,558757,679656,551163,500103,483480,674568,708209,674568,708209,633253,633253,654903,654903,597262,597262,603070,603070,707078,655565,655527,654877,707078,655565,655527,654877,734948,595375,734948,595375,599354,599354,744836,744836,664764,664764,730703,705423,730703,705423,731692,706104,731692,706104,637596,637596,729507,729507,700493,740330,700493,740330,660265,660265,555319,520554,526483,555319,520554,526483,728533,639405,568487,728533,639405,568487,692735,682916,692735,682916,725280,709860,725280,709860,561264,561264,738279,752379,751879,735032,723445,676023,679251,746076,676571,738279,752379,751879,735032,723445,676023,679251,746076,676571,715057,688971,683317,715057,688971,683317,717882,657573,717882,657573,738661,738661,679787,696640,538267,642088,606642,679787,696640,538267,642088,606642,714581,714581,736849,736849,671247,671247,710559,709479,710559,709479,707091,707091,660272,660272,695029,622411,531961,437449,695029,622411,531961,437449,725009,725009,726028,700499,182803,726028,700499,182803,719113,719113,524944,524944,718284,606268,644637,718284,606268,644637,750180,750180,626631,595975,615492,602223,626631,595975,615492,602223,554660,554660,714625,714625,719117,719117,734921,734921,715057,701229,701399,715057,701229,701399,741179,741179,656758,656758,738653,738653,646839,634913,646839,634913,519752,519752,705377,705377,699040,699040,663466,663466,614966,614966,668583,668583,591695,591695,746939,746939,704124,704124,747847,747847,418384,558925,585975,418384,558925,585975,558989,558989,731812,731812,718947,718947,723458,723458,738661,738661,699683,706501,513224,485941,679905,568460,642801,519726,699683,706501,513224,485941,679905,568460,642801,519726,709482,680921,537828,538628,709482,680921,537828,538628,682573,682573,726114,726114,600150,600150,543665,543665,706442,706442,612120,612120,723471,723471,707613,707613,719110,719110,610103,590078,608351,655500,672766,585660,556483,516182,522070,531551,569498,548002,501429,610103,590078,608351,655500,672766,585660,556483,516182,522070,531551,569498,548002,501429,708206,708206,749595,614776,749595,614776,601587,601587,723462,723462,548685,677643,548685,677643,690071,690071,614966,614966,723447,723447,688924,723133,680862,657324,630697,638756,531341,688924,723133,680862,657324,630697,638756,531341,713672,713672,736830,706438,736830,706438,633253,633253,706045,706045,751583,751583,508292,508292,728193,744727,728193,744727,751162,751162,746713,746713,734167,674854,571289,597287,734167,674854,571289,597287,622140,622140,633253,565708,633253,565708,716390,716390,748944,748944,622140,622140,699789,699789,711656,730884,711656,730884,579558,579558,713025,713025,643840,643840,725865,712302,725563,681205,633845,630243,725865,712302,725563,681205,633845,630243,712248,712248,639337,591513,639337,591513,633668,633668,723190,616995,723190,616995,682839,682839,718580,681633,718580,681633,737509,715097,681114,612096,737509,715097,681114,612096,606616,606616,575010,575010,624136,624136,726114,726114,738661,738661,673033,605765,673033,605765,751682,751682,750069,653863,591457,581946,523405,750069,653863,591457,581946,523405,644696,644696,633445,633445,752387,654196,752387,654196,682041,641893,682041,641893,716843,716843,730035,719116,730035,719116,710160,620122,710160,620122,718991,718991,600807,600807,699789,699789,652202,650519,652965,652202,650519,652965,518811,518811,750190,672870,750190,672870,747349,747349,614773,614773,556524,530962,556524,530962,721196,721196,733325,733324,733323,733322,589546,733325,733324,733323,733322,589546,436817,677400,520650,519616,515922,494107,436817,677400,520650,519616,515922,494107,675195,675195,637308,637308,652065,500879,652065,500879,688895,688895,680954,680954,609243,609243,752845,752845,627706,627706,614582,614582,725632,721710,702250,725632,721710,702250,637308,637308,622978,622978,709847,709847,719106,719106,655905,655905,730470,714579,730470,714579,619332,619332,742414,616421,742414,616421,723464,723464,626199,626199,622140,622140,750192,750192,662330,662330,733813,733813,618421,618421,590106,590106,724234,724234,705641,705641,680862,690010,664974,630697,591599,545657,680862,690010,664974,630697,591599,545657,608128,608219,608128,608219,628236,628236,721437,711643,627712,721437,711643,627712,736695,561277,485941,745142,736695,561277,485941,745142,598896,545244,598896,545244,728960,724374,633123,728960,724374,633123,695524,695524,735274,735274,694344,694344,632917,632917,606455,606455,636957,636957,716232,750783,716232,750783,614966,614966,717568,717568,737942,737942,726206,726206,658780,658780,644132,644132,636600,636600,&include_fields=summary,status,id,resolution [HTTP/1.1 200 OK 45973ms]

Can we get some pagination, or tighter queries?

Comment 1

[Robert Kaiser]

(In reply to Stephen Donner [:stephend] from comment #0)
> ilk -- do we really need to display 300 signature at once?

We often want to find-in-page to see on which position a particular signatures is in this version - or if it's in the top 300 at all.
If there's some other good way to do this, pagination and only showing 100 at once is good, but fewer than 100 on a page tend to become unhelpful fast when investigating crashes.
What we could do is to have a selector where we can show the top 50/100/200/300, and in that case, we might even get away with having 50 by default.

Comment 2

[David Burns :automatedtester]

This is causing issues with automation of the Socorro and other webqa automation since we are maxing out the capabilities of the browser when it hits an error.

We move things to a canvas and then get the image from that when taking the screenshot. We are also trying to grab the html which means we are having a bad time.

Can we default to a small pagination set and if people want to have a bigger set then they can. This will vastly improve WebQA's ability to test the application.

@Laura is this something that we can solve in the near future?

Comment 3

[Laura Thomson :laura]

Still a problem.  Need to flatten this out a bit.

Comment 4

[Peter Bengtsson [:peterbe]]

The biggest win would be to use CSS sprites and to do a better job caching the bugzilla titles

http://tools.pingdom.com/fpt/#!/djpCaE/https://crash-stats.mozilla.com/topcrasher/products/Firefox/versions/25.0a2?days=7

Comment 5

[Peter Bengtsson [:peterbe]]

I suggest we use localStorage to store the bugzilla titles but it'll need to be async and it'll need a TTL.

Comment 6

[Robert Kaiser]

See also my earlier comment:

(In reply to Robert Kaiser (:[e-mail]) from comment #1)
> What we could do is to have a selector where we can show the top
> 50/100/200/300, and in that case, we might even get away with having 50 by
> default.

Comment 7

[[github robot]]

Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/a971ddb4c363b21b0fcfedfee5260989eba02f87
Fix Bug 753686, enable results count filter with a default of 50

https://github.com/mozilla/socorro/commit/fa55a8022e0a827d44280a44edec201fd8b49302
Merge pull request #1683 from ossreleasefeed/bug753686-enable-filter-to-select-number-of-results-tcbs

Fix Bug 753686, enable results count filter with a default of 50

Comment 8

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED; while we haven't addressed the # of DOM elements, per se, we have reduced the queries-on-hand to 50, by default, which takes the page load down to ~12 seconds + number of elements rendered by default to ~2900; see full report, below:

http://www.webpagetest.org/result/131121_JX_2QB/

Updated URL from original report, since we've shipped (bug 941259), on prod, is:

https://crash-stats.mozilla.com/topcrasher/products/Firefox/versions/25.0/date_range_type/report/crash_type/browser/os_name/None/result_count/50?days=14

Comment 9

[Stephen Donner [:stephend] Not actively reading bugmail]

Attachment: Post-fix screenshot of prod