(In reply to Jonathan Kingston [:jkt] from comment #9) > [...] > I downloaded the zip that the proxy service uses and from what I can tell nothing was using appcache. > [...] > Some questions: > - If the problem is indeed that tumblr or a subresource used appcache can we update the test as it doesn't look like they still do. If the downloaded zip wasn't hinting anything about our recordings using appcache, then probably this isn't something trivial to check. Still, we need to know for sure if these recordings are using appcache or not. Please try and simulate something at Firefox source code level, so we can tell if the appcache mechanics are hit when running the affected perf tests. > - If they do on live (I don't have an account to verify their account pages right now), can we accept the performance hit. If we can properly confirm that the appcache mechanics are hit, then we can consider accepting the performance hit. > - Is there anything else that might introduce this bimodality? Not really. We've done many retriggers & concluded that bug 1237782 brought the bimodality. > I ran the tests again a few times: > > Inbound (https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=269781247&revision=4c706780fe5e7ddf1bc781edcd2c41444a648999): > 667.98, 743.64, 728.74, 621.98, 733.17, 641.49, 720.56 > > Inboud with patches (https://treeherder.mozilla.org/#/jobs?repo=try&revision=7579cdd52e02def2baf447fc861fe00489d3cac4): > 738.8, 688.22, 660.91, 642.16, 713.79, 727.71, 755.35, 745.04 > > Inbound with backout (https://treeherder.mozilla.org/#/jobs?repo=try&revision=708c015b2805a70c1b185bb4592f9f6a013caa69): > 697.62, 700.58, 619.79, 699.12, 610.98, 608.65, 706.36, 784.55 > > I don't see anything obvious in any of these, maybe try isn't stable enough for perf testing this? I don't think there are issues with simulating this on Try. We checked the noise levels, which are similar with what we have on trunk branches.
Bug 1585582 Comment 20 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
(In reply to Jonathan Kingston [:jkt] from comment #9) > [...] > I downloaded the zip that the proxy service uses and from what I can tell nothing was using appcache. > [...] > Some questions: > - If the problem is indeed that tumblr or a subresource used appcache can we update the test as it doesn't look like they still do. If the downloaded zip wasn't hinting anything about our recordings using appcache, then probably this isn't something trivial to check. Still, we need to know for sure if these recordings are using appcache or not. Please try and simulate something at Firefox source code level, so we can tell if the appcache mechanics are hit when running the affected perf tests. > - If they do on live (I don't have an account to verify their account pages right now), can we accept the performance hit. If we can properly confirm that the appcache mechanics are hit, then we can consider accepting the performance hit. > - Is there anything else that might introduce this bimodality? Not really. We've done many retriggers & concluded that bug 1237782 alone brought the bimodality. > I ran the tests again a few times: > > Inbound (https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=269781247&revision=4c706780fe5e7ddf1bc781edcd2c41444a648999): > 667.98, 743.64, 728.74, 621.98, 733.17, 641.49, 720.56 > > Inboud with patches (https://treeherder.mozilla.org/#/jobs?repo=try&revision=7579cdd52e02def2baf447fc861fe00489d3cac4): > 738.8, 688.22, 660.91, 642.16, 713.79, 727.71, 755.35, 745.04 > > Inbound with backout (https://treeherder.mozilla.org/#/jobs?repo=try&revision=708c015b2805a70c1b185bb4592f9f6a013caa69): > 697.62, 700.58, 619.79, 699.12, 610.98, 608.65, 706.36, 784.55 > > I don't see anything obvious in any of these, maybe try isn't stable enough for perf testing this? I don't think there are issues with simulating this on Try. We checked the noise levels, which are similar with what we have on trunk branches.