708270 - Flash endurance tests expose potential performance degradation regression

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect, P3)

Description

[u279076]

When running the Flash-based endurance tests with a high number of iterations and entities, the page-load performance degrades over time. Page-loads start around 5s and degrade to over 30s after the test has been running for 30 minutes. This is not reproducible manually so this may be a regression only Endurance tests can see.

The purpose of this bug is to investigate where this regression exists (Mozmill, Endurance framework, or Firefox) and escalate for a fix.

Comment 1

[u279076]

This bug blocks all of our Flash-based endurance tests. Dave, Henrik, any advice how we can debug this issue?

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #1)
> This bug blocks all of our Flash-based endurance tests. Dave, Henrik, any
> advice how we can debug this issue?

Run our usual steps for regression tests of Firefox and find out if it is a regression on a specific branch and on which platforms. I do not think it's anything related to Mozmill here. Also watch out the memory usage while running this test.

Comment 3

[u279076]

Could we potentially be looking at something related to bug 689121?

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Dave, do we have an update here? Is that still valid?

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I don't have an update. Perhaps we can revisit the blocked tests to see if this is still an issue.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Lets get this added for at least an investigation -> P3.

Comment 7

[Daniela Petrovici]

Attachment: memory log (MB)

You can find the memory consumption logs attached. These are the steps I have performed in order to test this issue:
1) default delay of endurance tests set to 5 seconds
2) Ran 8 times the endurance testruns on MAC OS 10.7.5 with:
a) Nightly 22.0a1 - without the patches from blocked bugs
b) Nightly 22.0a1 - with the patches from blocked bugs
c) Aurora 21.0a1 - without the patches from blocked bugs
d) Aurora 21.0a1 - with the patches from blocked bugs

I have added the memory in MB into a file and displayed it every hour (since a single endurance run takes 1.5 hours)

2) delay of 0.1 seconds as it was previously when this bug was logged, but I am unable to reproduce the issues in the two dependent bugs. I will try with a heavy loaded system as it still is on the MAC machines. 

The tests were performed on Nightly 22.0a1 with and without the patch from the two dependent bugs.

Except for running endurance testruns on a heavy loaded system with delay 0.1 seconds. Can you please tell me if there is anything else I need to investigate here?

Comment 8

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I don't think anything else is needed here. I suspect the performance improvements in Firefox have helped us here. It looks like we should go ahead and get these remaining tests in. Thanks Daniela!

Comment 9

[Daniela Petrovici]

(In reply to Dave Hunt (:davehunt) from comment #8)
> I don't think anything else is needed here. I suspect the performance
> improvements in Firefox have helped us here. It looks like we should go
> ahead and get these remaining tests in. Thanks Daniela!
I have updated both bugs that are dependent on this one. The patches still apply properly on Nightly and no errors were encountered. Can we close this issue if no more investigation is necessary?

Comment 10

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Daniela Petrovici from comment #9)
> I have updated both bugs that are dependent on this one. The patches still
> apply properly on Nightly and no errors were encountered. Can we close this
> issue if no more investigation is necessary?

Yes, thanks Daniela.