676087 - Add membuster endurance test to the daily testruns

Status: RESOLVED INVALID

(Mozilla QA Graveyard :: Mozmill Automation, defect)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

We should run the membuster endurance test daily on all supported platforms.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Add membuster endurance test to the daily testruns. v1.0

Henrik: Is this the correct approach to adding membuster to the daily testruns?

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Add membuster endurance test to the daily testruns. v1.1

Previous patch included an unwanted change.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 554836 [details] [diff] [review]
Add membuster endurance test to the daily testruns. v1.1

As mentioned in our 1-1 yesterday, we also have to update testrun.py to ensure that we do not abort the testrun if the specified reserved test-run doesn't exist, i.e. 3.6.20. We do not want to use dummy tests as we do now, because they pollute our test results on the dashboard.

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Can you raise a bug and mark it as a blocker for this?

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 554836 [details] [diff] [review]
Add membuster endurance test to the daily testruns. v1.1

This patch is very out of date. The new approach would probably be a job in Mozmill CI that runs the specific test. What do you think Henrik?

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Right. CI should do it. So we should get an issue filed. Can our automation scripts call this test type already? If yes we should close out this bug.

Comment 7

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Yes, this can be run via our automation scripts (example below). This bug was more about running for each build. I have raised https://github.com/mozilla/mozmill-ci/issues/218 to take care of this.

$ ./testrun_endurance.py --iterations=2 --entities=5 --reserved=membuster /Applications/FirefoxNightly.app/