601632 - MIPS build slave should run acceptance for 10 minutes with release and debug shells

Status: VERIFIED FIXED

(Tamarin Graveyard :: Build Config, defect, P3)

Description

[Brent Baker]

Currently the MIPS build slave will only run the "smokes" phase and that is only using the release shell. Then a full acceptance pass is done with the release and debug shells in the deep phase. We should add running 10 minutes of acceptance on the release shell using --random and then also 10 minutes with the debug shell.

This will at least give us more coverage on MIPS and still be well within current build times.

Comment 1

[Brent Baker]

Attachment: buildbot config to run random acceptance tests on release and debug

Comment 2

[Brent Baker]

Comment on attachment 480943 [details] [diff] [review]
buildbot config to run random acceptance tests on release and debug

patch pushed as 5323:a837d40687b9

Comment 3

[Edwin Smith]

Instead of this being random, could it be a more deterministic approach?  Here are two possibilities:

* keep track of what recent builds have tested, and cycle through all the acceptance tests in round-robin order.  

* keep track of what has failed recently, and re-test those no matter what, similar to smokes.  come to think of it, does the smokes phase rerun tests that have recently failed in the acceptance phase?  is grooming the smokes list still a manual step?

Comment 4

[Trevor Baker]

The patch does 5 mins per shell and includes 2 shells for a 10 min max.  We have more time that that.  Either increase the timeout or add additional shells.  Increasing timeout is probably best.

Comment 5

[Brent Baker]

Update to run 10 minutes of release and 10 minutes of debug in patch 5479:e4cce27bc2df.

NOTE: Full acceptance passes are made in the deep phase of testing.