Closed Bug 804158 Opened 13 years ago Closed 11 years ago

Mozmill restart tests should skip following tests in case of a failure

Categories

(Testing Graveyard :: Mozmill, defect)

Product:
Testing Graveyard
Component:
Mozmill
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: AlexLakatos, Unassigned, Mentored)

Details

If a test file is failing in our current logic, the next files are still run. In most cases, they fail as well, as they rely on the fact that the previous test file passed. So we should add more logic to mozmill-restart and skip those. Bug 786306 implements that logic directly on our test files and https://bugzilla.mozilla.org/attachment.cgi?id=670311 has a proof of concept on how to do it. We should replicate that logic and functionality inside mozmill-restart
Whiteboard: [mozmill-2.0?][good first bug][mentor=hskupin][lang=js]
Just a note which I have mentioned in the meeting and which hasn't added here. It can be that the test does final computation in the very last teardownModule() method. We have to ensure that we do not skip that. It's clearly not a good first bug. I will remove that entry.
Whiteboard: [mozmill-2.0?][good first bug][mentor=hskupin][lang=js] → [mozmill-2.0?][mentor=hskupin][lang=js]
Whiteboard: [mozmill-2.0?][mentor=hskupin][lang=js] → [mozmill-2.1?][mentor=hskupin][lang=js]
Whiteboard: [mozmill-2.1?][mentor=hskupin][lang=js] → [mozmill-2.1?][mentor=whimboo][lang=js]
Whiteboard: [mozmill-2.1?][mentor=whimboo][lang=js] → [mozmill-2.1?][mentor=whimboo][lang=js][sprint2013-32]
Whiteboard: [mozmill-2.1?][mentor=whimboo][lang=js][sprint2013-32] → [mozmill-2.1?][mentor=whimboo][lang=js]
This bug will not be taken for version 2.1. We might want to re-evaluate for 2.2.
Whiteboard: [mozmill-2.1?][mentor=whimboo][lang=js] → [mozmill-2.2?][mentor=whimboo][lang=js]
Whiteboard: [mozmill-2.2?][mentor=whimboo][lang=js] → [mozmill-2.2?][mentor=whimboo][lang=js][good first bug]
Restart logic is hard and this is by far not a good first bug.
Summary: make mozmill-restart skip test in a folder after one has failed → Mozmill restart tests should skip following tests in case of a failure
Whiteboard: [mozmill-2.2?][mentor=whimboo][lang=js][good first bug] → [mozmill-2.2?][mentor=whimboo][lang=js]
Mentor: hskupin
Whiteboard: [mozmill-2.2?][mentor=whimboo][lang=js] → [mozmill-2.2?][lang=js]
Mozmill will reach its end of life soon. We are currently working on getting all the tests for Firefox ported to Marionette. For status updates please see bug 1080766.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → WONTFIX
Whiteboard: [mozmill-2.2?][lang=js]
Product: Testing → Testing Graveyard
You need to log in before you can comment on or make changes to this bug.