647990 - Several crashtests are still (incorrectly?) marked as skip-if(Android)

Status: NEW

(Testing :: Reftest, defect)

Description

[Jesse Ruderman]

According to bug 637660 comment 12, the crashtests disabled in that bug should have been re-enabled when bug 638844 (dup of bug 638846) was fixed.  But that was mentioned in neither the manifest nor the followup bug, so of course it didn't happen.

PLEASE annotate manifests with bug numbers whenever you disable tests!  (Reviewers: please enforce this!)  Otherwise, it's nearly impossible for my automated tools to detect when you forget to re-enable them when the bug is fixed.  I only happened to notice this mistake because I was behind in reading the changelog, and came across both bugs at the same time.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

these tests were disabled without any bug indicating why they failed.  The Mobile team was busy releasing instead of debugging and they are now working on fixing failed crash/reftests and/or documenting why they are not running.

bug 638844 had nothing to do with fixing the bug, rather turning the tests on.  Probably a mixed up bug or message somewhere.

Comment 2

[Jesse Ruderman]

Explained to me at the all-hands:

The mobile automated testing team wanted to get the suites green quickly so they could be shown on TBPL and prevent ongoing regressions. They decided to mark tests as skip-if(Android) etc for now and file bugs later.

I hope that happens relatively soon :)