889719 - Implement a way to skip tests on unsupported applications (Fennec)

Status: RESOLVED FIXED

(Add-on SDK Graveyard :: General, defect)

Description

[Jordan Santell [:jsantell] [@jsantell]]

Modify harness to skip tests when a required module is not supported in an application. Right now we have to add a lot of boilerplate for every test

In sdk/places/bookmarks:
`module.metadata: { "engines" : { "Firefox" : "*" } }`

When loaded in test `test-places-bookmarks`, this should skip over tests in Fennec, for example, rather than implementing a try/catch

Comment 1

[Jordan Santell [:jsantell] [@jsantell]]

After some investigating, options:

* Check metadata of a module before loading said module -- if unsupported, escape in tests that use it. Not sure if we can access that easily from cuddlefish. A strong pro however is having support meta all in one place (module)

* Add metadata to tests, as there's not a 1:1 relationship between tests and modules. Same as above, not sure if we can access this from cuddlefish.

* Ignore `Unsupported Application` errors, which are thrown by loader when loading an unsupported application. After digging, not sure if cuddlefish has access to this, as the harness_options resultFile just gets OK/FAIL, without any additional information.

Comment 2

[Jordan Santell [:jsantell] [@jsantell]]

Attachment: GH Pull Request 1078

Comment 3

[[github robot]]

Commits pushed to master at https://github.com/mozilla/addon-sdk

https://github.com/mozilla/addon-sdk/commit/6f6b500340491b579ce8d406ed1ae6a0395dffdd
Bug 889719 Skip tests when application is considered unsupported

https://github.com/mozilla/addon-sdk/commit/c01d686b17ed1d182f1f5256b8b6e8730d89171b
Merge pull request #1078 from jsantell/unsupported-test-skip-889719

Fix bug 889719 Skip tests when application is considered unsupported, r+=@ZER0

Comment 4

[Jordan Santell [:jsantell] [@jsantell]]

Comment on attachment 770971 [details]
GH Pull Request 1078

r+'d on GH

Comment 5

[Matteo Ferretti [:zer0] [:matteo]]

Comment on attachment 770971 [details]
GH Pull Request 1078

My apologies for this 'dancing' of review's field, but it's useful when we have to generate bugzilla reports for the weekly status (so I know that I reviewed that stuff even if I don't remember it :) ).