'cfx testall' should run tests in example packages

RESOLVED FIXED

Status

Add-on SDK
General
P1
critical
RESOLVED FIXED
8 years ago
7 years ago

People

(Reporter: atul, Assigned: atul)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Assignee)

Description

8 years ago
Entering 'examples/reading-data' and running 'cfx test' currently results in the following error:

  error: TEST FAILED: test-main.test_id (failure)
  error: fail: "{6724fc1b-3ec4-40e2-8583-8061088b3185}" != "MyUniqueID"

This particular failure isn't so important as is the fact that the example packages aren't having their tests run on 'cfx testall'. Because of this, we froze the tree for 0.4 yesterday without realizing that some of our tests were actually failing.
(Assignee)

Updated

8 years ago
Severity: normal → critical
Priority: -- → P1
Tentatively blocking 0.4 on this. It's ok to ship with known issues - but we should at least know what they are.
Blocks: 566904
OS: Mac OS X → All
Hardware: x86 → All
Atul: is this something you can look at?
Assignee: nobody → avarma
(Assignee)

Comment 3

8 years ago
I've just added 'cfx testex', which tests all example code:

  http://hg.mozilla.org/labs/jetpack-sdk/rev/6d1036b00989

I'm thinking that next it might be most useful to make 'cfx testpkgs', which essentially does what 'cfx testall' currently does, and changing 'cfx testall' to effectively run 'cfx testcfx && cfx testpkgs && cfx testex'. All of them unfortunately require different test runners and/or environments, which is why there are currently three different ones. Oye.
(In reply to comment #3)
> I've just added 'cfx testex', which tests all example code:
> 
>   http://hg.mozilla.org/labs/jetpack-sdk/rev/6d1036b00989

Does that mean we can resolve this bug fixed?  Or do you think we still need to make testall include those tests?


> I'm thinking that next it might be most useful to make 'cfx testpkgs', which
> essentially does what 'cfx testall' currently does, and changing 'cfx testall'
> to effectively run 'cfx testcfx && cfx testpkgs && cfx testex'. All of them
> unfortunately require different test runners and/or environments, which is why
> there are currently three different ones. Oye.

We should probably target the overall cleanup of these testing commands for 0.5 while doing the simplest, lowest-risk fix for 0.4, given where we're at.  Can we just get cfx testall to also run cfx testex in addition to what it currently runs?
(Assignee)

Comment 5

8 years ago
Well, we should probably also have 'testall' do 'testcfx' in addition to what it currently does too, because currently that isn't being run either. Alas.
(Assignee)

Comment 6

8 years ago
Ok, it should be fixed now:

  http://hg.mozilla.org/labs/jetpack-sdk/rev/94949f3c59fa

That said, the reading-data example still fails, which we need to fix asap.
(Assignee)

Updated

8 years ago
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
The Add-on SDK is no longer a Mozilla Labs experiment and has become a big enough project to warrant its own Bugzilla product, so the "Add-on SDK" product has been created for it, and I am moving its bugs to that product.

To filter bugmail related to this change, filter on the word "looptid".
Component: Jetpack SDK → General
Product: Mozilla Labs → Add-on SDK
QA Contact: jetpack-sdk → general
Version: Trunk → unspecified
You need to log in before you can comment on or make changes to this bug.