Closed Bug 675112 Opened 13 years ago Closed 13 years ago

Create "how to test the SDK" guide

Categories

(Add-on SDK Graveyard :: Documentation, defect, P2)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: wbamberg, Assigned: wbamberg)

Details

Attachments

(1 file)

This is for people outside the SDK team who are trying to reproduce and diagnose  failures in the SDK's unit tests - for example, when a platform change causes a regression.

It should explain:
* where to get the latest version of the SDK
* how to install it
* how to run the tests
* how to interpret the output

It should probably be delivered just as a Wiki page.
Let me know if this is what you were looking for. I wondered if it could have more about what failing tests look like, and an example of that? At the moment it really just describes what gets run when you test.
Attachment #549459 - Flags: review?(myk)
Comment on attachment 549459 [details] [diff] [review]
Wiki guide created

This is a pretty good.  The only major piece that feels missing is a "quick start" section at the beginning that would give the minimal amount of information necessary to get the SDK, activate it, run its tests, and attach a debugger to a test run.
To elaborate a bit (sorry, I wrote that last comment as I was running for the train), the information in the document right now feels like good detailed information about how the tests work and how to run them.  It's the kind of information I would read after I first dove into testing but then wasn't able to get the test harness to do what I wanted.

But my first instinct, when I started reading the document (imagining I was a platform developer who has been told that his change has broken the SDK, and he needs to figure out why), was to skip down to the part where the doc tells me how to run the tests.  And it isn't until the very end that the doc mentions cfx testall, even though that's the command that the test automation runs.

So I think a Quick Start section at the beginning is key for us to reach the target audience, even if it duplicates information that is already available elsewhere at greater length.  Here's what I think it should say:

--------------------------------------------------------------------------------
First, get the SDK from git:

  git git://github.com/mozilla/addon-sdk.git

Then enter the SDK directory and activate the SDK:

  cd addon-sdk
  source bin/activate

Finally, run all tests:

  cfx testall

To run tests on a specific build of Firefox, specify the build via the --binary flag:

  cfx testall --binary /Applications/Nightly.app/

  cfx testall --binary ~/Mozilla/source/obj-dir/dist/bin/firefox

  cfx testall --binary /c/Program\ Files\Nightly\firefox.exe

To attach gdb before running tests, specify the --no-run flag, then invoke the command it tells you to run:

  cfx testall --no-run

To run only the API tests (typically the ones that break when Firefox changes), invoke testpkgs instead of testall:

  cfx testpkgs

To run only certain test suites, specify the --filter flag with a regular expression that matches the filenames of the test suites you want to run:

  cfx testpkgs --filter panel

Need more detail? Read the rest of this document to learn all about the SDK's test suites and harness!
--------------------------------------------------------------------------------

Does that make sense?
It makes a lot of sense, thanks. I added a slightly-tweaked version of the above.
OS: Mac OS X → All
Priority: -- → P2
Hardware: x86 → All
Target Milestone: --- → 1.1
Comment on attachment 549459 [details] [diff] [review]
Wiki guide created

Looks great, thanks!
Attachment #549459 - Flags: review?(myk) → review+
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: