Closed Bug 406227 Opened 14 years ago Closed 13 years ago

Enable Unit Tests on a Thunderbird trunk tinderbox

Categories

(Thunderbird :: Build Config, defect, P2)

defect

Tracking

(Not tracked)

RESOLVED FIXED
Thunderbird 3

People

(Reporter: standard8, Assigned: standard8)

References

Details

Attachments

(1 file)

Now that bug 405147 has provided some initial unit tests for mailnews, we should enable unit testing on at least one of the Thunderbird trunk tinderboxes.

(SeaMonkey has already got xpcshell tests running http://tinderbox.mozilla.org/SeaMonkey/)

I would suggest getting a unit test box aka Firefox, but I don't think we need to that just yet as we haven't got anything complicated on the way, and I think the initial address book tests that we need can all be done at the xpcshell test level which is what SeaMonkey has running on their main tinderboxes.

As File I/O tends to be slow on Windows I'd suggest not using that one.

If we enable it on the Linux box, then we'll need to supply a packages file (like we do for windows) so that it doesn't package the test files that get generated. I'm guessing the same may apply for the Mac box, though I'm not fully sure about that.

So any thoughts/comments on this? If nothing major is raised, I'll put together a packages-static file for Thunderbird Linux and sort out what we'll need to enable where.
So I was starting to look at this and realised that although I'd removed --disable-tests from my mozconfig, I still wasn't getting the tests built (the address book tests I've done so far I've tested with SeaMonkey - but that's not too bad as I know the address book code is common.

Having done some investigation, it turns out that ENABLE_TESTS is forced to "nothing" in mail/confvars.sh, the default in configure.in is to build with tests, so having "ENABLE_TESTS=" in confvars.sh means you can't build Thunderbird with tests.

I then did some more digging, the Thunderbird trunk tinderboxes all have --disable-tests in their mozconfig files:

http://mxr.mozilla.org/seamonkey/source/tools/tinderbox-configs/thunderbird/win32/mozconfig
http://mxr.mozilla.org/seamonkey/source/tools/tinderbox-configs/thunderbird/linux/mozconfig
http://mxr.mozilla.org/seamonkey/source/tools/tinderbox-configs/thunderbird/macosx/mozconfig

So in summary, this'll affect personal builds only (where --disable-tests hasn't already been put into the mozconfig file). I'll do a post to mozilla.dev.apps.thunderbird when I check the patch in so that hopefully some folks will spot it.

It also prepares us for enabling unit tests on one or more of the tinderboxes once I've got the other issues sorted out (see comment 0) and checked that the tests pass.
Attachment #291530 - Flags: superreview?(bienvenu)
Attachment #291530 - Flags: review?(bienvenu)
Attachment #291530 - Flags: superreview?(bienvenu)
Attachment #291530 - Flags: superreview+
Attachment #291530 - Flags: review?(bienvenu)
Attachment #291530 - Flags: review+
Attachment #291530 - Attachment description: Allow tests to be built on dev builds of Thunderbird → Allow tests to be built on dev builds of Thunderbird (checked in)
Just to let everyone know the current status of this bug.

Its currently possible to build Thunderbird with tests and run the xpcshell tests. The ones in /mailnews pass (as per SeaMonkey).

The ones for cookie testing fail because Thunderbird blocks any cookies apart from RSS. I haven't yet thought of a way of getting around this problem.
(In reply to comment #2)
> The ones for cookie testing fail because Thunderbird blocks any cookies apart
> from RSS. I haven't yet thought of a way of getting around this problem.

Additionally, I found out yesterday the password manager tests will fail because Thunderbird doesn't build with FTP, and the PM tests use ftp to test some of the migration.

I'm starting to think of a few options:

1) Find a way just to run the /mailnews specific tests on tinderbox (yet another tinderbox hack :-( )
2) Complete migration to xulrunner and then xulrunner can do its tests and we can do ours (sort of connected to item 1).
3) Find a way to "miss out" core tests.
4) Add some extra code to the tests (in a TB specific sense) to provide the ftp protocol/enable cookies/whatever is required so that those tests pass.
Would it be possible (well, easy) to make tb build with FTP also? Would be needed e.g. for bug 273476. Possibly bug 390057 is also a result of lacking of ftp support.
QA Contact: build → build-config
Blocks: 421049
Nominating to keep it on the radar.
Flags: blocking-thunderbird3?
Depends on: 427203
4) seems like the right strategy here.  Core tests should not be written in such a way that they are executed but fail in the non-Firefox case.
Flags: blocking-thunderbird3? → blocking-thunderbird3+
Ideally, wouldn't we just ifdef MOZ_FTP or whatever around the tests, so they only get run if they can succeed?
Ted: sounds right w.r.t. the FTP tests.  The cookie tests presumably require a slightly different tack.
Blocks: 422817
Depends on: 378991
Depends on: 431124
Depends on: 431125
Depends on: 431130
Depends on: 431131
Depends on: 431139
Depends on: 431146
Depends on: 431147
Depends on: 431159
No longer depends on: 431139
No longer depends on: 431131
bug 431146 is a debug only problem, not blocking this bug. I'm detailing debug-only problems here: http://wiki.mozilla.org/MailNews:Automated_Testing#xpcshell_tests
No longer depends on: 431146
No longer depends on: 431147
Priority: -- → P2
This bug appears fixed to me.  Resolving, and removing the blocking flag.
Status: NEW → RESOLVED
Closed: 13 years ago
Flags: blocking-thunderbird3+
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 3
You need to log in before you can comment on or make changes to this bug.