752112 - Test failures from Jetpack suite aren't parsed

Status: RESOLVED FIXED

(Tree Management Graveyard :: TBPL, defect)

Description

[Mark A. Hershberger (hexmode)]

Since test failures from addons don't show up in tinderbox, code that breaks jetpack gets committed without anyone fixing it.

Because the addons-sdk team wants to keep dependencies to a minimum, only python's unittest is used. tldp should be extended as necessary to parse the test code output by jetpack.

It looks like this will mean adding a LineFilter implementation for the test output.

Comment 1

[Robert Helmer [:rhelmer]]

Thanks for offering to take this on!

Comment 2

[(no longer active)]

Hrm, those test suites are just hidden AFAIK (see https://tbpl.mozilla.org/?noignore=1) since they've been perma-orange, so looking at them doesn't really make much sense.  Am I missing what this bug is about?

Comment 3

[Phil Ringnalda (:philor)]

Yeah, he means that we need an errorparser that recognizes the jetpack suite's way of marking test failures ("^error: TEST FAILED" at least, possibly more), since it doesn't do TEST-UNEXPECTED-FAIL.

Comment 4

[Mark A. Hershberger (hexmode)]

Attachment: minimal patch to support jetpack output

Here is a minimal patch that got it to work for me.

Comment 5

[Mark A. Hershberger (hexmode)]

Attachment: Patch so log fetching would work on my virtual machine

It took me a while to figure out what was going on.  Part of the problem was the hidden errors (the '@' prefix)

When I removed those, I saw the following:

  PHP Warning:  ftp_fget(): Illegal PORT command (Stingray Traffic Manager FTP
  proxy). in ..../php/inc/RawGzLogDownloader.php on line 29,

I don't know how much large the files that RawGzLogDownloader usually gets are, but so far it works for a test box.

Comment 6

[Arpad Borsos [:Swatinem]]

Comment on attachment 621360 [details] [diff] [review]
Patch so log fetching would work on my virtual machine

Review of attachment 621360 [details] [diff] [review]:
-----------------------------------------------------------------

::: php/inc/RawGzLogDownloader.php
@@ -26,5 @@
> -    if (!@ftp_login($ftpstream, "anonymous", ""))
> -      throw new Exception("Couldn't connect to Mozilla FTP server.");
> -    $fp = tmpfile();
> -    if (!@ftp_fget($ftpstream, $fp, $path, FTP_BINARY))
> -      throw new Exception("Log not available at URL {$this->logURL}.");

Just wondering what kind of error handling the new code has. What happens to file_get_contents when the url is a 404?

Comment 7

[Mark A. Hershberger (hexmode)]

Comment on attachment 621360 [details] [diff] [review]
Patch so log fetching would work on my virtual machine

moving fixed patch to Bug 754420

Comment 8

[Phil Ringnalda (:philor)]

http://hg.mozilla.org/users/mstange_themasta.com/tinderboxpushlog/rev/d36a2aea379f (probably be deployed to tbpl.m.o early next week)