771029 - Use relative paths of tests for test_start and test_end events

Status: RESOLVED FIXED

(Testing Graveyard :: Mozmill, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Showing only the filename of the test as run by Mozmill is not that helpful:

TEST-START | test1.js | setupModule
TEST-START | test1.js | testInstallTheme

The test object should have a relpath property which gives us the relative path to the root manifest:

TEST-START | restartTests/testAddons_changeTheme/test1.js | setupModule
TEST-START | restartTests/testAddons_changeTheme/test1.js | testInstallTheme

This will help a lot with debugging the tests and also for our dashboard so that we no longer have to determine that folder on our own.

A fix for this issue will be dependent on a manifestparser update, which I will file in a bit.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Patch

Pointer to Github pull-request

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 639206 [details]
Patch

This patch will make use of relpath if available. Otherwise it will fallback to the absolute path of the test.

Comment 3

[Jeff Hammel]

(In reply to Henrik Skupin (:whimboo) from comment #2)
> Comment on attachment 639206 [details]
> Patch
> 
> This patch will make use of relpath if available. Otherwise it will fallback
> to the absolute path of the test.

I'll review this after bug 771030 is landed.  Also, this will require mozmill to use the new (and as yet unreleased) ManifestDestiny

Comment 4

[Jeff Hammel]

Comment on attachment 639206 [details]
Patch

taking this down for review as the dependent bug isn't landed

Comment 5

[Jeff Hammel]

(In reply to Jeff Hammel [:jhammel] from comment #4)
> Comment on attachment 639206 [details]
> Patch
> 
> taking this down for review as the dependent bug isn't landed

Hmmm...and now that the blocking bug is fixed this gives me a 404

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 639206 [details]
Patch

Because the mozmill repository has been moved. I will create a new request.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Pointer to Github pull request: https://github.com/mozilla/mozmill/pull/63

Pointer to Github pull-request

Comment 8

[Jeff Hammel]

Comment on attachment 650291 [details]
Pointer to Github pull request: https://github.com/mozilla/mozmill/pull/63

Looks good to me.  You may also want to bump the required version of ManifestDestiny for mozmill (which, to be fair, isn't released yet: bug 781105)

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Updated pull request with a follow-up comment which bumps ManifestDestiny to 0.5.5. Lets wait for bug 781105 now.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+1]]

https://github.com/mozilla/mozmill/commit/91d319a0dcf7acb8b7bdc9e30ad88713de4819b2