596023 - [synchornization] update mozmill-tests on mozilla-central

Status: RESOLVED WONTFIX

(Testing Graveyard :: Mozmill, defect)

Description

[Jeff Hammel]

The mozmill-tests on mozilla-central is outdated.  They should be updated to include a subset of the tests on http://hg.mozilla.org/qa/mozmill-tests

Additionally, they should satisfy three criteria:
- the tests should not touch network outside of mozilla domains
- the tests should reliably run with Minefield and otherwise be stable
- bug 569831 probably blocks add-on tests

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to comment #0)
> - bug 569831 probably blocks add-on tests

Typo? But anyway, we do not plan to run add-on tests on buildbot. So please exclude those. For all the other tasks lets move over the dependencies.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

First set of Mozmill tests we can land:
* testAwesomeBar/testFaviconInAutocomplete.js
* testAwesomeBar/testGoButton.js
* testPreferences/testSetToCurrentPage.js
* testPrivateBrowsing/testCloseWindow.js
* testPrivateBrowsing/testStartStopPBMode.js
* testPrivateBrowsing/testTabRestoration.js
* testSessionStore/testUndoTabFromContextMenu.js
* testTechnicalTools/testAccessPageInfoDialog.js
* testToolbar/testBackForwardButtons.js

Please also include all files from:
* firefox/test-files/
* shared-modules/

All the above tests are using local test data and haven't been failed recently. We should be good to go with this set of initial tests.

Comment 4

[cmtalbert]

Attachment: part 1

This is the first part of the patch.  It removes all the test directories we aren't using (plus the testPreferences directory accidentally, which is added back in the second patch).

Comment 5

[cmtalbert]

Attachment: part 2

This adds in all the tests that you mentioned in your comment that we want to go live with first.  Please make sure I didn't miss anything.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 476255 [details] [diff] [review]
part 1

Please also remove the orphaned / unused folders which are scripts, templates, and also add-ons. We will never use those on m-c.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 476259 [details] [diff] [review]
part 2

I'm missing the back and forward test from the testToolbar folder. Could you come up with a combined patch please? That would be perfect.

Comment 8

[cmtalbert]

(In reply to comment #6)
> Comment on attachment 476255 [details] [diff] [review]
> part 1
> 
> Please also remove the orphaned / unused folders which are scripts, templates,
> and also add-ons. We will never use those on m-c.
The patch does remove the folders from hg but it does not remove them when it is applied because hg has no notion of folders.  Unfortunately this leaves a bunch of local empty folders that you have to delete yourself.  We have the same problem with the mozmill folders too. :(

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

So where does the readme file come from? I always see it inside those folders which could be the reason?

Comment 10

[cmtalbert]

Attachment: Initial tests [checked-in]

Ok, sorry about that.  Here is the combined patch.  I've gone over this one a couple of times and it looks like everything is in there.  Please double check.

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 476382 [details] [diff] [review]
Initial tests [checked-in]

Patch looks fine. Did a couple of testruns on OS X and everything looks fine. Jeff, do you want to check test-runs on prepared buildbot references images before we go live?

Comment 12

[cmtalbert]

http://hg.mozilla.org/mozilla-central/rev/7e9bbef4434e

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Lets leave this bug open for further checkins.

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Just for clarification. For the restart test please use firefox/restartTests/testDefaultBookmarks/

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+1]]

With bug 579965 fixed we now have 35 more tests which use local test data. We will check our boxes in the next day to figure out which tests can get run reliable for phase 2 on buildbot.

Comment 16

[cmtalbert]

Attachment: Update tests for going live on bbot

Geo, Henrik, this is the current diff of your tests/shared-modules/test-files with what we currently have in m-c.  It represents changes you have made to mozilla-tests that have not happened yet on m-c.  Stated another way, this patch brings the stuff in m-c to parity with what is currently in mozilla-tests (it also includes the changes from bug 604365, so it should represent the most current state of the mozmill-tests).

Comment 17

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

Comment on attachment 484199 [details] [diff] [review]
Update tests for going live on bbot

r+ing. Obviously a bit much to hand-check, but I talked to ctalbert re: how patch was made, and risk if it's wrong. It's easy enough to roll back, so let's go with it.

Comment 18

[Jeff Hammel]

see also https://bugzilla.mozilla.org/show_bug.cgi?id=676078

Comment 19

[Jeff Hammel]

Is this bug closable?

Comment 20

[cmtalbert]

(In reply to Jeff Hammel [:jhammel] from comment #19)
> Is this bug closable?

yes