Closed Bug 1162514 Opened 5 years ago Closed 5 years ago

Autophone - 2015-05-06 deployment incorrectly is running Mochitest, Reftest and Robocop tests

Categories

(Testing :: Autophone, defect)

defect
Not set

Tracking

(firefox40 affected)

RESOLVED FIXED
Tracking Status
firefox40 --- affected

People

(Reporter: bc, Assigned: bc)

References

Details

Attachments

(2 files)

804 bytes, text/plain
Details
44 bytes, text/x-github-pull-request
gbrown
: review+
Details | Review
The deployment last night incorrectly enabled the Reftest, Mochitest, Robocop tests. I've shutdown the system until it can be corrected.

Tomcat, did you hide all of the Autophone tests?

The Autophone s1s2 t, webappstartup w, and Mochitest Mm tests should still remain visible.

The Autophone crashtest C, Javascript J, Mochitest M1-M16, Mdb, Mdm, Msk, Mtw, the reftest R1-R16, Rov, Rwv, Mochitest Robocop rc should be hidden.
(In reply to Bob Clary [:bc:] from comment #0)
> The deployment last night incorrectly enabled the Reftest, Mochitest,
> Robocop tests. I've shutdown the system until it can be corrected.
> 
> Tomcat, did you hide all of the Autophone tests?
> 

yeah did this
Yeah, that was correct. Don't make any of them visible again.
This is a regression from bug 1153992 where I removed the test repo tracking.
Blocks: 1153992
Attached file developer-test.ini
example test manifest.

This should run tests as follows:

mozilla-inbound
===============
Android 2.3 A(Mm t t t w)
Android 4.0 A(Mm)

nexus-one-1 android 2.3 blank-local, twitter-local, nytimes-local, webappstartup
nexus-one-3 android 2.3 mochitest-media
samsung-gs3-3 android 4.0 mochitest-media

fx-team
=======
Android 2.3 A(t t t w)

nexus-one-2 android 2.3 blank-remote, twitter-remote, nytimes-remote, webappstartup

mozilla-central
===============
Android 2.3 A(t t t w w)

nexus-one-2 android 2.3 blank-remote, twitter-remote, nytimes-remote, webappstartup
nexus-one-3 android 2.3 webappstartup

b2g-inbound
===========
Android 4.0 A(w)

samsung-gs3 android 4.0 webappstartup

Running a test with the current master shows the repo selection is totally broken and all repos are run regardless of what the test manifest says. This reproduces the problem with the deployment on 2015-05-06.

https://treeherder.allizom.org/#/jobs?repo=mozilla-inbound&filter-searchStr=autophone

https://treeherder.allizom.org/#/jobs?repo=fx-team&revision=aadbe230abec&filter-searchStr=autophone

https://treeherder.allizom.org/#/jobs?repo=b2g-inbound&revision=8b6e80d8d44f&filter-searchStr=autophone

https://treeherder.allizom.org/#/jobs?repo=mozilla-central&revision=86203ac87a08&filter-searchStr=autophone
(In reply to Bob Clary [:bc:] from comment #4)

With this patch


> 
> mozilla-inbound
> ===============
> Android 2.3 A(Mm t t t w)
> Android 4.0 A(Mm)
> 

https://treeherder.allizom.org/#/jobs?repo=mozilla-inbound&revision=4323557b111d&filter-searchStr=autophone

Ok.
 
> fx-team
> =======
> Android 2.3 A(t t t w)

https://treeherder.allizom.org/#/jobs?repo=fx-team&revision=dbec5281f625&filter-searchStr=autophone

Ok.

> mozilla-central
> ===============
> Android 2.3 A(t t t w w)

https://treeherder.allizom.org/#/jobs?repo=mozilla-central&revision=cef2b76b082d&filter-searchStr=autophone

Ok.
 
> b2g-inbound
> ===========
> Android 4.0 A(w)

This build is the same as tested without the patch so the other tests interfere with the display. You can check that the Android 4.0 A(w) requested at  Fri May 8, 11:48:34 is the only one from this run. When I have a complete set of builds on all branches I'll trigger new builds so there is no confusion.
Attached file PR 32
Attachment #8603489 - Flags: review?(gbrown)
Blocks: 1163129
Comment on attachment 8603489 [details] [review]
PR 32

That all looks fine to me.
Attachment #8603489 - Flags: review?(gbrown) → review+
(In reply to Bob Clary [:bc:] from comment #4)

> mozilla-inbound
> ===============
> Android 2.3 A(Mm t t t w)
> Android 4.0 A(Mm)

https://treeherder.allizom.org/#/jobs?repo=mozilla-inbound&revision=ab4feae9bff8&filter-searchStr=autophone

Ok.

> fx-team
> =======
> Android 2.3 A(t t t w)

https://treeherder.allizom.org/#/jobs?repo=fx-team&revision=aa4eac3a4336&filter-searchStr=autophone

Ok, With one extra t s1s2-blank-remote.ini on nexus-one-2 due to a retry.

> mozilla-central
> ===============
> Android 2.3 A(t t t w w)

https://treeherder.allizom.org/#/jobs?repo=mozilla-central&revision=9c0bb1e02555&filter-searchStr=autophone

Ok.

> b2g-inbound
> ===========
> Android 4.0 A(w)

https://treeherder.allizom.org/#/jobs?repo=b2g-inbound&revision=2f8b7d236713&filter-searchStr=autophone

Ok.
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Depends on: 1164505
You need to log in before you can comment on or make changes to this bug.