1001421 - Some testruns are not ran on OS X 10.6

Status: RESOLVED INCOMPLETE

(Mozilla QA Graveyard :: Infrastructure, defect)

Description

[Andreea Matei [:AndreeaMatei]]

We've seen in the last testrun reports we check that sometimes we don't run tests on this particular platform version.

https://docs.google.com/spreadsheet/ccc?key=0AgbJACdAek5ndEY5cmlFMzRlcnB2aDZWVFJMMk5UdWc&usp=sharing#gid=9

On April 4th we also had this:
    Remote run with beta fr and it on OS X 10.6 did not ran
    Functional run with beta de and en-us on OS X 10.6 did not ran
https://docs.google.com/spreadsheet/ccc?key=0ApWA7fkeN63IdEtqeWZNc3YwZ255NldMUHJsQ0tDd0E#gid=7

I know we had some issues with mounted DMGs on these machines, but we might want to investigate and check daily what happens with the testruns there, if they are triggered but don't get through or not (we might sometimes retrigger them only the second day morning).

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Given that we only keep 500 builds we should really start to check the results a day after the beta builds have been produced and have been tested. I will talk to Matt about responsibilities here.

Comment 2

[Andrei Eftimie]

We ought to have a sanity check that all expected testruns have actually been done.
This will be hard to track manually.

We might be able to assert this based on our mozmill-ci config files...

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Andreea Matei [:AndreeaMatei] from comment #0)
> On April 4th we also had this:
>     Remote run with beta fr and it on OS X 10.6 did not ran
>     Functional run with beta de and en-us on OS X 10.6 did not ran

Where did you get those information from? The dashboard? Keep in mind that those tests might have been run but no report has been sent. We had issues like that in the past, and I can remember that one day Cosmin had seen something similar even with Mozmill 2.0.6. I think it was in April.

Comment 4

[Andreea Matei [:AndreeaMatei]]

Oh, I missed to reply here. The information was based on the reports only, from the dashboard, as what we check for the testrun reports we do manually. So indeed it could be that we didn't sent the report only, we'll keep in mind to check jenkins jobs as well, but so far I would leave this open a bit more.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We never figured out, and Mozmill-CI will be shutdown on Monday next week. So this is not needed anymore.