1087560 - Mochitest doesn't handle test files which don't start with 'test_'

Status: RESOLVED FIXED

(Core :: Graphics: CanvasWebGL, defect)

Description

[Kelsey Gilbert [:jgilbert]]

Attachment: 0001-Rename-mochitest-tests-to-match-test_.patch

It's been (nearly) silently not running a couple tests because they don't match the desired filename format. Making this assert is bug 1087558.

Comment 1

[Kelsey Gilbert [:jgilbert]]

Attachment: 0002-Fix-tests.patch

Comment 2

[Kelsey Gilbert [:jgilbert]]

https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=02cd8fdbda48

Fun, broken on Mac.

Comment 3

[Ryan VanderMeulen [:RyanVM]]

Why are you asking me to land something that has test failures on the Try run?

Also, what reftests were you expecting to fail from these patches? Or xpcshell. Or mochitest-bc. Etc. Try to consider that when deciding which test suites to run? :)
https://wiki.mozilla.org/Sheriffing/How:To:Recommended_Try_Practices

Comment 4

[Kelsey Gilbert [:jgilbert]]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #3)
> Why are you asking me to land something that has test failures on the Try
> run?
> 
> Also, what reftests were you expecting to fail from these patches? Or
> xpcshell. Or mochitest-bc. Etc. Try to consider that when deciding which
> test suites to run? :)
> https://wiki.mozilla.org/Sheriffing/How:To:Recommended_Try_Practices

Hah, sorry. I forgot to remove checkin-needed.

Comment 5

[Kelsey Gilbert [:jgilbert]]

 https://hg.mozilla.org/integration/mozilla-inbound/rev/508d70d46898
 https://hg.mozilla.org/integration/mozilla-inbound/rev/8c366a73988c

Bug 1087634 should have fixed the failing test.

Comment 6

[Carsten Book [:Tomcat]]

Hi Jeff, sorry had to back this out for test failures like https://treeherder.mozilla.org/ui/logviewer.html#?job_id=3269743&repo=mozilla-inbound

Comment 7

[Carsten Book [:Tomcat]]

oh and https://treeherder.mozilla.org/ui/logviewer.html#?job_id=3271946&repo=mozilla-inbound might be related too

Comment 8

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/265a63f3f5e1

Comment 9

[Kelsey Gilbert [:jgilbert]]

That was just the CLOBBER update.

Comment 10

[Kelsey Gilbert [:jgilbert]]

Attachment: 0002-Fix-tests.patch

I changed this, sorry!

Comment 11

[Kelsey Gilbert [:jgilbert]]

Attachment: 0003-Build-GetContextAttributes-from-mOptions-now-that-th.patch

Comment 12

[Kelsey Gilbert [:jgilbert]]

Green:
https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=93f48e5d2885

Comment 13

[Kelsey Gilbert [:jgilbert]]

remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/1b8b46616028
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/12a016a84d6d
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/ecf4df03029a

This might need a clobber if we get missing file complains for "test-backbuffer[...]".

Comment 14

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Backed out in  for various failures:

https://treeherder.mozilla.org/ui/logviewer.html#?job_id=3341963&repo=mozilla-inbound
https://treeherder.mozilla.org/ui/logviewer.html#?job_id=3342941&repo=mozilla-inbound

Comment 15

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Clipboard failed. Backout commit for all three is https://hg.mozilla.org/integration/mozilla-inbound/rev/982fc6c2f76b

Comment 16

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

I should probably add that https://treeherder.mozilla.org/ui/logviewer.html#?job_id=3344490&repo=mozilla-inbound occurred after one of the following pushes touched CLOBBER, so it's not just needs-clobber. 

These failures were on b2g mochitest-1, not the b2g mochitest-2 that your try runs ran.

Comment 17

[Kelsey Gilbert [:jgilbert]]

Ugh, not again.

Comment 18

[Kelsey Gilbert [:jgilbert]]

Attachment: 0003-Skip-WebGL-tests-on-B2G-Desktop.patch

Skip those tests, since B2G Desktop doesn't get WebGL contexts yet.

Comment 19

[Kelsey Gilbert [:jgilbert]]

Comment on attachment 8512188 [details] [diff] [review]
0003-Build-GetContextAttributes-from-mOptions-now-that-th.patch

This landed in bug 912196.

Comment 20

[Kelsey Gilbert [:jgilbert]]

remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/4f040d3522a4
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/64c7041ab581
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/ed403e959506

Comment 21

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

And re-clobbering for the re-landing: https://hg.mozilla.org/integration/mozilla-inbound/rev/01ac588cd123

Comment 22

[Phil Ringnalda (:philor)]

Backed out yet again, in https://hg.mozilla.org/integration/mozilla-inbound/rev/3a8b152acd9c, for causing a permacrash in Android 2.3 opt mochitest-10 in test_imagecapture.html, and permaorange in b2g emulator, opt mochitest-3 and debug mochitest-10, misfiled as bug 1090649.

Please do not under any circumstances push this again without either filing a bug about the way it needs a clobber and touching /CLOBBER as part of your push, or figuring out and fixing why it needs a clobber.

Please do not under any circumstances push this again without doing a "try: -b do -p all -u all -t none" try push (in theory you ought to be able to do -u mochitests, but that's merely theory, and I'm sick of seeing this turn the tree orange for seven or eight hours at a time).

Comment 24

[Kelsey Gilbert [:jgilbert]]

https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=4b00fca7e89e

Green except for some likely-intermittent oranges. I retriggered those. When I land again, I'll include a CLOBBER touch.

Comment 25

[Kelsey Gilbert [:jgilbert]]

Is this clean? There are two intermittents that seem common. Neither is directly related to my patches.
https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=4b00fca7e89e

There's nothing left I can do other than periodically rebase and do a Try run until it happens to not cause problems.

Comment 26

[Phil Ringnalda (:philor)]

Sadly, I think that is what amounts to a clean run, though it rather neatly does point out that linux32 debug mochitest-e10s-2 doesn't even remotely come close to meeting the requirements to be visible, and Mullet 5 isn't far behind.

Fire away!

Comment 27

[Kelsey Gilbert [:jgilbert]]

remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/10ec61ff9b6c
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/1bc2c5b8873a
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/6a96db9759b7
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/0c9087692248

No whammy, no whammy, no whammy...

Comment 28

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/10ec61ff9b6c
https://hg.mozilla.org/mozilla-central/rev/1bc2c5b8873a
https://hg.mozilla.org/mozilla-central/rev/6a96db9759b7
https://hg.mozilla.org/mozilla-central/rev/0c9087692248

These all landed with the wrong bug number, by the way. Please be more careful in the future.

Comment 29

[Kelsey Gilbert [:jgilbert]]

Hah, sorrry about that!