741301 - Disable broken l10n crop tests until false positives have been resolved

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As long as this test is not working properly we have to disable it.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

It's mostly because of bug 614579 that we are failing, but there can also be other instances.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Patch v1 [landed]

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Landed as:
http://hg.mozilla.org/qa/mozmill-tests/rev/e5ec1a6cd456 (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/8549fc9351dd (aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/1c96f957d923 (beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/862fd76c592a (release)
http://hg.mozilla.org/qa/mozmill-tests/rev/e259aeb38376 (esr10)
http://hg.mozilla.org/qa/mozmill-tests/rev/68f1be683c47 (1.9.2)

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This should not have been closed. Reopening, so we can get it landed once bug 741762 is fixed.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Backed-out across branches:
http://hg.mozilla.org/qa/mozmill-tests/rev/bb258f3d26b0 (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/0f2d675e5f19 (aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/e511a2f442ae (beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/747c5b6d8ea1 (release)
http://hg.mozilla.org/qa/mozmill-tests/rev/985b35c3bc54 (esr10)

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I had to re-land this patch to skip the crop tests. Right now we have too many false positives and I'm inclined not to enable those tests again until we have solved all of them. Please file dependent bugs so we can get those tracked.

Pushed:
http://hg.mozilla.org/qa/mozmill-tests/rev/c47167154d35
http://hg.mozilla.org/qa/mozmill-tests/rev/986c74f95b91
http://hg.mozilla.org/qa/mozmill-tests/rev/dbec55005135
http://hg.mozilla.org/qa/mozmill-tests/rev/b3de6a4b835d
http://hg.mozilla.org/qa/mozmill-tests/rev/8d43b51bf2d7

Comment 7

[Jonathan French (:jfrench)]

A minor note; as a result of landed bug 873473, the bug referenced in the skip for this bug has been updated to this bugnumber (741301).

Comment 8

[Andrei Eftimie]

Attachment: unskip - nightly, aurora

Comment 9

[Andrei Eftimie]

Attachment: unskip - beta, release

Comment 10

[Andrei Eftimie]

Attachment: unskip - esr17

No failures across all branches and OS's
Had to make a different patch for Beta/Release and ESR17 (very minor changes eg. spacing)

Let's land this and if no errors on CI, we can close all dependencies.

Some reports:

OSX
---
Nightly: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc9477b714b
Aurora: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc9477b9322
Beta: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc9477c0fa0
Release: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc9477c5775
ESR17: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc9477cc951

Linux
-----
Nightly: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc9477e51c3
Aurora: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc9477e9ee8
Beta: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc9477ed4f8
Release: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc9477f2c5f
ESR17: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc9477f5f8b

Windows
-------
Nightly: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc947824294
Aurora: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc9478284f2
Beta: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc947836b02
Release: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc94783ba05
ESR17: http://mozmill-crowd.blargon7.com/#/l10n/report/5aa1ca5e7015e3740d269dc94783d55f

Comment 11

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 776406 [details] [diff] [review]
unskip - nightly, aurora

Review of attachment 776406 [details] [diff] [review]:
-----------------------------------------------------------------

Re-enabled:
http://hg.mozilla.org/qa/mozmill-tests/rev/ee401c7796a6 (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/f8783208f5d1 (aurora)

Comment 12

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 776407 [details] [diff] [review]
unskip - beta, release

Review of attachment 776407 [details] [diff] [review]:
-----------------------------------------------------------------

http://hg.mozilla.org/qa/mozmill-tests/rev/2f91220a3207 (beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/1d70a0cbf8d3 (release)
http://hg.mozilla.org/qa/mozmill-tests/rev/d5e646d69443 (esr17)

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Wonderful news. Thanks so much Andrei for working on this! That's a big step forward.