987892 - Clear up oranges for debug mochitest-browser-chrome jobs on mozilla-aurora, mozilla-beta, mozilla-release and mozilla-esr24

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Armen [:armenzg]]

https://tbpl.mozilla.org/?tree=Mozilla-Aurora&jobname=Ubuntu.*debug.*mochitest-browser-chrome

T-FAIL CRASH:
https://tbpl.mozilla.org/php/getParsedLog.php?id=36674042&tree=Mozilla-Aurora&full=1
https://tbpl.mozilla.org/php/getParsedLog.php?id=36673963&tree=Mozilla-Aurora&full=1

Comment 1

[Karl Tomlinson (:karlt)]

This is a duplicate of bug 933680, unless you want to track the different branch in a different bug.

Comment 2

[Armen [:armenzg]]

I think it makes sense as long as bug 933680 blocks bug 864866 for visibility purposes.

jmaher, have any preference?

Updated URL (since we hid the jobs):
https://tbpl.mozilla.org/?tree=Mozilla-Aurora&jobname=Ubuntu.*debug.*mochitest-browser-chrome&showall=1

Comment 3

[Armen [:armenzg]]

jmaher: what are the next steps to get m-a green?
https://tbpl.mozilla.org/?tree=Mozilla-Aurora&jobname=Ubuntu.*debug.*mochitest-browser-chrome&showall=1

Could you also please request for approvals for m-b, m-r & esr24?

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: disable failing/crashing tests on aurora (1.0)

As we have not had any luck getting the fix backported, we need to disable this.  If we can get the fixes backported, I would be happy to turn these tests on.

Note: this patch will work for aurora, but probably not for beta, release or esr as we were not using build time filtering via manifests.

Comment 5

[Armen [:armenzg]]

Comment on attachment 8401285 [details] [diff] [review]
disable failing/crashing tests on aurora (1.0)

Review of attachment 8401285 [details] [diff] [review]:
-----------------------------------------------------------------

What are our options for other release branches?

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

landed on aurora:
https://hg.mozilla.org/releases/mozilla-aurora/rev/fb0a8ce06b17

for beta/release/esr, we need to see what is failing and do makefile cleanup.

Comment 7

[Armen [:armenzg]]

==Status==
* Now that we have m-b, m-r and m-esr24 we can see what needs to be greened up

https://tbpl.mozilla.org/?tree=Mozilla-Aurora&jobname=Ubuntu.*debug.*mochitest-browser-chrome
https://tbpl.mozilla.org/?tree=Mozilla-Beta&jobname=Ubuntu.*debug.*mochitest-browser-chrome&showall=1
https://tbpl.mozilla.org/?tree=Mozilla-Release&jobname=Ubuntu.*debug.*mochitest-browser-chrome&showall=1
https://tbpl.mozilla.org/?tree=Mozilla-Esr24&jobname=Ubuntu.*debug.*mochitest-browser-chrome&showall=1

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: disable 5 tests which are [perma-]failing on beta (1.0)

looking at the brief history we have a clear pattern of failures.  I assume this will work and we won't have others- we won't know until we land.

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

https://hg.mozilla.org/releases/mozilla-beta/rev/13bf6fe8df1f

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: disable 4 tests which are [perma-]failing on beta (1.0)

ok, this is for m-r, bc3 is still leaking, but I want to get bc1 and bc2 green before getting a loaner ec2 instance to figure out the leak on bc3.

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: disable 1 test which is [perma-]failing on esr24 (1.0)

Comment 12

[Joel Maher ( :jmaher ) (UTC -8)]

landed the mozilla-release patch:
https://hg.mozilla.org/releases/mozilla-release/rev/a5cdae95b5c7

Comment 13

[Joel Maher ( :jmaher ) (UTC -8)]

esr24:
https://hg.mozilla.org/releases/mozilla-esr24/rev/b5d5a0c1ef8b


I believe we will have a single failure left to investigate, I will see how these three branches play out over the rest of the day.

Comment 14

[Armen [:armenzg]]

I've unhidden the builders that are green on m-b, m-r and m-esr24 [1].
* m-b is all green
* m-r is all orange
* m-esr24 is failing only one

[1]
https://tbpl.mozilla.org/?tree=Mozilla-Beta&jobname=Ubuntu.*debug.*mochitest-browser-chrome
https://tbpl.mozilla.org/?tree=Mozilla-Release&jobname=Ubuntu.*debug.*mochitest-browser-chrome&showall=1
https://tbpl.mozilla.org/?tree=Mozilla-Esr24&jobname=Ubuntu.*debug.*mochitest-browser-chrome&showall=1

Comment 15

[Joel Maher ( :jmaher ) (UTC -8)]

I am awaiting a loaner machine in bug 993177, to figure out the exact tests to disable.

Comment 16

[Joel Maher ( :jmaher ) (UTC -8)]

I am going to clean up ESR24 and ignore mozilla-release for the sole reason we will be uplifting in 3 weeks, so mozilla-beta will be there.  That should get us off of fedora slaves for browser-chrome by the end of the month.

Comment 17

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: disable browser_connection.js on esr24 (1.0)

on my loaner slave this fixes bc1- we will see if that does it in production.

Comment 18

[Joel Maher ( :jmaher ) (UTC -8)]

landed on esr24:
https://hg.mozilla.org/releases/mozilla-esr24/rev/ba2cd0446388

Comment 19

[Joel Maher ( :jmaher ) (UTC -8)]

esr24 is green!  I believe we are done unless somebody objects to my plan to skip mozilla-release and let the uplift fix it in 3 weeks.

Comment 20

[Armen [:armenzg]]

+1 to plan.

On me to do config changes.
FYI Tomorrow I will be swamped with meetings so I might not have it done right away.

Comment 21

[Jonathan Griffin (:jgriffin)]

+1 to plan from me too.  Thanks guys.