1020473 - Make Jetpack meet the TBPL visibility requirements

Status: RESOLVED FIXED

(Add-on SDK Graveyard :: General, defect, P1)

Description

[Ed Morley [:emorley]]

At present, Jetpack sadly doesn't meet the visibility requirements for TBPL.

Notably:
https://wiki.mozilla.org/Sheriffing/Job_Visibility_Policy#6.29_Outputs_failures_in_a_TBPL-starrable_format

In the form of bug 1020458 and bug 1020463.

In addition, the intermittent failure rate is quite high at present.

As such Jetpack, has been hidden from the default TBPL view for all trunk trees (it was already hidden on Windows 7 for bug 942111 / bug 981003).

Comment 1

[Ed Morley [:emorley]]

To view the jobs, you'll need to use the &showall=1 param:

https://tbpl.mozilla.org/?showall=1&jobname=jetpack
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&showall=1&jobname=jetpack
https://tbpl.mozilla.org/?tree=B2g-Inbound&showall=1&jobname=jetpack
https://tbpl.mozilla.org/?tree=Fx-Team&showall=1&jobname=jetpack

Comment 2

[Ed Morley [:emorley]]

18:40 <philor> edmorley|sheriffduty: and once they finish timing out, https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=jetpack&showall=1&fromchange=223a185a7ee2&tochange=bf5c893c2733 should show that it was bug 997894 that we didn't see breaking jetpack

I've backed out bug 997894 - however the overall failure rate and the lack of timeout handling/test names given as 'unknown' are still problematic.

Comment 3

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

Hidden on try to match inbound

Comment 4

[Ed Morley [:emorley]]

Jetpack tests are currently orange on every platform on trunk, was regressed by:
https://tbpl.mozilla.org/?tree=Fx-Team&showall=1&jobname=jetpack&rev=fea2a90e2d22

Comment 5

[Erik Vold [:erikvold] (please needinfo? me)]

For the record, the test-child_process tests have been disabled, https://github.com/mozilla/addon-sdk/commit/8e68d49c71bac4202c689d6d677a8aed77f05dd4

So bug 979675, bug 979334, and bug 979039 should no longer be an issue at the moment, and I think we should remove them as blockers for this now at least.

Does this sound ok to you Ed?

Comment 6

[Ed Morley [:emorley]]

Yeah, if tests have been disabled, the intermittent (or permanent) failure bugs for them are definitely not blockers to this any more :-)

JP is still orange though:
https://tbpl.mozilla.org/?tree=Fx-Team&showall=1&jobname=jetpack

Comment 7

[Erik Vold [:erikvold] (please needinfo? me)]

I wanted to try to work on this more, but I'm not sure that I will be able to now.

Comment 8

[(Away)]

> JP is still orange though:
> https://tbpl.mozilla.org/?tree=Fx-Team&showall=1&jobname=jetpack

Shu, this sounds like temporal dead zone stuff, right?

TypeError: redeclaration of variable sliceTest

Comment 9

[Dave Townsend [:mossop]]

(In reply to David Major [:dmajor] from comment #8)
> > JP is still orange though:
> > https://tbpl.mozilla.org/?tree=Fx-Team&showall=1&jobname=jetpack
> 
> Shu, this sounds like temporal dead zone stuff, right?
> 
> TypeError: redeclaration of variable sliceTest

Most of that stuff has already been fixed in github and needs to be uplifted.

Comment 10

[Shu-yu Guo [:shu]]

That's right, if it's a newly surfaced "redeclaration" error, it's almost guaranteed to be TDZ.

Comment 11

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-09-26T02:34:24
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=try&job_id=2045491
repository: try
who: btseng[at]mozilla[dot]com
machine: tst-linux32-spot-054
buildname: Ubuntu VM 12.04 try opt test jetpack
revision: a0a2261d5632

TypeError: redeclaration of variable sliceTest
TypeError: redeclaration of variable sliceTest
TEST-UNEXPECTED-FAIL | addon-sdk/tests/test-browser-events.test browser events ignore other wins | Test output exceeded timeout (300s).
Exception: Test output exceeded timeout (300s).

Comment 12

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-09-26T02:34:49
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=try&job_id=2046569
repository: try
who: btseng[at]mozilla[dot]com
machine: tst-linux32-spot-276
buildname: Ubuntu VM 12.04 try debug test jetpack
revision: a0a2261d5632

TypeError: redeclaration of variable sliceTest
TypeError: redeclaration of variable sliceTest
TEST-UNEXPECTED-FAIL | addon-sdk/tests/test-browser-events.test browser events ignore other wins | Test output exceeded timeout (300s).
Exception: Test output exceeded timeout (300s).

Comment 13

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-09-26T02:35:04
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=try&job_id=2045401
repository: try
who: btseng[at]mozilla[dot]com
machine: tst-linux64-spot-617
buildname: Ubuntu VM 12.04 x64 try opt test jetpack
revision: a0a2261d5632

TypeError: redeclaration of variable sliceTest
TypeError: redeclaration of variable sliceTest
TEST-UNEXPECTED-FAIL | addon-sdk/tests/test-browser-events.test browser events ignore other wins | Test output exceeded timeout (300s).
Exception: Test output exceeded timeout (300s).

Comment 14

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-09-26T02:35:13
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=try&job_id=2045621
repository: try
who: btseng[at]mozilla[dot]com
machine: tst-linux64-spot-878
buildname: Ubuntu VM 12.04 x64 try debug test jetpack
revision: a0a2261d5632

TypeError: redeclaration of variable sliceTest
TypeError: redeclaration of variable sliceTest
TEST-UNEXPECTED-FAIL | addon-sdk/tests/test-browser-events.test browser events ignore other wins | Test output exceeded timeout (300s).
Exception: Test output exceeded timeout (300s).

Comment 15

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-09-26T02:35:26
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=try&job_id=2046346
repository: try
who: btseng[at]mozilla[dot]com
machine: t-snow-r4-0033
buildname: Rev4 MacOSX Snow Leopard 10.6 try opt test jetpack
revision: a0a2261d5632

TypeError: redeclaration of variable sliceTest
TypeError: redeclaration of variable sliceTest
TEST-UNEXPECTED-FAIL | addon-sdk/tests/test-browser-events.test browser events ignore other wins | Test output exceeded timeout (300s).
Exception: Test output exceeded timeout (300s).

Comment 16

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-09-26T02:35:38
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=try&job_id=2045472
repository: try
who: btseng[at]mozilla[dot]com
machine: t-snow-r4-0131
buildname: Rev4 MacOSX Snow Leopard 10.6 try debug test jetpack
revision: a0a2261d5632

TypeError: redeclaration of variable sliceTest
TypeError: redeclaration of variable sliceTest
TEST-UNEXPECTED-FAIL | addon-sdk/tests/test-browser-events.test browser events ignore other wins | Test output exceeded timeout (300s).
Exception: Test output exceeded timeout (300s).

Comment 17

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-09-26T02:35:50
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=try&job_id=2046101
repository: try
who: btseng[at]mozilla[dot]com
machine: t-xp32-ix-003
buildname: Windows XP 32-bit try opt test jetpack
revision: a0a2261d5632

TypeError: redeclaration of variable sliceTest
TypeError: redeclaration of variable sliceTest
TEST-UNEXPECTED-FAIL | addon-sdk/tests/test-browser-events.test browser events ignore other wins | Test output exceeded timeout (300s).
Exception: Test output exceeded timeout (300s).

Comment 18

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-09-26T02:35:56
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=try&job_id=2046458
repository: try
who: btseng[at]mozilla[dot]com
machine: t-xp32-ix-027
buildname: Windows XP 32-bit try debug test jetpack
revision: a0a2261d5632

TypeError: redeclaration of variable sliceTest
TypeError: redeclaration of variable sliceTest
TEST-UNEXPECTED-FAIL | addon-sdk/tests/test-browser-events.test browser events ignore other wins | Test output exceeded timeout (300s).
Exception: Test output exceeded timeout (300s).

Comment 19

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-09-26T02:36:02
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=try&job_id=2046016
repository: try
who: btseng[at]mozilla[dot]com
machine: t-w732-ix-074
buildname: Windows 7 32-bit try opt test jetpack
revision: a0a2261d5632

TypeError: redeclaration of variable sliceTest
TypeError: redeclaration of variable sliceTest
TEST-UNEXPECTED-FAIL | addon-sdk/tests/test-browser-events.test browser events ignore other wins | Test output exceeded timeout (300s).
Exception: Test output exceeded timeout (300s).

Comment 20

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-09-26T02:36:08
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=try&job_id=2046457
repository: try
who: btseng[at]mozilla[dot]com
machine: t-w732-ix-045
buildname: Windows 7 32-bit try debug test jetpack
revision: a0a2261d5632

TypeError: redeclaration of variable sliceTest
TypeError: redeclaration of variable sliceTest
TEST-UNEXPECTED-FAIL | addon-sdk/tests/test-browser-events.test browser events ignore other wins | Test output exceeded timeout (300s).
Exception: Test output exceeded timeout (300s).

Comment 21

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-09-26T02:36:14
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=try&job_id=2046041
repository: try
who: btseng[at]mozilla[dot]com
machine: t-w864-ix-122
buildname: WINNT 6.2 try opt test jetpack
revision: a0a2261d5632

TypeError: redeclaration of variable sliceTest
TypeError: redeclaration of variable sliceTest
TEST-UNEXPECTED-FAIL | addon-sdk/tests/test-browser-events.test browser events ignore other wins | Test output exceeded timeout (300s).
Exception: Test output exceeded timeout (300s).

Comment 22

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-09-26T02:36:21
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=try&job_id=2046509
repository: try
who: btseng[at]mozilla[dot]com
machine: t-w864-ix-026
buildname: WINNT 6.2 try debug test jetpack
revision: a0a2261d5632

TypeError: redeclaration of variable sliceTest
TypeError: redeclaration of variable sliceTest
TEST-UNEXPECTED-FAIL | addon-sdk/tests/test-browser-events.test browser events ignore other wins | Test output exceeded timeout (300s).
Exception: Test output exceeded timeout (300s).

Comment 23

[Dave Townsend [:mossop]]

Ugh, having tbpl spam this bug is extremely unhelpful

Comment 24

[Ryan VanderMeulen [:RyanVM]]

Not arguing with you on that (not sure why he's starring jobs that are supposed to be hidden anyway), but FYI, you can hide the comments entirely using the "Hide TinderboxPushlog Comments" to the right.

Comment 25

[Ed Morley [:emorley]]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #24)
> Not arguing with you on that (not sure why he's starring jobs that are
> supposed to be hidden anyway), but FYI, you can hide the comments entirely
> using the "Hide TinderboxPushlog Comments" to the right.

The TBPLbot comments are treeherder-style message formatting; https://treeherder.mozilla.org/ui/#/jobs?repo=try&searchQuery=jetpack has jetpack jobs not hidden - could you tweak the profile? :-)

Comment 26

[Ryan VanderMeulen [:RyanVM]]

Hidden on Try

Comment 27

[Dave Townsend [:mossop]]

I'm going to assert that with mochitest-jetpack we're now at the point where jetpack tests meet the required standards. Opt builds have been green since they were turned on, aside from some windows failures which were actually detecting a regression in mozilla-central. There may still be some intermittents left but they are low enough that I don't think they should block us further at this point. We should start disabling tests if they are too intermittent.