1159566 - browser_telemetry_button_responsive.js is going to permafail when Gecko 40 merges to Aurora

Status: RESOLVED WORKSFORME

(DevTools :: General, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

I'm not sure if this is a test issue or a Telemetry issue. Nor do I see any obvious recent landings that could be at fault.

https://treeherder.mozilla.org/logviewer.html#?job_id=7013389&repo=try

12:27:28 INFO - 5900 INFO Clicking button command-button-responsive
12:27:28 INFO - 5901 INFO Clicking button command-button-responsive
12:27:28 INFO - 5902 INFO Clicking button command-button-responsive
12:27:28 INFO - 5903 INFO TEST-PASS | browser/devtools/shared/test/browser_telemetry_button_responsive.js | DEVTOOLS_RESPONSIVE_OPENED_BOOLEAN has more than one entry
12:27:28 INFO - 5904 INFO TEST-PASS | browser/devtools/shared/test/browser_telemetry_button_responsive.js | All DEVTOOLS_RESPONSIVE_OPENED_BOOLEAN entries are === true
12:27:28 INFO - 5905 INFO TEST-PASS | browser/devtools/shared/test/browser_telemetry_button_responsive.js | Per user value DEVTOOLS_RESPONSIVE_OPENED_PER_USER_FLAG has a single value of true
12:27:28 INFO - 5906 INFO TEST-UNEXPECTED-FAIL | browser/devtools/shared/test/browser_telemetry_button_responsive.js | DEVTOOLS_RESPONSIVE_TIME_ACTIVE_SECONDS has more than one entry -
12:27:28 INFO - Stack trace:
12:27:28 INFO - chrome://mochitests/content/browser/browser/devtools/shared/test/browser_telemetry_button_responsive.js:checkResults:80
12:27:28 INFO - chrome://mochitests/content/browser/browser/devtools/shared/test/browser_telemetry_button_responsive.js:testButton:34
12:27:28 INFO - self-hosted:InterpretGeneratorResume:716
12:27:28 INFO - self-hosted:next:624
12:27:28 INFO - Handler.prototype.process@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:867:23
12:27:28 INFO - this.PromiseWalker.walkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:746:7
12:27:28 INFO - this.PromiseWalker.scheduleWalkerLoop/<@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:688:37
12:27:28 INFO - Promise*this.PromiseWalker.scheduleWalkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:688:5
12:27:28 INFO - this.PromiseWalker.schedulePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:709:7
12:27:28 INFO - this.PromiseWalker.completePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:671:7
12:27:28 INFO - ProfilerFront.prototype._request/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource:///modules/devtools/performance/compatibility.js:66:7
12:27:28 INFO - emit@resource://gre/modules/commonjs/toolkit/loader.js -> resource://gre/modules/commonjs/sdk/event/core.js:104:9
12:27:28 INFO - Request.prototype.emit@resource://gre/modules/devtools/dbg-client.jsm:1195:29
12:27:28 INFO - DebuggerClient.prototype.onPacket@resource://gre/modules/devtools/dbg-client.jsm:1012:7
12:27:28 INFO - LocalDebuggerTransport.prototype.send/<@resource://gre/modules/devtools/dbg-client.jsm -> resource://gre/modules/devtools/transport/transport.js:561:11
12:27:28 INFO - makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://gre/modules/devtools/DevToolsUtils.js:82:14
12:27:28 INFO - makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://gre/modules/devtools/DevToolsUtils.js:82:14

Comment 1

[Brian Grinstead [:bgrins]]

Mike, any ideas what's going on with this telemetry test?

Comment 2

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

Seems to be related to the previous test throwing an exception. The first time we toggle paintflashing on and off context.environment is set but the next time we toggle it on context.environment is undefined.

Seems like runAt support was added in http://hg.mozilla.org/mozilla-central/diff/6ecd33215340/toolkit/devtools/gcli/commands/paintflashing.js so asking Joe if he has any insights.

12:27:23 INFO - 5885 INFO TEST-START | browser/devtools/shared/test/browser_telemetry_button_paintflashing.js
...
12:27:25 INFO - console.error:
12:27:25 INFO - Message: TypeError: context.environment is undefined
12:27:25 INFO - Stack:
12:27:25 INFO - exports.items<.exec@resource://gre/modules/commonjs/toolkit/loader.js -> resource://gre/modules/devtools/gcli/commands/paintflashing.js:160:7
12:27:25 INFO - exports.exec@resource://gre/modules/commonjs/toolkit/loader.js -> resource://gre/modules/devtools/gcli/util/host.js:72:10
12:27:25 INFO - Requisition.prototype.exec@resource://gre/modules/commonjs/toolkit/loader.js -> resource://gre/modules/devtools/gcli/cli.js:2072:14
12:27:25 INFO - Requisition.prototype.updateExec/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://gre/modules/devtools/gcli/cli.js:2112:12
12:27:25 INFO - Handler.prototype.process@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:867:23
12:27:25 INFO - this.PromiseWalker.walkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:746:7
12:27:25 INFO - this.PromiseWalker.scheduleWalkerLoop/<@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:688:37
12:27:25 INFO - Promise*this.PromiseWalker.scheduleWalkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:688:5
12:27:25 INFO - this.PromiseWalker.schedulePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:709:7
12:27:25 INFO - Promise.prototype.then@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:417:5
12:27:25 INFO - Requisition.prototype.update@resource://gre/modules/commonjs/toolkit/loader.js -> resource://gre/modules/devtools/gcli/cli.js:1511:1
12:27:25 INFO - Requisition.prototype.updateExec@resource://gre/modules/commonjs/toolkit/loader.js -> resource://gre/modules/devtools/gcli/cli.js:2111:10
12:27:25 INFO - CommandUtils.createButtons/</</<@resource:///modules/devtools/DeveloperToolbar.jsm:134:11
12:27:25 INFO - delayedClick@chrome://mochitests/content/browser/browser/devtools/shared/test/browser_telemetry_button_paintflashing.js:44:7
...
12:27:26 INFO - 5886 INFO TEST-OK | browser/devtools/shared/test/browser_telemetry_button_paintflashing.js | took 2942ms

Comment 3

[Joe Walker [:jwalker]]

2 things are confusing me. Firstly why do we not get the `info("Clicking button " + node.id);` from line 43 in the output logs? And secondly, why does Mike say this worked the first time through - we've got 2 stack traces in the output, right?

Comment 4

[Ryan VanderMeulen [:RyanVM]]

Any news here? The uplift is Monday.

Comment 5

[Brian Grinstead [:bgrins]]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #4)
> Any news here? The uplift is Monday.

I have a new try push going here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=e28f227d37ca.  I'll also see if I can reproduce locally - it seems weird this is only an issue when running an aurora build.

Comment 6

[Brian Grinstead [:bgrins]]

(In reply to Michael Ratcliffe [:miker] [:mratcliffe] from comment #2)
> Seems to be related to the previous test throwing an exception. The first
> time we toggle paintflashing on and off context.environment is set but the
> next time we toggle it on context.environment is undefined.

I don't think this is the case.  I just reproduced locally with the simulation patch applied and `mochitest-devtools browser/devtools/shared/test/browser_telemetry_button_responsive.js`

Comment 7

[Brian Grinstead [:bgrins]]

So I don't know why or how (since it fails locally with the simulation patch applied) but it seems to be passing on the most recent try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=e28f227d37ca.  I'm seeing the spam from Bug 1162961 but not this test failing.

Comment 8

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

Lets look at two try pushes: one simulating Aurora and one not:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4a765f7a2e77 (Simulating Aurora)

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b1b0ecb67e1d (Default try push)

Comment 9

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Tests disabled on beta for permafail:

https://hg.mozilla.org/releases/mozilla-beta/rev/1a465b2ef025

Comment 10

[Ryan VanderMeulen [:RyanVM]]

Disabled again on Beta41.

https://hg.mozilla.org/releases/mozilla-beta/rev/b8bcbc25809b

Joe, can you please find someone to look into this? Otherwise I'm inclined to disable the test across the board given the lack of motivation in getting it fixed.

Comment 11

[Joe Walker [:jwalker]]

Mike - could you recommend what we should do with this please.

Comment 12

[Joe Walker [:jwalker]]

ni? ping.

Comment 13

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

These are tilt and responsive design tests.

Tilt doesn't work at all on e10s so I would suggest purging Tilt and the tests from the browser... there is little point in fixing a bug on an unused feature.

I believe that the responsive design is jryans area of interest so he is the best person to ask about what we should do with the responsive design test.

Comment 14

[Ryan VanderMeulen [:RyanVM]]

I'll double-check later to confirm (I'm not easily in a position to do so at the moment), but I think this actually went away on its own and can be closed out as worksforme. Setting a ni? on myself to verify that.

Comment 15

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

I will await an update from :RyanVM.

Comment 16

[Ryan VanderMeulen [:RyanVM]]

Yeah, this failure is gone now.