Intermittent browser_webconsole_check_stubs_network_event.js | The networkEvent stubs file needs to be updated by running `mach test devtools/client/webconsole/new-console-outp

RESOLVED FIXED in Firefox 55

Status

()

Firefox
Developer Tools: Console
P3
normal
RESOLVED FIXED
a year ago
10 months ago

People

(Reporter: Treeherder Bug Filer, Assigned: nchevobbe)

Tracking

({intermittent-failure})

unspecified
Firefox 56
intermittent-failure
Points:
---

Firefox Tracking Flags

(firefox-esr52 unaffected, firefox54 unaffected, firefox55 fixed, firefox56 fixed)

Details

(Whiteboard: [stockwell fixed])

MozReview Requests

()

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(1 attachment)

Comment 1

a year ago
8 failures in 812 pushes (0.01 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* mozilla-inbound: 3
* autoland: 3
* mozilla-central: 1
* graphics: 1

Platform breakdown:
* linux64: 7
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-02-20&endday=2017-02-26&tree=all

Comment 2

a year ago
11 failures in 783 pushes (0.014 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* autoland: 6
* mozilla-inbound: 2
* try: 1
* oak: 1
* graphics: 1

Platform breakdown:
* linux64: 10
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-02-27&endday=2017-03-05&tree=all

Comment 3

a year ago
11 failures in 790 pushes (0.014 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 5
* try: 2
* mozilla-central: 2
* mozilla-aurora: 1
* autoland: 1

Platform breakdown:
* linux32: 6
* linux64: 3
* osx-10-10: 1
* linux32-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-03-06&endday=2017-03-12&tree=all

Comment 4

a year ago
10 failures in 777 pushes (0.013 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 4
* mozilla-inbound: 2
* try: 1
* oak: 1
* mozilla-central: 1
* mozilla-aurora: 1

Platform breakdown:
* linux64: 7
* linux32: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-03-13&endday=2017-03-19&tree=all

Comment 5

a year ago
10 failures in 845 pushes (0.012 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 3
* mozilla-inbound: 2
* mozilla-central: 2
* mozilla-aurora: 2
* try: 1

Platform breakdown:
* linux64: 6
* linux64-nightly: 3
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-03-27&endday=2017-04-02&tree=all
Priority: -- → P3

Comment 6

a year ago
5 failures in 894 pushes (0.006 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 2
* mozilla-inbound: 1
* mozilla-central: 1
* mozilla-aurora: 1

Platform breakdown:
* linux32: 3
* linux64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-04-10&endday=2017-04-16&tree=all

Comment 7

a year ago
7 failures in 770 pushes (0.009 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 3
* mozilla-inbound: 2
* try: 1
* mozilla-central: 1

Platform breakdown:
* linux64: 5
* linux32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-05-01&endday=2017-05-07&tree=all
this will increase in frequency as we are going to be running the devtools tests on ubuntu 16.04 today- I don't think it will be outrageous, but probably <50/week.

Comment 9

a year ago
17 failures in 879 pushes (0.019 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 6
* graphics: 3
* autoland: 3
* try: 2
* oak: 1
* mozilla-central: 1
* jamun: 1

Platform breakdown:
* linux64: 10
* linux32: 4
* linux64-nightly: 2
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-05-08&endday=2017-05-14&tree=all
Summary: Intermittent devtools/client/webconsole/new-console-output/test/fixtures/stub-generators/browser_webconsole_check_stubs_network_event.js | The networkEvent stubs file needs to be updated by running `mach test devtools/client/webconsole/new-console-outp → Intermittent browser_webconsole_check_stubs_network_event.js | The networkEvent stubs file needs to be updated by running `mach test devtools/client/webconsole/new-console-outp
13 failures in 777 pushes (0.017 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 6
* autoland: 4
* try: 1
* mozilla-central: 1
* graphics: 1

Platform breakdown:
* linux64: 8
* linux32: 5

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-05-15&endday=2017-05-21&tree=all
15 failures in 891 pushes (0.017 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 8
* mozilla-beta: 3
* mozilla-central: 2
* try: 1
* autoland: 1

Platform breakdown:
* linux64: 8
* linux32: 4
* linux64-nightly: 2
* linux64-devedition: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-05-22&endday=2017-05-28&tree=all
8 failures in 820 pushes (0.01 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 4
* autoland: 3
* try: 1

Platform breakdown:
* linux64: 6
* linux32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-05-29&endday=2017-06-04&tree=all
15 failures in 132 pushes (0.114 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* try: 9
* mozilla-inbound: 3
* autoland: 2
* mozilla-central: 1

Platform breakdown:
* windows7-32-vm: 7
* linux64: 7
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-06-06&endday=2017-06-06&tree=all
this test case has increased in failures in the last week or so, 44 failures looking at the past week (orange factor will post tomorrow night the official number).

this is primarily on linux 32/64 opt/pgo and e10s (with a few non-e10s failures mixed in).

here is a log file:
https://treeherder.mozilla.org/logviewer.html#?repo=try&job_id=106005790&lineNumber=3022

and a related screenshot:
https://public-artifacts.taskcluster.net/U17-cNtpQKSX2FTr8FBmkQ/0/public/test_info//mozilla-test-fail-screenshot_ra8FXY.png

this is what I see in the log:
task 2017-06-10T01:01:58.031994Z] 01:01:58     INFO - TEST-START | devtools/client/webconsole/new-console-output/test/fixtures/stub-generators/browser_webconsole_check_stubs_network_event.js
[task 2017-06-10T01:01:59.278889Z] 01:01:59     INFO - TEST-INFO | started process screentopng
[task 2017-06-10T01:01:59.782362Z] 01:01:59     INFO - TEST-INFO | screentopng: exit 0
[task 2017-06-10T01:01:59.782637Z] 01:01:59     INFO - Buffered messages logged at 01:01:58
[task 2017-06-10T01:01:59.782819Z] 01:01:59     INFO - Entering test bound 
[task 2017-06-10T01:01:59.783626Z] 01:01:59     INFO - Adding a new tab with URL: http://example.com/browser/devtools/client/webconsole/new-console-output/test/fixtures/stub-generators/test-network-event.html
[task 2017-06-10T01:01:59.783708Z] 01:01:59     INFO - Tab added and finished loading
[task 2017-06-10T01:01:59.784487Z] 01:01:59     INFO - Opening the toolbox
[task 2017-06-10T01:01:59.784660Z] 01:01:59     INFO - Toolbox opened and focused
[task 2017-06-10T01:01:59.785429Z] 01:01:59     INFO - Buffered messages logged at 01:01:59
[task 2017-06-10T01:01:59.786146Z] 01:01:59     INFO - Removing tab.
[task 2017-06-10T01:01:59.789221Z] 01:01:59     INFO - Waiting for event: 'TabClose' on [object XULElement].
[task 2017-06-10T01:01:59.791322Z] 01:01:59     INFO - Got event: 'TabClose' on [object XULElement].
[task 2017-06-10T01:01:59.793120Z] 01:01:59     INFO - Tab removed and finished closing
[task 2017-06-10T01:01:59.795076Z] 01:01:59     INFO - Buffered messages finished
[task 2017-06-10T01:01:59.797105Z] 01:01:59     INFO - TEST-UNEXPECTED-FAIL | devtools/client/webconsole/new-console-output/test/fixtures/stub-generators/browser_webconsole_check_stubs_network_event.js | The networkEvent stubs file needs to be updated by running `mach test devtools/client/webconsole/new-console-output/test/fixtures/stub-generators/browser_webconsole_update_stubs_network_event.js` - 
[task 2017-06-10T01:01:59.798774Z] 01:01:59     INFO - Stack trace:
[task 2017-06-10T01:01:59.800389Z] 01:01:59     INFO -     chrome://mochitests/content/browser/devtools/client/webconsole/new-console-output/test/fixtures/stub-generators/browser_webconsole_check_stubs_network_event.js:null:17
[task 2017-06-10T01:01:59.803034Z] 01:01:59     INFO -     Tester_execTest@chrome://mochikit/content/browser-test.js:801:9
[task 2017-06-10T01:01:59.804911Z] 01:01:59     INFO -     Tester.prototype.nextTest</<@chrome://mochikit/content/browser-test.js:714:7
[task 2017-06-10T01:01:59.806076Z] 01:01:59     INFO -     SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:795:59
[task 2017-06-10T01:01:59.807314Z] 01:01:59     INFO - Leaving test bound 
[task 2017-06-10T01:01:59.808449Z] 01:01:59     INFO - GECKO(2105) | MEMORY STAT | vsize 2165MB | residentFast 344MB | heapAllocated 141MB
[task 2017-06-10T01:01:59.809662Z] 01:01:59     INFO - TEST-OK | devtools/client/webconsole/new-console-output/test/fixtures/stub-generators/browser_webconsole_check_stubs_network_event.js | took 1266ms


is it possible we are not generating the stubs file during the build or maybe not seeing it in the uploaded package?

:bgrins, could you take a look at this sometime in the next 2 weeks?
Flags: needinfo?(bgrinstead)
Whiteboard: [stockwell needswork]
49 failures in 864 pushes (0.057 failures/push) were associated with this bug in the last 7 days. 

This is the #27 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 18
* mozilla-inbound: 16
* try: 10
* mozilla-central: 5

Platform breakdown:
* linux64: 29
* linux32: 11
* windows7-32-vm: 7
* linux64-nightly: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-06-05&endday=2017-06-11&tree=all
Comment hidden (mozreview-request)
(Assignee)

Updated

a year ago
Assignee: nobody → nchevobbe
Status: NEW → ASSIGNED
(Assignee)

Comment 17

a year ago
This is due to network update not being fired everytime in the same order.
In the test we assume that the network update "eventTiming" is the last one but it might not be the case.

This is a bit tricky here, my only idea would be to not add stubs for those network updates, or at least, maybe not converting them to messages and only hand pick what we need (httpVersion, totalTime, responseStatusText, responseStatusCode)
Flags: needinfo?(bgrinstead)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
(Assignee)

Comment 24

a year ago
The last patch should fix the intermittent.
Here's a TRY patch of the previous one (there was only an eslint error I had to fix) : https://treeherder.mozilla.org/#/jobs?repo=try&revision=14b56d0ecdfd97b412c8663b1f65f8e9b23cf8bf&selectedJob=107018177

Comment 25

a year ago
mozreview-review
Comment on attachment 8876596 [details]
Bug 1341308 - Fix network stub check test intermittent.

https://reviewboard.mozilla.org/r/147926/#review153558

I'm going to forward this to Honza since he knows the network stuff better than me

::: devtools/client/webconsole/new-console-output/test/fixtures/stubs/networkEvent.js
(Diff revision 7)
>        "url": "http://example.com/browser/devtools/client/webconsole/new-console-output/test/fixtures/stub-generators/inexistent.html",
>        "method": "GET",
>        "headersSize": 489
>      },
> -    "isXHR": false,
> -    "cause": {

What did `cause` get removed here?
Attachment #8876596 - Flags: review?(bgrinstead) → review?(odvarko)
(Assignee)

Comment 26

a year ago
mozreview-review-reply
Comment on attachment 8876596 [details]
Bug 1341308 - Fix network stub check test intermittent.

https://reviewboard.mozilla.org/r/147926/#review153558

> What did `cause` get removed here?

it's because we now only get what we actually use in the messages for the network update packet.
Doing that we're safe with property ordering and things like that.
I think this isn't much of a big deal, because if the stub change in a way that would break the console, this would be caught here.

Do you think of anything that could go wrong that I haven't thought of ?

Comment 27

a year ago
mozreview-review
Comment on attachment 8876596 [details]
Bug 1341308 - Fix network stub check test intermittent.

https://reviewboard.mozilla.org/r/147926/#review153964

Couple of inline questions, but I think this is ready.

R+

Thanks Nicolas!

Honza

::: devtools/client/webconsole/new-console-output/new-console-output-wrapper.js:190
(Diff revision 7)
>  
>    dispatchMessageUpdate: function (message, res) {
> -    // network-message-updated will emit when eventTimings message arrives
> -    // which is the last one of 8 updates happening on network message update.
> -    if (res.packet.updateType === "eventTimings") {
> +    // network-message-updated will emit when all the update message arrives.
> +    // Since we can't ensure the order of the network update, we check
> +    // that networkInfo.updates has all we need.
> +    const NUMBER_OF_NETWORK_UPDATE = 8;

Counting number of updates feels risky. At least, it should happen on the backend and perhaps the backend should be responsible for sending an update like "done". But, I am not going to block on this.

Also, does this also mean that in case of long response body download the UI is updated only when the download is finished? I.e. the user can't observe how the response bar in the timeline column is nicely growing, right?

(I guess it's the same behavior as before)

::: devtools/client/webconsole/new-console-output/test/fixtures/stubs/networkEvent.js:83
(Diff revision 7)
> -stubPreparedMessages.set("XHR GET request eventTimings", new NetworkEventMessage({
> +stubPreparedMessages.set("XHR GET request update", new NetworkEventMessage({
>    "id": "1",
> -  "actor": "server1.conn1.child1/netEvent30",
> +  "actor": "server1.conn0.child1/netEvent31",
>    "level": "log",
> -  "isXHR": true,
>    "request": {

How come we don't need the 'isXHR' field? Note that the key (title) still says "XHR GET request update"
Attachment #8876596 - Flags: review?(odvarko) → review+
(Assignee)

Comment 28

a year ago
mozreview-review-reply
Comment on attachment 8876596 [details]
Bug 1341308 - Fix network stub check test intermittent.

https://reviewboard.mozilla.org/r/147926/#review153964

> Counting number of updates feels risky. At least, it should happen on the backend and perhaps the backend should be responsible for sending an update like "done". But, I am not going to block on this.
> 
> Also, does this also mean that in case of long response body download the UI is updated only when the download is finished? I.e. the user can't observe how the response bar in the timeline column is nicely growing, right?
> 
> (I guess it's the same behavior as before)

Yes, we don't have this kind of things in the console at the moment.
The only thing we do is:
- show the original information we get from the initial network packet
- show the totalTime, statusText, statusCode and httpVersion when we get them.

I think we discussed integration some network inspector component in the future to have the kind of feature you're describing
(In reply to Nicolas Chevobbe [:nchevobbe] from comment #26)
> Comment on attachment 8876596 [details]
> Bug 1341308 - Fix network stub check test intermittent.
> 
> https://reviewboard.mozilla.org/r/147926/#review153558
> 
> > What did `cause` get removed here?
> 
> it's because we now only get what we actually use in the messages for the
> network update packet.
> Doing that we're safe with property ordering and things like that.
> I think this isn't much of a big deal, because if the stub change in a way
> that would break the console, this would be caught here.
> 
> Do you think of anything that could go wrong that I haven't thought of ?

Thanks - I don't have any particular concerns with it, just wanted to understand why it went away
Comment hidden (mozreview-request)

Comment 31

a year ago
mozreview-review
Comment on attachment 8876596 [details]
Bug 1341308 - Fix network stub check test intermittent.

https://reviewboard.mozilla.org/r/147926/#review154038
Attachment #8876596 - Flags: review?(bgrinstead) → review+

Comment 32

a year ago
Pushed by nchevobbe@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6bf02efb9d13
Fix network stub check test intermittent. r=bgrins,Honza

Comment 33

a year ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/6bf02efb9d13
Status: ASSIGNED → RESOLVED
Last Resolved: a year ago
status-firefox56: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 56

Comment 34

11 months ago
33 failures in 814 pushes (0.041 failures/push) were associated with this bug in the last 7 days.   

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 9
* mozilla-inbound: 8
* mozilla-central: 6
* try: 5
* mozilla-beta: 5

Platform breakdown:
* linux64: 18
* linux32: 7
* linux64-nightly: 3
* linux32-nightly: 3
* linux64-devedition: 1
* linux32-devedition: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-06-12&endday=2017-06-18&tree=all
status-firefox54: --- → unaffected
status-firefox55: --- → affected
status-firefox-esr52: --- → unaffected

Comment 35

11 months ago
bugherderuplift
https://hg.mozilla.org/releases/mozilla-beta/rev/bb9de44b0906
status-firefox55: affected → fixed
Whiteboard: [stockwell needswork] → [stockwell fixed]

Comment 36

11 months ago
1 failures in 718 pushes (0.001 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-central: 1

Platform breakdown:
* linux64-jsdcov: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-06-26&endday=2017-07-02&tree=all

Comment 37

10 months ago
1 failures in 1008 pushes (0.001 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-central: 1

Platform breakdown:
* windows8-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-07-24&endday=2017-07-30&tree=all

Comment 38

10 months ago
1 failures in 901 pushes (0.001 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-beta: 1

Platform breakdown:
* linux64-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1341308&startday=2017-08-07&endday=2017-08-13&tree=all
You need to log in before you can comment on or make changes to this bug.