1448484 - Intermittent devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite1 is "Unset" - Got undefined, expected Unset

Status: RESOLVED INCOMPLETE

(DevTools :: Storage Inspector, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: ncsoregi [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=169970387&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/Bn5cQ3Z6TCeHMSNM18Yuyw/runs/0/artifacts/public/logs/live_backing.log

20:07:23     INFO -  131 INFO TEST-START | devtools/client/storage/test/browser_storage_cookies_samesite.js
20:07:24     INFO -  TEST-INFO | started process screenshot
20:07:24     INFO -  TEST-INFO | screenshot: exit 0
20:07:24     INFO -  Buffered messages logged at 20:07:23
20:07:24     INFO -  132 INFO Entering test bound
20:07:24     INFO -  133 INFO Adding a new tab with URL: http://test1.example.org/browser/devtools/client/storage/test/storage-cookies-samesite.html
20:07:24     INFO -  Buffered messages logged at 20:07:24
20:07:24     INFO -  134 INFO Tab added and finished loading
20:07:24     INFO -  135 INFO Found a window: complete
20:07:24     INFO -  136 INFO Opening the storage inspector
20:07:24     INFO -  137 INFO Opening the toolbox
20:07:24     INFO -  138 INFO Console message: [JavaScript Warning: "XUL box for vbox element contained an inline label child, forcing all its children to be wrapped in a block." {file: "chrome://devtools/content/storage/storage.xul" line: 0}]
20:07:24     INFO -  139 INFO Waiting for the stores to update
20:07:24     INFO -  140 INFO Making sure that the toolbox's frame is focused
20:07:24     INFO -  141 INFO Selecting tree item cookies > http://test1.example.org
20:07:24     INFO -  142 INFO "cookies,http://test1.example.org" is already selected, returning.
20:07:24     INFO -  143 INFO TEST-PASS | devtools/client/storage/test/browser_storage_cookies_samesite.js | There is correct number of rows in cookies > http://test1.example.org -
20:07:24     INFO -  144 INFO TEST-PASS | devtools/client/storage/test/browser_storage_cookies_samesite.js | There is item with name 'test1{9d414cc5-8319-0a04-0586-c0a6ae01670a}test1.example.org{9d414cc5-8319-0a04-0586-c0a6ae01670a}/browser/devtools/client/storage/test/' in cookies > http://test1.example.org -
20:07:24     INFO -  145 INFO TEST-PASS | devtools/client/storage/test/browser_storage_cookies_samesite.js | There is item with name 'test2{9d414cc5-8319-0a04-0586-c0a6ae01670a}test1.example.org{9d414cc5-8319-0a04-0586-c0a6ae01670a}/browser/devtools/client/storage/test/' in cookies > http://test1.example.org -
20:07:24     INFO -  146 INFO TEST-PASS | devtools/client/storage/test/browser_storage_cookies_samesite.js | There is item with name 'test3{9d414cc5-8319-0a04-0586-c0a6ae01670a}test1.example.org{9d414cc5-8319-0a04-0586-c0a6ae01670a}/browser/devtools/client/storage/test/' in cookies > http://test1.example.org -
20:07:24     INFO -  Buffered messages finished
20:07:24    ERROR -  147 INFO TEST-UNEXPECTED-FAIL | devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite1 is "Unset" - Got undefined, expected Unset
20:07:24     INFO -  Stack trace:
20:07:24     INFO -  chrome://mochikit/content/browser-test.js:test_is:1280
20:07:24     INFO -  chrome://mochitests/content/browser/devtools/client/storage/test/browser_storage_cookies_samesite.js:null:32
20:07:24     INFO -  Not taking screenshot here: see the one that was previously logged
20:07:24    ERROR -  148 INFO TEST-UNEXPECTED-FAIL | devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite2 is "Lax" - Got undefined, expected Lax
20:07:24     INFO -  Stack trace:
20:07:24     INFO -  chrome://mochikit/content/browser-test.js:test_is:1280
20:07:24     INFO -  chrome://mochitests/content/browser/devtools/client/storage/test/browser_storage_cookies_samesite.js:null:33
20:07:24     INFO -  Not taking screenshot here: see the one that was previously logged
20:07:24    ERROR -  149 INFO TEST-UNEXPECTED-FAIL | devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite3 is "Strict" - Got undefined, expected Strict

Comment 1

[Intermittent Failures Robot]

3 failures in 822 pushes (0.004 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 3

Platform breakdown:
* windows10-64: 2
* windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-03-20&endday=2018-03-26&tree=trunk

Comment 2

[Intermittent Failures Robot]

10 failures in 718 pushes (0.014 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 6
* try: 2
* autoland: 2

Platform breakdown:
* windows10-64: 7
* windows7-32: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-03-26&endday=2018-04-01&tree=trunk

Comment 3

[Intermittent Failures Robot]

2 failures in 765 pushes (0.003 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1
* autoland: 1

Platform breakdown:
* windows7-32: 1
* windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-04-02&endday=2018-04-08&tree=trunk

Comment 4

[Intermittent Failures Robot]

9 failures in 792 pushes (0.011 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 6
* mozilla-inbound: 1
* mozilla-central: 1
* autoland: 1

Platform breakdown:
* windows10-64-devedition: 4
* windows10-64: 4
* windows10-64-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-04-09&endday=2018-04-15&tree=trunk

Comment 5

[Intermittent Failures Robot]

5 failures in 782 pushes (0.006 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 2
* mozilla-central: 2
* autoland: 1

Platform breakdown:
* windows7-32: 2
* windows10-64: 2
* windows10-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-04-16&endday=2018-04-22&tree=trunk

Comment 6

[Intermittent Failures Robot]

17 failures in 780 pushes (0.022 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 8
* try: 4
* mozilla-inbound: 3
* mozilla-central: 2

Platform breakdown:
* windows10-64: 10
* windows7-32: 4
* windows10-64-nightly: 2
* windows10-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-04-23&endday=2018-04-29&tree=trunk

Comment 7

[Intermittent Failures Robot]

28 failures in 551 pushes (0.051 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 8
* autoland: 8
* try: 6
* mozilla-central: 5
* mozilla-beta: 1

Platform breakdown:
* windows10-64: 22
* windows7-32: 2
* windows10-64-nightly: 2
* windows7-32-nightly: 1
* windows10-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-04-30&endday=2018-05-06&tree=trunk

Comment 8

[Intermittent Failures Robot]

22 failures in 660 pushes (0.033 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 8
* autoland: 6
* try: 4
* mozilla-inbound: 4

Platform breakdown:
* windows10-64: 11
* windows10-64-nightly: 4
* osx-10-10: 3
* windows7-32-nightly: 2
* windows10-64-devedition: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-05-07&endday=2018-05-13&tree=trunk

Comment 9

[Intermittent Failures Robot]

34 failures in 639 pushes (0.053 failures/push) were associated with this bug in the last 7 days. 

This is the #38 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 14
* try: 10
* mozilla-inbound: 5
* mozilla-central: 3
* mozilla-beta: 2

Platform breakdown:
* windows10-64: 20
* windows10-64-devedition: 4
* linux64: 3
* windows10-64-nightly: 2
* windows7-32-nightly: 1
* windows7-32: 1
* osx-10-10: 1
* linux64-nightly: 1
* linux32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-05-14&endday=2018-05-20&tree=trunk

Comment 10

[Stefan Hindli [:stefan_hindli]]

Over the last 7 days this bug has 30 failures. These happen on windows10-64, windows7-32, linux64-nightly, linux64.

Here is the most recent log example: https://treeherder.mozilla.org/logviewer.html#?job_id=179543506&repo=autoland&lineNumber=8838

19:40:50    ERROR -  664 INFO TEST-UNEXPECTED-FAIL | devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite1 is "Unset" - Got undefined, expected Unset
19:40:50     INFO -  Stack trace:
19:40:50     INFO -  chrome://mochikit/content/browser-test.js:test_is:1285
19:40:50     INFO -  chrome://mochitests/content/browser/devtools/client/storage/test/browser_storage_cookies_samesite.js:null:32
19:40:50     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest/<:1083
19:40:50     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest:1074
19:40:50     INFO -  chrome://mochikit/content/browser-test.js:nextTest/<:976
19:40:50     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
19:40:50     INFO -  Not taking screenshot here: see the one that was previously logged
19:40:50    ERROR -  665 INFO TEST-UNEXPECTED-FAIL | devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite2 is "Lax" - Got undefined, expected Lax
19:40:50     INFO -  Stack trace:
19:40:50     INFO -  chrome://mochikit/content/browser-test.js:test_is:1285
19:40:50     INFO -  chrome://mochitests/content/browser/devtools/client/storage/test/browser_storage_cookies_samesite.js:null:33
19:40:50     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest/<:1083
19:40:50     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest:1074
19:40:50     INFO -  chrome://mochikit/content/browser-test.js:nextTest/<:976
19:40:50     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
19:40:50     INFO -  Not taking screenshot here: see the one that was previously logged
19:40:50    ERROR -  666 INFO TEST-UNEXPECTED-FAIL | devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite3 is "Strict" - Got undefined, expected Strict

Comment 11

[Robert O'Callahan (:roc) (email my personal email if necessary)]

We captured this bug and made it debuggable in the Pernosco debugger:
https://pernos.co/debug/WeSzQc6c7vmfckZ8CsVHaA/index.html
Admittedly it may be difficult to debug without actual JS support.
Access details here: https://mana.mozilla.org/wiki/display/TS/Debugging+with+Pernosco

Comment 12

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

Attachment: Bug 1448484 - Intermittent devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite1 is Unset - Got undefined, expected Unset

The problem is that we use this:
this.front.on("stores-update", this.onEdit);

But this.onEdit is async so we have race issues here. Luckily, only tests are fast enough to trigger race conditions in this type of instance.

We also have similar issues caused by:
- this.tree.on("select", this.onHostSelect);
- this.table.on(TableWidget.EVENTS.SCROLL_END, this.handleScrollEnd);
- this._refreshButton.addEventListener("command", this.onRefreshTable);

Sadly this is a difficult issue to fix because adding async support to the event emitter would mean moving emit calls inside shared libraries into async method chains... the problem being that maybe we may not want the emitter to be asynchronous.

The easy workaround for the moment is to add a small pause before reading the row values. This makes the most sense, especially considering that this tool needs converting to React / HTML at some point.

Review commit: https://reviewboard.mozilla.org/r/245670/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/245670/

Comment 13

[Patrick Brosset <:pbro>]

Comment on attachment 8979498 [details]
Bug 1448484 - Intermittent devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite1 is Unset - Got undefined, expected Unset

https://reviewboard.mozilla.org/r/245670/#review252532

I don't know the storage inspector and its tests very well so I don't understand what onEdit races against. Can you explain the specifics?
In any case, the way we usually go about this is by emitting an event when some action is done in the tool and having tests wait on this event. Much more bullet-proof than waiting for some delay.
I see store-objects-edit is emitted already in the code, couldn't that be used?

Comment 14

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

(In reply to Patrick Brosset <:pbro> from comment #13)
> Comment on attachment 8979498 [details]
> Bug 1448484 - Intermittent
> devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite1
> is Unset - Got undefined, expected Unset
> 
> https://reviewboard.mozilla.org/r/245670/#review252532
> 
> I don't know the storage inspector and its tests very well so I don't
> understand what onEdit races against. Can you explain the specifics?
> In any case, the way we usually go about this is by emitting an event when
> some action is done in the tool and having tests wait on this event. Much
> more bullet-proof than waiting for some delay.
> I see store-objects-edit is emitted already in the code, couldn't that be
> used?

Yes, we usually have an event to say that all the data is loaded. The problem with doing that in the storage inspector is that it is "live" so the data has never all been received. To work around this we set a 200ms timer every time a storage item is received and when a batch of objects finishes arriving we send the stores-update event.

The problem is that if the try servers are busy then there can easily be gaps of 200ms between receiving the cookies... then how many stores-update events should we listen for?

Still, I would like to try to get to the bottom of this particular case... maybe we need to add a new event to show that the table has been updated or move the store-objects-edit event.

Comment 15

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

Comment on attachment 8979498 [details]
Bug 1448484 - Intermittent devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite1 is Unset - Got undefined, expected Unset

Now I can't even reproduce it, even when running --verify (I tried 10 runs).

Comment 16

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

The screenshots show that the Storage Inspector is reporting "There is no data for the selected host" when this happens.

Comment 17

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

There are a few interesting points of note:
1. Although the screenshot shows an empty storage inspector we report "There is correct number of rows in cookies."
2. The number of cookies is established by counting the items inside the TableWidget.
3. The items inside the TableWidget are set when they are pushed by ui.js::populateTable() and this happens before the "store-objects-updated" event.
4. The label.value property of the cells is being written in all of my testing but it seems that occasionally when we read it we get undefined.

I am beginning to suspect that some kind of silent error could be preventing the label.values being written.

Comment 18

[Intermittent Failures Robot]

64 failures in 632 pushes (0.101 failures/push) were associated with this bug in the last 7 days. 

This is the #29 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* mozilla-inbound: 21
* autoland: 18
* mozilla-central: 10
* try: 9
* mozilla-beta: 6

Platform breakdown:
* windows10-64: 31
* linux64: 11
* windows10-64-nightly: 7
* osx-10-10: 6
* windows7-32: 4
* windows10-64-devedition: 2
* linux32: 2
* windows7-32-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-05-21&endday=2018-05-27&tree=trunk

Comment 19

[Natalia Csoregi [:nataliaCs]]

Update:
 There were 44 failures in the last 7 days:
 - 4 failures on linux 32 opt
 - 7 failures on linux 64 opt/pgo
 - 4 failures on osx 10.10
 - 6 failures on windows 7x32 debug/pgo
 - 23 failures on windows 10x64 pgo
Example failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=180908606&repo=mozilla-inbound&lineNumber=11749 

13:45:10    ERROR -  8585 INFO TEST-UNEXPECTED-FAIL | devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite1 is "Unset" - Got undefined, expected Unset
13:45:10     INFO -  Stack trace:
13:45:10     INFO -  chrome://mochikit/content/browser-test.js:test_is:1285
13:45:10     INFO -  chrome://mochitests/content/browser/devtools/client/storage/test/browser_storage_cookies_samesite.js:null:32
13:45:10     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest/<:1083
13:45:10     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest:1074
13:45:10     INFO -  chrome://mochikit/content/browser-test.js:nextTest/<:976
13:45:10     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
13:45:10     INFO -  Not taking screenshot here: see the one that was previously logged

Comment 20

[Intermittent Failures Robot]

53 failures in 655 pushes (0.081 failures/push) were associated with this bug in the last 7 days. 

This is the #24 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 25
* mozilla-inbound: 15
* try: 7
* mozilla-beta: 4
* mozilla-central: 2

Platform breakdown:
* windows10-64: 29
* linux64: 6
* windows7-32: 5
* osx-10-10: 4
* windows10-64-nightly: 3
* windows10-64-devedition: 3
* linux32: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-05-28&endday=2018-06-03&tree=trunk

Comment 21

[Intermittent Failures Robot]

7 failures in 99 pushes (0.071 failures/push) were associated with this bug yesterday.    

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. ** 

Repository breakdown:
* try: 5
* mozilla-inbound: 1
* mozilla-central: 1

Platform breakdown:
* windows10-64: 3
* windows7-32: 1
* windows10-64-nightly: 1
* windows10-64-devedition: 1
* linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-06-04&endday=2018-06-04&tree=trunk

Comment 22

[Eliza Balazs [:ebalazs_]]

Attachment: Skipped browser_storage_cookies_samesite.js for frequent failures on win 10 !debug

Comment 23

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8983331 [details] [diff] [review]
Skipped browser_storage_cookies_samesite.js for frequent failures on win 10 !debug

Review of attachment 8983331 [details] [diff] [review]:
-----------------------------------------------------------------

looks great

Comment 24

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e498ba8d51d8
disable browser_storage_cookies_samesite.js for frequent failures on win 10 !debug. r=jmaher

Comment 25

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/e498ba8d51d8

Comment 26

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

Because I cannot reproduce this locally and looking at the source I don't see how this could happen I think the best thing we can do is add a lot of logging and re-enable the test... we need to find out where this is happening.

Comment 27

[Intermittent Failures Robot]

25 failures in 669 pushes (0.037 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 7
* mozilla-inbound: 7
* mozilla-beta: 5
* autoland: 4
* mozilla-central: 2

Platform breakdown:
* windows10-64: 5
* windows7-32: 4
* windows10-64-devedition: 4
* linux32: 4
* windows10-64-nightly: 3
* linux64: 3
* osx-10-10: 1
* macosx64-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-06-04&endday=2018-06-10&tree=trunk

Comment 28

[Intermittent Failures Robot]

9 failures in 408 pushes (0.022 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 4
* mozilla-inbound: 3
* mozilla-beta: 2

Platform breakdown:
* linux32: 3
* osx-10-10: 2
* windows7-32: 1
* windows10-64-nightly: 1
* windows10-64-devedition: 1
* linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-06-11&endday=2018-06-17&tree=trunk

Comment 29

[Intermittent Failures Robot]

24 failures in 521 pushes (0.046 failures/push) were associated with this bug in the last 7 days. 

This is the #35 most frequent failure this week.   

Repository breakdown:
* mozilla-inbound: 6
* mozilla-beta: 6
* mozilla-central: 4
* try: 3
* autoland: 3
* mozilla-release: 2

Platform breakdown:
* linux64: 6
* windows10-64-nightly: 4
* osx-10-10: 4
* windows7-32: 2
* macosx64-nightly: 2
* macosx64-devedition: 2
* linux32: 2
* windows7-32-devedition: 1
* windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-06-18&endday=2018-06-24&tree=trunk

Comment 30

[Intermittent Failures Robot]

16 failures in 718 pushes (0.022 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 7
* autoland: 5
* try: 2
* mozilla-central: 2

Platform breakdown:
* windows7-32: 8
* linux64: 3
* linux32: 3
* osx-10-10: 1
* macosx64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-06-25&endday=2018-07-01&tree=trunk

Comment 31

[Intermittent Failures Robot]

13 failures in 622 pushes (0.021 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 8
* mozilla-beta: 2
* try: 1
* mozilla-release: 1
* mozilla-inbound: 1

Platform breakdown:
* windows7-32: 4
* osx-10-10: 3
* linux64: 2
* windows10-64-nightly: 1
* linux64-devedition: 1
* linux32-devedition: 1
* linux32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-07-02&endday=2018-07-08&tree=trunk

Comment 32

[Intermittent Failures Robot]

24 failures in 623 pushes (0.039 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 8
* autoland: 6
* try: 4
* mozilla-inbound: 3
* mozilla-central: 3

Platform breakdown:
* windows7-32: 6
* osx-10-10: 6
* windows7-32-nightly: 4
* windows10-64: 3
* linux64: 3
* linux32-devedition: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-07-09&endday=2018-07-15&tree=trunk

Comment 33

[Intermittent Failures Robot]

31 failures in 653 pushes (0.047 failures/push) were associated with this bug in the last 7 days. 

This is the #35 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* mozilla-inbound: 13
* mozilla-central: 8
* try: 5
* autoland: 5

Platform breakdown:
* windows7-32: 18
* osx-10-10: 4
* linux64: 2
* windows7-32-nightly: 1
* windows7-32-msvc: 1
* windows10-64: 1
* macosx64-nightly: 1
* macosx64-devedition: 1
* linux64-devedition: 1
* linux32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-07-16&endday=2018-07-22&tree=trunk

Comment 34

[Bogdan Tara[:bogdan_tara | bogdant]]

In the last 7 days there have been 30 occurrences, most (19) on Windows 7 32 Opt and pgo. Also occurs on Linux 32& 64 and Os X opt and pgo. 


Recent failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=190169279&repo=mozilla-inbound&lineNumber=3013

03:33:06     INFO -  638 INFO TEST-START | devtools/client/storage/test/browser_storage_cookies_samesite.js
03:33:06     INFO -  TEST-INFO | started process screenshot
03:33:07     INFO -  TEST-INFO | screenshot: exit 0
03:33:07     INFO -  Buffered messages logged at 03:33:06
03:33:07     INFO -  639 INFO Entering test bound
03:33:07     INFO -  640 INFO Adding a new tab with URL: http://test1.example.org/browser/devtools/client/storage/test/storage-cookies-samesite.html
03:33:07     INFO -  641 INFO Tab added and finished loading
03:33:07     INFO -  642 INFO Found a window: complete
03:33:07     INFO -  643 INFO Opening the storage inspector
03:33:07     INFO -  644 INFO Opening the toolbox
03:33:07     INFO -  645 INFO Console message: [JavaScript Warning: "XUL box for vbox element contained an inline label child, forcing all its children to be wrapped in a block." {file: "chrome://devtools/content/storage/index.xul" line: 0}]
03:33:07     INFO -  646 INFO Waiting for the stores to update
03:33:07     INFO -  647 INFO Making sure that the toolbox's frame is focused
03:33:07     INFO -  648 INFO Selecting tree item cookies > http://test1.example.org
03:33:07     INFO -  649 INFO "cookies,http://test1.example.org" is already selected, returning.
03:33:07     INFO -  650 INFO TEST-PASS | devtools/client/storage/test/browser_storage_cookies_samesite.js | There is correct number of rows in cookies > http://test1.example.org -
03:33:07     INFO -  651 INFO TEST-PASS | devtools/client/storage/test/browser_storage_cookies_samesite.js | There is item with name 'test1{9d414cc5-8319-0a04-0586-c0a6ae01670a}test1.example.org{9d414cc5-8319-0a04-0586-c0a6ae01670a}/browser/devtools/client/storage/test/' in cookies > http://test1.example.org -
03:33:07     INFO -  652 INFO TEST-PASS | devtools/client/storage/test/browser_storage_cookies_samesite.js | There is item with name 'test2{9d414cc5-8319-0a04-0586-c0a6ae01670a}test1.example.org{9d414cc5-8319-0a04-0586-c0a6ae01670a}/browser/devtools/client/storage/test/' in cookies > http://test1.example.org -
03:33:07     INFO -  653 INFO TEST-PASS | devtools/client/storage/test/browser_storage_cookies_samesite.js | There is item with name 'test3{9d414cc5-8319-0a04-0586-c0a6ae01670a}test1.example.org{9d414cc5-8319-0a04-0586-c0a6ae01670a}/browser/devtools/client/storage/test/' in cookies > http://test1.example.org -
03:33:07     INFO -  Buffered messages finished
03:33:07    ERROR -  654 INFO TEST-UNEXPECTED-FAIL | devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite1 is "Unset" - Got undefined, expected Unset
03:33:07     INFO -  Stack trace:
03:33:07     INFO -  chrome://mochikit/content/browser-test.js:test_is:1305
03:33:07     INFO -  chrome://mochitests/content/browser/devtools/client/storage/test/browser_storage_cookies_samesite.js:null:32
03:33:07     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest/<:1103
03:33:07     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest:1094
03:33:07     INFO -  chrome://mochikit/content/browser-test.js:nextTest/<:996
03:33:07     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
03:33:07     INFO -  Not taking screenshot here: see the one that was previously logged
03:33:07    ERROR -  655 INFO TEST-UNEXPECTED-FAIL | devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite2 is "Lax" - Got undefined, expected Lax
03:33:07     INFO -  Stack trace:
03:33:07     INFO -  chrome://mochikit/content/browser-test.js:test_is:1305
03:33:07     INFO -  chrome://mochitests/content/browser/devtools/client/storage/test/browser_storage_cookies_samesite.js:null:33
03:33:07     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest/<:1103
03:33:07     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest:1094
03:33:07     INFO -  chrome://mochikit/content/browser-test.js:nextTest/<:996
03:33:07     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
03:33:07     INFO -  Not taking screenshot here: see the one that was previously logged
03:33:07    ERROR -  656 INFO TEST-UNEXPECTED-FAIL | devtools/client/storage/test/browser_storage_cookies_samesite.js | sameSite3 is "Strict" - Got undefined, expected Strict
03:33:07     INFO -  Stack trace:
03:33:07     INFO -  chrome://mochikit/content/browser-test.js:test_is:1305
03:33:07     INFO -  chrome://mochitests/content/browser/devtools/client/storage/test/browser_storage_cookies_samesite.js:null:34
03:33:07     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest/<:1103
03:33:07     INFO -  chrome://mochikit/content/browser-test.js:Tester_execTest:1094
03:33:07     INFO -  chrome://mochikit/content/browser-test.js:nextTest/<:996
03:33:07     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
03:33:07     INFO -  657 INFO Removing tab.
03:33:07     INFO -  658 INFO Waiting for event: 'TabClose' on [object XULElement].
03:33:07     INFO -  659 INFO Got event: 'TabClose' on [object XULElement].
03:33:07     INFO -  660 INFO Tab removed and finished closing
03:33:07     INFO -  661 INFO Leaving test bound
03:33:07     INFO -  GECKO(3848) | MEMORY STAT | vsize 702MB | vsizeMaxContiguous 653MB | residentFast 210MB | heapAllocated 89MB
03:33:07     INFO -  662 INFO TEST-OK | devtools/client/storage/test/browser_storage_cookies_samesite.js | took 653ms
03:33:07     INFO -  663 INFO checking window state
03:33:07     INFO -  664 INFO TEST-START | devtools/client/storage/test/browser_storage_cookies_tab_navigation.js
03:33:08     INFO -  GECKO(3848) | MEMORY STAT | vsize 711MB | vsizeMaxContiguous 653MB | residentFast 212MB | heapAllocated 89MB
03:33:08     INFO -  665 INFO TEST-OK | devtools/client/storage/test/browser_storage_cookies_tab_navigation.js | took 889ms

Comment 35

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

I thought we had disabled this test. I will do that as soon as I get chance.

Comment 36

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

Attachment: Bug 1448484 - Disable browser_storage_cookies_samesite.js due to intermittent failures

Review commit: https://reviewboard.mozilla.org/r/259902/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/259902/

Comment 37

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

Comment on attachment 8995467 [details]
Bug 1448484 - Disable browser_storage_cookies_samesite.js due to intermittent failures

https://reviewboard.mozilla.org/r/259902/#review266942

Comment 38

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

Disabled test.

Comment 39

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b9c3e9ffe061
Disable browser_storage_cookies_samesite.js due to intermittent failures r=miker

Comment 40

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/b9c3e9ffe061

Comment 41

[Intermittent Failures Robot]

27 failures in 760 pushes (0.036 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 13
* mozilla-inbound: 8
* try: 3
* mozilla-central: 2
* mozilla-beta: 1

Platform breakdown:
* windows7-32: 15
* osx-10-10: 4
* windows7-32-nightly: 2
* windows10-64: 2
* linux64: 2
* windows7-32-msvc: 1
* macosx64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-07-23&endday=2018-07-29&tree=trunk

Comment 42

[Intermittent Failures Robot]

5 failures in 2579 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-release: 2
* mozilla-beta: 3

Platform breakdown:
* windows10-64-nightly: 1
* windows7-32-nightly: 1
* windows10-64: 1
* linux32-devedition: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-08-06&endday=2018-08-12&tree=all

Comment 43

[Intermittent Failures Robot]

2 failures in 2672 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-beta: 2

Platform breakdown:
* linux32-devedition: 1
* windows7-32-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-08-13&endday=2018-08-19&tree=all

Comment 44

[Intermittent Failures Robot]

1 failures in 2902 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-release: 1

Platform breakdown:
* macosx64-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1448484&startday=2018-09-17&endday=2018-09-23&tree=all

Comment 45

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.