1338848 - Intermittent browser/base/content/test/newtab/browser_newtab_bug725996.js | Test timed out -

Status: RESOLVED FIXED

(Firefox :: New Tab Page, defect)

Description

[Treeherder Bug Filer]

Filed by: philringnalda [at] gmail.com

https://treeherder.mozilla.org/logviewer.html#?job_id=76619225&repo=mozilla-inbound

https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-inbound-win32-debug/1486852059/mozilla-inbound_win7_vm-debug_test-mochitest-e10s-browser-chrome-2-bm140-tests1-windows-build264.txt.gz

Comment 1

[Intermittent Failures Robot]

8 failures in 833 pushes (0.01 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* mozilla-inbound: 5
* autoland: 2
* try: 1

Platform breakdown:
* windows8-64: 3
* osx-10-10: 3
* windows7-32-vm: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-02-13&endday=2017-02-19&tree=all

Comment 2

[Intermittent Failures Robot]

19 failures in 812 pushes (0.023 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* mozilla-inbound: 15
* autoland: 3
* mozilla-central: 1

Platform breakdown:
* windows8-64: 9
* osx-10-10: 8
* windows7-32-vm: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-02-20&endday=2017-02-26&tree=all

Comment 3

[Intermittent Failures Robot]

18 failures in 783 pushes (0.023 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* autoland: 10
* mozilla-inbound: 6
* try: 1
* oak: 1

Platform breakdown:
* windows8-64: 13
* osx-10-10: 3
* windows7-32-vm: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-02-27&endday=2017-03-05&tree=all

Comment 4

[Intermittent Failures Robot]

36 failures in 790 pushes (0.046 failures/push) were associated with this bug in the last 7 days.   

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* mozilla-inbound: 17
* autoland: 11
* mozilla-central: 2
* mozilla-aurora: 2
* graphics: 2
* try: 1
* oak: 1

Platform breakdown:
* windows8-64: 21
* osx-10-10: 13
* windows7-32-vm: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-03-06&endday=2017-03-12&tree=all

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

this crossed our threshold and doubled in frequency in the last week. this is e10s only and debug only.  I see it heavy on windows8 and osx.  

looking on windows8, I see a recent failure log:
https://archive.mozilla.org/pub/firefox/tinderbox-builds/graphics-win64-debug/1489376856/graphics_win8_64-debug_test-mochitest-e10s-browser-chrome-7-bm127-tests1-windows-build5.txt.gz

which maps to this screenshot:
http://mozilla-releng-blobs.s3.amazonaws.com/blobs/graphics/sha512/0876c04524564e45cec284656a8b1c1e6df8a8256d4f24f4e38c3442e9fe3bc2e815247efb698e1ca3ad3e7dfa0f9c341129cca7dde2e5f429fa9fdf2000a8c9

and has this text in the log file:
21:53:57     INFO - TEST-START | browser/base/content/test/newtab/browser_newtab_bug725996.js
21:53:58     INFO - --DOMWINDOW == 3 (000000A3F868F000) [pid = 3532] [serial = 29] [outer = 0000000000000000] [url = about:blank]
21:53:59     INFO - ++DOCSHELL 0000007CF183C800 == 24 [pid = 3672] [id = {291abf0b-1a00-4896-ad75-f9f33767f4ec}]
21:53:59     INFO - ++DOMWINDOW == 61 (0000007CF183E800) [pid = 3672] [serial = 121] [outer = 0000000000000000]
21:53:59     INFO - [Parent 3672] WARNING: NS_ENSURE_TRUE(standardURL) failed: file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/caps/nsPrincipal.cpp, line 229
21:53:59     INFO - ++DOMWINDOW == 62 (0000007CF33E3000) [pid = 3672] [serial = 122] [outer = 0000007CF183E800]
21:53:59     INFO - [Parent 3672] WARNING: NS_ENSURE_TRUE(standardURL) failed: file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/caps/nsPrincipal.cpp, line 229
21:53:59     INFO - [Parent 3672] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80004005: file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/extensions/cookie/nsPermissionManager.cpp, line 107
21:53:59     INFO - [Parent 3672] WARNING: 'NS_FAILED(aResult)', file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/extensions/cookie/nsPermissionManager.cpp, line 614
21:53:59     INFO - [Parent 3672] WARNING: NS_ENSURE_TRUE(standardURL) failed: file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/caps/nsPrincipal.cpp, line 229
21:53:59     INFO - [Parent 3672] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80004005: file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/extensions/cookie/nsPermissionManager.cpp, line 107
21:53:59     INFO - [Parent 3672] WARNING: 'NS_FAILED(aResult)', file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/extensions/cookie/nsPermissionManager.cpp, line 614
21:53:59     INFO - [Parent 3672] WARNING: NS_ENSURE_TRUE(standardURL) failed: file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/caps/nsPrincipal.cpp, line 229
21:53:59     INFO - [Parent 3672] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80004005: file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/extensions/cookie/nsPermissionManager.cpp, line 107
21:53:59     INFO - [Parent 3672] WARNING: 'NS_FAILED(aResult)', file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/extensions/cookie/nsPermissionManager.cpp, line 614
21:53:59     INFO - ++DOMWINDOW == 63 (0000007CF5B05000) [pid = 3672] [serial = 123] [outer = 0000007CF183E800]
21:53:59     INFO - [Parent 3672] WARNING: 'NS_FAILED(rv)', file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/dom/xhr/XMLHttpRequestMainThread.cpp, line 2000
21:53:59     INFO - [Parent 3672] WARNING: 'NS_FAILED(rv)', file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/dom/xhr/XMLHttpRequestMainThread.cpp, line 2000
21:54:00     INFO - --DOCSHELL 000000DD79EA5000 == 0 [pid = 4256] [id = {4ff70426-085c-4fbf-8595-433e9587000a}]
21:54:00     INFO - --DOMWINDOW == 3 (000000DD7A9B4800) [pid = 4256] [serial = 8] [outer = 0000000000000000] [url = about:blank]
21:54:00     INFO - --DOMWINDOW == 2 (000000DD7A9C1800) [pid = 4256] [serial = 17] [outer = 0000000000000000] [url = about:blank]
21:54:00     INFO - --DOMWINDOW == 62 (0000007C875DB800) [pid = 3672] [serial = 100] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - --DOMWINDOW == 61 (0000007C875AC800) [pid = 3672] [serial = 103] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - --DOMWINDOW == 60 (0000007C82F40000) [pid = 3672] [serial = 91] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - --DOMWINDOW == 59 (0000007CFAEA5800) [pid = 3672] [serial = 88] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - --DOMWINDOW == 58 (0000007C87595800) [pid = 3672] [serial = 97] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - --DOMWINDOW == 57 (0000007CFB9C0000) [pid = 3672] [serial = 94] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - --DOMWINDOW == 56 (0000007CEF5AD800) [pid = 3672] [serial = 64] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - --DOMWINDOW == 55 (0000007CF8869000) [pid = 3672] [serial = 73] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - --DOMWINDOW == 54 (0000007CF887B000) [pid = 3672] [serial = 82] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - --DOMWINDOW == 53 (0000007C82F42800) [pid = 3672] [serial = 79] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - --DOMWINDOW == 52 (0000007CF333E000) [pid = 3672] [serial = 67] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - --DOMWINDOW == 51 (0000007CF5EBD800) [pid = 3672] [serial = 70] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - --DOMWINDOW == 50 (0000007CF9D45800) [pid = 3672] [serial = 85] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - --DOMWINDOW == 49 (0000007C80988000) [pid = 3672] [serial = 76] [outer = 0000000000000000] [url = about:newtab]
21:54:00     INFO - [Parent 3672] WARNING: 'NS_FAILED(rv)', file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/dom/xhr/XMLHttpRequestMainThread.cpp, line 2000
21:54:02     INFO - --DOMWINDOW == 1 (000000DD7A31B800) [pid = 4256] [serial = 18] [outer = 0000000000000000] [url = about:blank]
21:54:02     INFO - --DOMWINDOW == 2 (000000A3F91C3800) [pid = 3532] [serial = 30] [outer = 0000000000000000] [url = about:blank]
21:54:05     INFO - --DOCSHELL 0000007C8363F800 == 23 [pid = 3672] [id = {1f61949f-8b8e-4ec5-bb80-0a15c4a84920}]
21:54:05     INFO - --DOCSHELL 0000007C87595000 == 22 [pid = 3672] [id = {0343aa0a-e0ba-4c2b-9355-ab70cda666ad}]
21:54:05     INFO - --DOCSHELL 0000007C80698800 == 21 [pid = 3672] [id = {7274b9bd-1535-4154-8034-1b7463f5f2df}]
21:54:05     INFO - --DOCSHELL 0000007CF690A000 == 20 [pid = 3672] [id = {6261ffb6-48c3-49b4-a69a-c3091a9a9108}]
21:54:05     INFO - --DOCSHELL 0000007C875AC000 == 19 [pid = 3672] [id = {0ed7cafd-94ee-4401-bcdc-613f000c5054}]
21:54:05     INFO - --DOCSHELL 0000007C80986000 == 18 [pid = 3672] [id = {fe2f4d61-529d-4f77-aa3a-d93a19e856fb}]
21:54:05     INFO - --DOCSHELL 0000007CFAE8C000 == 17 [pid = 3672] [id = {8c494e44-5b5c-409b-b0b1-afbdd5ae1cbe}]
21:54:05     INFO - --DOCSHELL 0000007CF28A9000 == 16 [pid = 3672] [id = {fecbd324-8a60-4a32-8d6a-4d530b2cea14}]
21:54:05     INFO - --DOCSHELL 0000007CF5EC1000 == 15 [pid = 3672] [id = {bc925c79-40dc-4bd7-be79-e2463bf5454d}]
21:54:05     INFO - --DOCSHELL 0000007CF8875800 == 14 [pid = 3672] [id = {89f8f2cd-df8a-420e-a4f0-42ffb055fd42}]
21:54:05     INFO - --DOCSHELL 0000007C82789000 == 13 [pid = 3672] [id = {d8d02b6b-f6bd-425b-b1a6-236677fc7c89}]
21:54:05     INFO - --DOCSHELL 0000007C82F40800 == 12 [pid = 3672] [id = {6e280d2e-167b-4025-8878-595b0185cc47}]
21:54:05     INFO - --DOCSHELL 0000007CF5E15000 == 11 [pid = 3672] [id = {f997129f-f8b2-4109-93be-e530f778df2d}]
21:54:05     INFO - --DOCSHELL 0000007CF4F60800 == 10 [pid = 3672] [id = {c565c83a-857d-467d-8482-0201b2592ff7}]
21:54:05     INFO - --DOCSHELL 0000007C88432800 == 9 [pid = 3672] [id = {2f7b006e-22db-4bb5-953f-92931c60031c}]
21:54:05     INFO - --DOCSHELL 0000007CF0BCB800 == 8 [pid = 3672] [id = {dbb7f1e8-bbf2-4cf9-979c-f75172b8a887}]
21:54:05     INFO - --DOMWINDOW == 48 (0000007C82D91800) [pid = 3672] [serial = 75] [outer = 0000000000000000] [url = about:newtab]
21:54:05     INFO - --DOMWINDOW == 47 (0000007CF8BC8000) [pid = 3672] [serial = 84] [outer = 0000000000000000] [url = about:newtab]
21:54:05     INFO - --DOMWINDOW == 46 (0000007C84631000) [pid = 3672] [serial = 81] [outer = 0000000000000000] [url = about:newtab]
21:54:05     INFO - --DOMWINDOW == 45 (0000007CF5214800) [pid = 3672] [serial = 69] [outer = 0000000000000000] [url = about:newtab]
21:54:05     INFO - --DOMWINDOW == 44 (0000007CF72B1800) [pid = 3672] [serial = 72] [outer = 0000000000000000] [url = about:newtab]
21:54:05     INFO - --DOMWINDOW == 43 (0000007CFBB69000) [pid = 3672] [serial = 87] [outer = 0000000000000000] [url = about:newtab]
21:54:05     INFO - --DOMWINDOW == 42 (0000007CF0E70800) [pid = 3672] [serial = 78] [outer = 0000000000000000] [url = about:newtab]
21:54:05     INFO - --DOMWINDOW == 41 (0000007CF185E000) [pid = 3672] [serial = 107] [outer = 0000000000000000] [url = about:blank]
21:54:05     INFO - --DOMWINDOW == 40 (0000007CFA660000) [pid = 3672] [serial = 101] [outer = 0000000000000000] [url = about:blank]
21:54:05     INFO - --DOMWINDOW == 39 (0000007C875DA800) [pid = 3672] [serial = 104] [outer = 0000000000000000] [url = about:blank]
21:54:05     INFO - --DOMWINDOW == 38 (0000007CF059A800) [pid = 3672] [serial = 92] [outer = 0000000000000000] [url = about:blank]
21:54:05     INFO - --DOMWINDOW == 37 (0000007C8759B800) [pid = 3672] [serial = 98] [outer = 0000000000000000] [url = about:blank]
21:54:05     INFO - --DOMWINDOW == 36 (0000007CFBDA8800) [pid = 3672] [serial = 95] [outer = 0000000000000000] [url = about:blank]
21:54:05     INFO - --DOMWINDOW == 35 (0000007C875A5000) [pid = 3672] [serial = 99] [outer = 0000000000000000] [url = about:newtab]
21:54:05     INFO - --DOMWINDOW == 34 (0000007CF0E7F000) [pid = 3672] [serial = 93] [outer = 0000000000000000] [url = about:newtab]
21:54:05     INFO - --DOMWINDOW == 33 (0000007CFC8C2800) [pid = 3672] [serial = 90] [outer = 0000000000000000] [url = about:newtab]
21:54:05     INFO - --DOMWINDOW == 32 (0000007CFC2E4800) [pid = 3672] [serial = 96] [outer = 0000000000000000] [url = about:newtab]
21:54:05     INFO - --DOMWINDOW == 31 (0000007C875EB800) [pid = 3672] [serial = 105] [outer = 0000000000000000] [url = about:newtab]
21:54:05     INFO - --DOMWINDOW == 30 (0000007C831CF000) [pid = 3672] [serial = 66] [outer = 0000000000000000] [url = about:newtab]
21:54:05     INFO - --DOMWINDOW == 29 (0000007C87594000) [pid = 3672] [serial = 102] [outer = 0000000000000000] [url = about:newtab]
21:54:06     INFO - --DOMWINDOW == 0 (000000DD7ABB3000) [pid = 4256] [serial = 19] [outer = 0000000000000000] [url = about:blank]
21:54:08     INFO - --DOMWINDOW == 28 (0000007CF5206800) [pid = 3672] [serial = 112] [outer = 0000000000000000] [url = about:newtab]
21:54:08     INFO - --DOMWINDOW == 27 (0000007C88439000) [pid = 3672] [serial = 109] [outer = 0000000000000000] [url = about:newtab]
21:54:08     INFO - --DOMWINDOW == 26 (0000007CF5EAC800) [pid = 3672] [serial = 115] [outer = 0000000000000000] [url = about:newtab]
21:54:08     INFO - --DOMWINDOW == 25 (0000007CF1846000) [pid = 3672] [serial = 106] [outer = 0000000000000000] [url = about:newtab]
21:54:11     INFO - --DOMWINDOW == 24 (0000007CF651E000) [pid = 3672] [serial = 116] [outer = 0000000000000000] [url = about:blank]
21:54:11     INFO - --DOMWINDOW == 23 (0000007C875AA000) [pid = 3672] [serial = 119] [outer = 0000000000000000] [url = about:blank]
21:54:11     INFO - --DOMWINDOW == 22 (0000007CF690A800) [pid = 3672] [serial = 117] [outer = 0000000000000000] [url = about:newtab]
21:54:11     INFO - --DOMWINDOW == 21 (0000007CF5B0C000) [pid = 3672] [serial = 114] [outer = 0000000000000000] [url = about:newtab]
21:54:11     INFO - --DOMWINDOW == 20 (0000007CF0535800) [pid = 3672] [serial = 111] [outer = 0000000000000000] [url = about:newtab]
21:54:11     INFO - --DOMWINDOW == 19 (0000007CF33E3000) [pid = 3672] [serial = 122] [outer = 0000000000000000] [url = about:blank]
21:54:11     INFO - --DOMWINDOW == 18 (0000007CF521C000) [pid = 3672] [serial = 113] [outer = 0000000000000000] [url = about:blank]
21:54:11     INFO - --DOMWINDOW == 17 (0000007CF1E22000) [pid = 3672] [serial = 108] [outer = 0000000000000000] [url = about:newtab]
21:54:11     INFO - --DOMWINDOW == 16 (0000007C88442800) [pid = 3672] [serial = 110] [outer = 0000000000000000] [url = about:blank]
21:54:31     INFO - [Parent 3672] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80520012: file c:/builds/moz2_slave/gfx-w64-d-00000000000000000000/build/src/widget/windows/WinUtils.cpp, line 1468
21:55:27     INFO - TEST-INFO | started process screenshot
21:55:27     INFO - TEST-INFO | screenshot: exit 0
21:55:27     INFO - Buffered messages logged at 21:53:57
21:55:27     INFO - innerHeight,,884
21:55:27     INFO - innerWidth,,1030
21:55:27     INFO - Entering test bound setup
21:55:27     INFO - Leaving test bound setup
21:55:27     INFO - Entering test bound 
21:55:27     INFO - Buffered messages logged at 21:53:59
21:55:27     INFO - TEST-PASS | browser/base/content/test/newtab/browser_newtab_bug725996.js | grid status = 0,1,2,3,4,5,6,7,8 - ["0","1","2","3","4","5","6","7","8"] == "0,1,2,3,4,5,6,7,8" - 
21:55:27     INFO - Buffered messages logged at 21:54:00
21:55:27     INFO - TEST-PASS | browser/base/content/test/newtab/browser_newtab_bug725996.js | first cell is pinned and contains the dropped site - 
21:55:27     INFO - Buffered messages finished
21:55:27     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/newtab/browser_newtab_bug725996.js | Test timed out - 
21:55:27     INFO - *************************
21:55:27     INFO - A coding exception was thrown and uncaught in a Task.
21:55:27     INFO - Full message: TypeError: grid is undefined
21:55:27     INFO - Full stack: @resource://testing-common/content-task.js line 52 > eval:6:9
21:55:27     INFO - TaskImpl_run@resource://gre/modules/Task.jsm:319:42
21:55:27     INFO - TaskImpl@resource://gre/modules/Task.jsm:277:3
21:55:27     INFO - asyncFunction@resource://gre/modules/Task.jsm:252:14
21:55:27     INFO - Task_spawn@resource://gre/modules/Task.jsm:166:12
21:55:27     INFO - @resource://testing-common/content-task.js:54:5
21:55:27     INFO - *************************
21:55:27     INFO - Not taking screenshot here: see the one that was previously logged
21:55:27     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/newtab/browser_newtab_bug725996.js | Uncaught exception - TypeError: grid is undefined
21:55:27     INFO - Leaving test bound 
21:55:28     INFO - MEMORY STAT | vsize 1783MB | vsizeMaxContiguous 4258025MB | residentFast 232MB | heapAllocated 78MB
21:55:28     INFO - TEST-OK | browser/base/content/test/newtab/browser_newtab_bug725996.js | took 90663ms


the push prior to this on win8 debug e10s yields a 2.5s runtime:
15:38:02     INFO - TEST-OK | browser/base/content/test/newtab/browser_newtab_bug725996.js | took 2579ms

so extending the timeout won't help out.  


I assume we are failing to get a value for grid in head.js:
https://dxr.mozilla.org/mozilla-central/source/browser/base/content/test/newtab/head.js#343


:ttaubert, I see you are the original author of this test, is this something you could look at (or help me find someone to look at this)- I will keep an eye on the frequency in case it starts to get better and this becomes less of a priority.  Maybe this is a quick fix :)

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

as a note, this is not coalesced by SETA, so the failure rate would be fairly accurate (# failures/ # pushes)

Comment 7

[Tim Taubert [:ttaubert] (inactive)]

Sorry, I don't work on this stuff anymore. And I have no idea who does these days...

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

:mconley, do you know  who works on the  new tab page?

Comment 9

[Mike Conley (:mconley) (:⚙️)]

(In reply to Joel Maher ( :jmaher) from comment #8)
> :mconley, do you know  who works on the  new tab page?

I believe nobody really does, as most effort is being put into its replacement, Activity Stream.

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

nice, lets see if the activity stream people want to own these tests or if we should disable them!

Peter, we have older new tab page tests, is this something you would be interested in continuing to see running?  If so, could you recommend someone to own these tests?

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

there are 35 tests for firefox new tab:
https://dxr.mozilla.org/mozilla-central/source/browser/base/content/test/newtab

Comment 12

[Intermittent Failures Robot]

8 failures in 777 pushes (0.01 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-aurora: 3
* autoland: 3
* try: 1
* graphics: 1

Platform breakdown:
* windows8-64: 5
* osx-10-10: 2
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-03-13&endday=2017-03-19&tree=all

Comment 13

[:Gijs (he/him)]

Dão, Drew and I have all done stuff to the new tab page in recent history (last few months), though that doesn't necessarily mean we know what's up here. Either way, these tests shouldn't be removed.

comment #5 seems to indicate there's a discrete cut-off point when the test started to be a lot slower. Is there a regression range for that change? I didn't spot one in comment #5...

I'll also point out that comment #12 seems to suggest this is now well below the threshold again...

Comment 14

[Joel Maher ( :jmaher ) (UTC -8)]

yes, this seems to have fixed itself, I wasn't able to find a specific range of why this got worse, it wasn't a clear marking on the orangefactor data.

I am happy to see these tests running, as long as when we have a problem there is someone to help sort things out.  In the last ~5 months this is the only newtab/ test that has been intermittent >30 times/week, so that is good :)

Comment 15

[Intermittent Failures Robot]

12 failures in 894 pushes (0.013 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 7
* mozilla-inbound: 2
* mozilla-central: 2
* oak: 1

Platform breakdown:
* osx-10-10: 7
* windows8-64: 4
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-04-10&endday=2017-04-16&tree=all

Comment 16

[Intermittent Failures Robot]

17 failures in 817 pushes (0.021 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 8
* mozilla-inbound: 5
* graphics: 2
* try: 1
* mozilla-central: 1

Platform breakdown:
* windows8-64: 7
* windows7-32-vm: 7
* osx-10-10: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-04-17&endday=2017-04-23&tree=all

Comment 17

[Intermittent Failures Robot]

42 failures in 883 pushes (0.048 failures/push) were associated with this bug in the last 7 days. 

This is the #23 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 17
* mozilla-inbound: 11
* mozilla-central: 7
* try: 3
* graphics: 3
* oak: 1

Platform breakdown:
* windows7-32-vm: 22
* osx-10-10: 12
* windows8-64: 8

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-04-24&endday=2017-04-30&tree=all

Comment 18

[:Gijs (he/him)]

Do we know when/why this went up so sharply again?

Comment 19

[Joel Maher ( :jmaher ) (UTC -8)]

thanks for bringing this to my attention, I will see if something looks like a culprit, overall, I suspect no luck as we were failing.

I did a lot of retriggers around the 24th when this seemed to be more sustained:
https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-searchStr=win%20vm%20debug%20e10s%20bc1&tochange=efa1ee67e2f18bf17d73af8a45b56f17910f0cf7&fromchange=9c7bde46a67a1975e2e3eb2ecbac00784b783e9e&selectedJob=96542744

many of the 'newtab' tests are failing, so possibly there is some underlying issue?

what is more concerning is that we are running this test infrequently (not every push by default), so the failure rate is higher than documented.

Comment 20

[Joel Maher ( :jmaher ) (UTC -8)]

related bugs would be:
bug 1244713 - browser_newtab_reportLinkAction.js (no real recent history, but retriggers show the failure)
bug 1338836 - browser_newtab_bug765628.js (win7/win8/osx debug failures - ~20/week)
bug 1260958 - browser_newtab_unpin.js (win7/debug ~5 failures in the last week)

a lot of common failures on win7/debug.  

:gijs, do you have ideas on what might be a common failure between this bug and the above mentioned 3 bugs.

Comment 21

[:Gijs (he/him)]

(In reply to Joel Maher ( :jmaher) from comment #19)
> thanks for bringing this to my attention, I will see if something looks like
> a culprit, overall, I suspect no luck as we were failing.

I don't really understand this sentence. Why can we/you not bisect when this upticked so sharply again?

> I did a lot of retriggers around the 24th when this seemed to be more
> sustained:
> https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-
> searchStr=win%20vm%20debug%20e10s%20bc1&tochange=efa1ee67e2f18bf17d73af8a45b5
> 6f17910f0cf7&fromchange=9c7bde46a67a1975e2e3eb2ecbac00784b783e9e&selectedJob=
> 96542744
> 
> many of the 'newtab' tests are failing, so possibly there is some underlying
> issue?

(In reply to Joel Maher ( :jmaher) from comment #20)
> related bugs would be:
> bug 1244713 - browser_newtab_reportLinkAction.js (no real recent history,
> but retriggers show the failure)
> bug 1338836 - browser_newtab_bug765628.js (win7/win8/osx debug failures -
> ~20/week)
> bug 1260958 - browser_newtab_unpin.js (win7/debug ~5 failures in the last
> week)
> 
> a lot of common failures on win7/debug.  
> 
> :gijs, do you have ideas on what might be a common failure between this bug
> and the above mentioned 3 bugs.

No idea without diving in, and I don't really have cycles to do that. :-(

I believe the activity stream team suggested they should own this code, so perhaps there's someone on that team who could look into this?

Comment 22

[Joel Maher ( :jmaher ) (UTC -8)]

there is not a clear spike when looking at the incoming failures:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-04-10&endday=2017-04-30&tree=all

I thought april 24th was the spike, but I couldn't narrow it down realistically, the failures are not that consistent and there are so many other failures.

Let me look at the activity stream team, I really need to find real owners for code who care about the data coming from the tests, not just people that are waiting another month or so to replace the tests with something new- I will follow up with them today!

Comment 23

[Joel Maher ( :jmaher ) (UTC -8)]

and one more bug for newtab:
bug 1119906 - browser_newtab_bug722273.js (primarily win8-debug)

Comment 24

[Joel Maher ( :jmaher ) (UTC -8)]

working with :mardak on this, some changes that will affect newtab tests in bug 1241390.

Comment 25

[Intermittent Failures Robot]

54 failures in 770 pushes (0.07 failures/push) were associated with this bug in the last 7 days. 

This is the #18 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* mozilla-inbound: 20
* autoland: 20
* try: 6
* mozilla-central: 5
* graphics: 2
* oak: 1

Platform breakdown:
* windows7-32-vm: 29
* windows8-64: 13
* osx-10-10: 12

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-05-01&endday=2017-05-07&tree=all

Comment 26

[Ed Lee :Mardak]

(In reply to OrangeFactor Robot from comment #25)
> https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-05-01&endday=2017-05-07&tree=all
Looks like 54 of 55 failures were e10s. Maybe there was something in the e10s-enabling of the test? from this commit:
https://hg.mozilla.org/mozilla-central/rev/e71083fd76ca


Bug 1241390 is landing, and there should be 50x runs of newtab/ browser-chrome tests:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1fa6bf712b71&filter-searchStr=browser-chrome%20e10s&group_state=expanded

There were none for this bug (browser_newtab_bug725996.js) but there were 5 for bug 1338836 (browser_newtab_bug765628.js).

Comment 27

[Intermittent Failures Robot]

31 failures in 879 pushes (0.035 failures/push) were associated with this bug in the last 7 days. 

This is the #45 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 16
* mozilla-inbound: 8
* mozilla-central: 3
* try: 2
* oak: 1
* graphics: 1

Platform breakdown:
* windows7-32-vm: 16
* windows8-64: 11
* osx-10-10: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-05-08&endday=2017-05-14&tree=all

Comment 28

[Intermittent Failures Robot]

7 failures in 777 pushes (0.009 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 3
* try: 2
* mozilla-inbound: 1
* mozilla-central: 1

Platform breakdown:
* osx-10-10: 4
* windows8-64: 2
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-05-15&endday=2017-05-21&tree=all

Comment 29

[Joel Maher ( :jmaher ) (UTC -8)]

did Bug 1241390 reduce this?  It looks like our failures reduced quite a bit, but hard to tell if there was a root cause for this.  We still get a few failures/week coming in:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-05-01&endday=2017-06-07&tree=all

Comment 30

[Intermittent Failures Robot]

13 failures in 864 pushes (0.015 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 6
* mozilla-inbound: 4
* try: 1
* mozilla-central: 1
* cedar: 1

Platform breakdown:
* windows8-64: 5
* osx-10-10: 5
* windows7-32-vm: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-06-05&endday=2017-06-11&tree=all

Comment 31

[Intermittent Failures Robot]

4 failures in 814 pushes (0.005 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 2
* mozilla-inbound: 1
* mozilla-beta: 1

Platform breakdown:
* windows7-32-vm: 2
* osx-10-10: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-06-12&endday=2017-06-18&tree=all

Comment 32

[Intermittent Failures Robot]

3 failures in 892 pushes (0.003 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 3

Platform breakdown:
* windows8-64: 2
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-06-19&endday=2017-06-25&tree=all

Comment 33

[Intermittent Failures Robot]

5 failures in 718 pushes (0.007 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 2
* mozilla-central: 1
* mozilla-beta: 1
* autoland: 1

Platform breakdown:
* windows8-64: 2
* osx-10-10: 2
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-06-26&endday=2017-07-02&tree=all

Comment 34

[Intermittent Failures Robot]

4 failures in 656 pushes (0.006 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 3
* mozilla-inbound: 1

Platform breakdown:
* windows8-64: 3
* osx-cross: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-07-03&endday=2017-07-09&tree=all

Comment 35

[Intermittent Failures Robot]

8 failures in 720 pushes (0.011 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 5
* mozilla-inbound: 2
* mozilla-central: 1

Platform breakdown:
* windows7-32-vm: 3
* windows8-64: 2
* osx-10-10: 2
* osx-cross: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-07-10&endday=2017-07-16&tree=all

Comment 36

[Intermittent Failures Robot]

3 failures in 822 pushes (0.004 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 2
* try: 1

Platform breakdown:
* windows7-32: 1
* windows10-64: 1
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-07-17&endday=2017-07-23&tree=all

Comment 37

[Intermittent Failures Robot]

2 failures in 1008 pushes (0.002 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 2

Platform breakdown:
* windows7-32: 1
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-07-24&endday=2017-07-30&tree=all

Comment 38

[Intermittent Failures Robot]

14 failures in 888 pushes (0.016 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 7
* autoland: 6
* mozilla-central: 1

Platform breakdown:
* osx-10-10: 10
* windows7-32: 3
* windows8-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-07-31&endday=2017-08-06&tree=all

Comment 39

[Intermittent Failures Robot]

15 failures in 901 pushes (0.017 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 8
* autoland: 4
* try: 2
* mozilla-central: 1

Platform breakdown:
* osx-10-10: 9
* windows7-32: 5
* windows8-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-08-07&endday=2017-08-13&tree=all

Comment 40

[Intermittent Failures Robot]

31 failures in 949 pushes (0.033 failures/push) were associated with this bug in the last 7 days.   

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 20
* mozilla-inbound: 6
* try: 2
* mozilla-central: 2
* mozilla-beta: 1

Platform breakdown:
* osx-10-10: 15
* windows8-64: 8
* windows7-32: 6
* windows7-32-stylo: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-08-14&endday=2017-08-20&tree=all

Comment 41

[Joel Maher ( :jmaher ) (UTC -8)]

:mardak, this failure has increased in the last week, is there any chance you have time or know of someone who might in order to look into this and try to fix the test?

Comment 42

[Ed Lee :Mardak]

Given that the tested functionality is off by default in 57, maybe it makes sense to just disable this test? Or at least disabled for debug mac/win?

Comment 43

[Joel Maher ( :jmaher ) (UTC -8)]

is there danger in reverting this default for 57?  Otherwise I think disabling this test in general is the right move- are there other tests in browser/base/content/test/newtab/* which are not testing anything by default?  Possibly we disable most or all of them?

Comment 44

[Ed Lee :Mardak]

Users can still turn off activity stream about:newtab to get back the original tiles about:newtab. These browser/base/content/test/newtab tests are running with activity-stream disabled: https://searchfox.org/mozilla-central/source/browser/base/content/test/newtab/head.js#9

It seems more likely that about:newtab will have activity stream in 57 but not as likely for about:home in 57. But I would think soon after (?) 57, the pref will go away and activity stream will be the only about:newtab.

Comment 45

[Ed Lee :Mardak]

Attachment: Bug 1338848 - Skip browser_newtab_bug725996.js for mac/win debug.

Review commit: https://reviewboard.mozilla.org/r/171814/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/171814/

Comment 46

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8900452 [details]
Bug 1338848 - Skip browser_newtab_bug725996.js for mac/win debug.

https://reviewboard.mozilla.org/r/171814/#review177116

thanks for this

Comment 47

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6e979acd063f
Skip browser_newtab_bug725996.js for mac/win debug. r=jmaher

Comment 48

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/6e979acd063f

Comment 49

[Ed Lee :Mardak]

jmaher, how should the bug flags/whiteboard be set? leave open and note disabled?

Comment 50

[Joel Maher ( :jmaher ) (UTC -8)]

set the flags- we should determine at a point a few releases in the future if we want these newtab tests at all.

Comment 51

[Intermittent Failures Robot]

6 failures in 908 pushes (0.007 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 3
* mozilla-inbound: 2
* mozilla-central: 1

Platform breakdown:
* windows8-64: 2
* windows7-32: 2
* osx-10-10: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-08-21&endday=2017-08-27&tree=all

Comment 52

[Intermittent Failures Robot]

1 failures in 947 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-release: 1

Platform breakdown:
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338848&startday=2017-10-09&endday=2017-10-15&tree=all

Comment 53

[Firefox Bug Husbandry Bot]

Removing leave-open keyword from resolved bugs, per :sylvestre.