1607894 - Intermittent normandy/test/browser/browser_RecipeRunner.js | recipes/all recipes with matching should have their signature verified - [] deepEqual [[{"name":"match","action":"matchAction","filter_expression":"true"},{"signature":"abc"},"

Status: REOPENED

(Firefox :: Normandy Client, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: rmaries [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=284050452&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Gky4q57EQbeyVZivuqTkwg/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Gky4q57EQbeyVZivuqTkwg/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

Comment 1

[Intermittent Failures Robot]

6 failures in 4472 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 4
• autoland: 2

Platform breakdown:

• macosx1014-64: 4
• macosx1014-64-shippable: 1
• linux1804-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1607894&startday=2020-01-06&endday=2020-01-12&tree=all

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 3

[Daniel Varga [:dvarga]]

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=289636111&repo=autoland&lineNumber=4997
This started to spike after Bug 1604363 got landed: https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedJob=289637798&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&searchStr=linux%2Cx64%2Cshippable%2Copt%2Cmochitests%2Ctest-linux64-shippable%2Fopt-mochitest-browser-chrome-e10s-6%2Cm%28bc6%29&revision=6f9bbe196ec0950038fc52e51688f88bcafae789

Comment 4

[Intermittent Failures Robot]

22 failures in 882 pushes (0.025 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 18
• mozilla-central: 2
• try: 2

Platform breakdown:

• linux64: 3
• linux64-shippable: 3
• linux1804-64: 1
• linux1804-64-shippable: 12
• linux1804-64-asan: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1607894&startday=2020-02-20&endday=2020-02-20&tree=all

Comment 5

[Intermittent Failures Robot]

34 failures in 4045 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

This is the #29 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 22
• mozilla-central: 9
• try: 3

Platform breakdown:

• linux64: 3
• linux64-shippable: 3
• linux1804-64: 5
• linux1804-64-shippable: 19
• linux1804-64-asan: 3
• linux1804-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1607894&startday=2020-02-17&endday=2020-02-23&tree=all

Comment 6

[Michael Cooper [:mythmon]]

Attachment: Bug 1607894 - Add debugging assert to Normandy test for intermittent r=leplatrem

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e385c9242708
Add debugging assert to Normandy test for intermittent r=leplatrem

Comment 8

[Cosmin Sabou [:CosminS]]

Backed out for bc failures on browser_RecipeRunner.js:

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&selectedJob=290691476&resultStatus=testfailed%2Cbusted%2Cexception&revision=e385c9242708eb4cae1fd0dfe200874c71b20423

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=290691476&repo=autoland

Backout link: https://hg.mozilla.org/integration/autoland/rev/f407f21e0921db4120696b843566f206ea8b38e0

[task 2020-02-27T02:09:33.203Z] 02:09:33     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_RecipeRunner.js | getCapabilities should be called - 
[task 2020-02-27T02:09:33.203Z] 02:09:33     INFO - Console message: [JavaScript Error: "1582769372615	app.normandy.recipe-runner	ERROR	Error checking filter for "undefined". Filter: [undefined]. Error: "TypeError: can't access property "replace", expr is undefined"" {file: "resource://gre/modules/Log.jsm" line: 723}]
[task 2020-02-27T02:09:33.204Z] 02:09:33     INFO - append@resource://gre/modules/Log.jsm:723:12
[task 2020-02-27T02:09:33.204Z] 02:09:33     INFO - log@resource://gre/modules/Log.jsm:379:16
[task 2020-02-27T02:09:33.204Z] 02:09:33     INFO - error@resource://gre/modules/Log.jsm:387:10
[task 2020-02-27T02:09:33.204Z] 02:09:33     INFO - getRecipeSuitability@resource://normandy/lib/RecipeRunner.jsm:493:11
[task 2020-02-27T02:09:33.204Z] 02:09:33     INFO - async*run@resource://normandy/lib/RecipeRunner.jsm:343:40
[task 2020-02-27T02:09:33.204Z] 02:09:33     INFO - async*test_run_includesCapabilities@chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/browser_RecipeRunner.js:257:24
[task 2020-02-27T02:09:33.204Z] 02:09:33     INFO - inner@resource://testing-common/NormandyTestUtils.jsm:221:17
[task 2020-02-27T02:09:33.204Z] 02:09:33     INFO - async*wrappedTestFunction@chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/head.js:300:15
[task 2020-02-27T02:09:33.205Z] 02:09:33     INFO - wrappedTestFunction@chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/head.js:300:15
[task 2020-02-27T02:09:33.205Z] 02:09:33     INFO - Tester_execTest/<@chrome://mochikit/content/browser-test.js:1062:34
[task 2020-02-27T02:09:33.205Z] 02:09:33     INFO - async*Tester_execTest@chrome://mochikit/content/browser-test.js:1097:11
[task 2020-02-27T02:09:33.205Z] 02:09:33     INFO - nextTest/<@chrome://mochikit/content/browser-test.js:925:14
[task 2020-02-27T02:09:33.205Z] 02:09:33     INFO - SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:914:23
[task 2020-02-27T02:09:33.205Z] 02:09:33     INFO - 
[task 2020-02-27T02:09:33.205Z] 02:09:33     INFO - Console message: [JavaScript Warning: "normandy.recipe_freshness - Truncating float/double number."]
[task 2020-02-27T02:09:33.205Z] 02:09:33     INFO - Console message: [JavaScript Error: "1582769372616	app.normandy.recipe-runner	ERROR	Could not execute recipe undefined:: Action undefined is either missing or invalid." {file: "resource://gre/modules/Log.jsm" line: 723}]
[task 2020-02-27T02:09:33.206Z] 02:09:33     INFO - append@resource://gre/modules/Log.jsm:723:12
[task 2020-02-27T02:09:33.206Z] 02:09:33     INFO - log@resource://gre/modules/Log.jsm:379:16
[task 2020-02-27T02:09:33.206Z] 02:09:33     INFO - error@resource://gre/modules/Log.jsm:387:10
[task 2020-02-27T02:09:33.206Z] 02:09:33     INFO - processRecipe@resource://normandy/lib/ActionsManager.jsm:96:11
[task 2020-02-27T02:09:33.206Z] 02:09:33     INFO - run@resource://normandy/lib/RecipeRunner.jsm:344:32
[task 2020-02-27T02:09:33.206Z] 02:09:33     INFO - async*test_run_includesCapabilities@chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/browser_RecipeRunner.js:257:24
[task 2020-02-27T02:09:33.206Z] 02:09:33     INFO - inner@resource://testing-common/NormandyTestUtils.jsm:221:17
[task 2020-02-27T02:09:33.206Z] 02:09:33     INFO - async*wrappedTestFunction@chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/head.js:300:15
[task 2020-02-27T02:09:33.207Z] 02:09:33     INFO - wrappedTestFunction@chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/head.js:300:15
[task 2020-02-27T02:09:33.207Z] 02:09:33     INFO - Tester_execTest/<@chrome://mochikit/content/browser-test.js:1062:34
[task 2020-02-27T02:09:33.207Z] 02:09:33     INFO - async*Tester_execTest@chrome://mochikit/content/browser-test.js:1097:11
[task 2020-02-27T02:09:33.207Z] 02:09:33     INFO - nextTest/<@chrome://mochikit/content/browser-test.js:925:14
[task 2020-02-27T02:09:33.207Z] 02:09:33     INFO - SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:914:23
[task 2020-02-27T02:09:33.207Z] 02:09:33     INFO - 
[task 2020-02-27T02:09:33.207Z] 02:09:33     INFO - Console message: [JavaScript Warning: "normandy.recipe_freshness - Truncating float/double number."]
[task 2020-02-27T02:09:33.207Z] 02:09:33     INFO - Leaving test bound test_run_includesCapabilities
[task 2020-02-27T02:09:33.207Z] 02:09:33     INFO - Entering test bound testReadFromRemoteSettings
[task 2020-02-27T02:09:33.207Z] 02:09:33     INFO - Buffered messages finished
[task 2020-02-27T02:09:33.207Z] 02:09:33     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/normandy/test/browser/browser_RecipeRunner.js | The recipes should be accesible from Remote Settings - [{"name":"match","action":"matchAction","filter_expression":"true"},{"name":"missing","action":"missingAction","filter_expression":"true"},{"name":"noMatch","action":"noMatchAction","filter_expression":"false"}] deepEqual [{"name":"match","action":"matchAction","filter_expression":"true"},{"name":"noMatch","action":"noMatchAction","filter_expression":"false"},{"name":"missing","action":"missingAction","filter_expression":"true"}] - JS frame :: chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/browser_RecipeRunner.js :: testReadFromRemoteSettings :: line 310
[task 2020-02-27T02:09:33.208Z] 02:09:33     INFO - Stack trace:
[task 2020-02-27T02:09:33.208Z] 02:09:33     INFO - chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/browser_RecipeRunner.js:testReadFromRemoteSettings:310
[task 2020-02-27T02:09:33.208Z] 02:09:33     INFO - chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/head.js:wrappedTestFunction:300
[task 2020-02-27T02:09:33.208Z] 02:09:33     INFO - chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/head.js:wrappedTestFunction:300
[task 2020-02-27T02:09:33.208Z] 02:09:33     INFO - chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/head.js:wrappedTestFunction:300
[task 2020-02-27T02:09:33.208Z] 02:09:33     INFO - chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/head.js:wrappedTestFunction:300
[task 2020-02-27T02:09:33.208Z] 02:09:33     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1062
[task 2020-02-27T02:09:33.208Z] 02:09:33     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1097
[task 2020-02-27T02:09:33.208Z] 02:09:33     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:925
[task 2020-02-27T02:09:33.208Z] 02:09:33     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:914
[task 2020-02-27T02:09:33.208Z] 02:09:33     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_RecipeRunner.js | all recipes should have their signature verified -

Comment 9

[Intermittent Failures Robot]

39 failures in 5233 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

This is the #34 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 18
• try: 13
• mozilla-central: 8

Platform breakdown:

• linux1804-64-asan: 6
• linux1804-64: 11
• linux1804-64-shippable: 16
• windows10-64: 1
• linux1804-64-devedition: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1607894&startday=2020-02-24&endday=2020-03-01&tree=all

Comment 10

[Stefan Hindli [:stefan_hindli]]

Over the last 7 days there are 33 failures present on this bug. These happen on linux1804-64, linux1804-64-asan, linux1804-64-shippable, macosx1014-64, windows7-32-shippable

Here is a recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=291938517&repo=autoland&lineNumber=5774

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ade1286ccf20
Add debugging assert to Normandy test for intermittent r=leplatrem

Comment 12

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/ade1286ccf20

Comment 13

[Intermittent Failures Robot]

37 failures in 5483 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

This is the #28 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 18
• try: 8
• mozilla-central: 11

Platform breakdown:

• linux1804-64-shippable: 20
• linux1804-64-asan: 8
• linux1804-64-devedition: 2
• linux1804-64: 3
• windows7-32-shippable: 1
• macosx1014-64: 1
• windows7-32-devedition: 1
• windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1607894&startday=2020-03-02&endday=2020-03-08&tree=all

Comment 14

[Intermittent Failures Robot]

1 failures in 4955 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• macosx1014-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1607894&startday=2020-03-16&endday=2020-03-22&tree=all

Comment 15

[Intermittent Failures Robot]

6 failures in 5276 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 6

Platform breakdown:

• linux1804-64: 1
• windows7-32: 1
• macosx1014-64: 1
• windows10-64: 1
• windows7-32-shippable: 1
• windows10-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1607894&startday=2020-05-04&endday=2020-05-10&tree=all