1876186 - Intermittent toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | single tracking bug

Status: RESOLVED FIXED

(Firefox :: Nimbus Desktop Client, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: imoraru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=444355215&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/U1yYa_3YTfyA1_xF0jaqVA/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/U1yYa_3YTfyA1_xF0jaqVA/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

[task 2024-01-23T23:41:02.181Z] 23:41:02     INFO -  TEST-START | toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js
[task 2024-01-23T23:41:03.133Z] 23:41:03  WARNING -  TEST-UNEXPECTED-FAIL | toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | xpcshell return code: 0
[task 2024-01-23T23:41:03.134Z] 23:41:03     INFO -  TEST-INFO took 952ms
[task 2024-01-23T23:41:03.134Z] 23:41:03     INFO -  >>>>>>>
[task 2024-01-23T23:41:03.135Z] 23:41:03     INFO -  TEST-PASS | toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | getTempFile - [getTempFile : 44] Sanity check the temporary file doesn't exist. - true == true
[task 2024-01-23T23:41:03.135Z] 23:41:03     INFO -  (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2024-01-23T23:41:03.135Z] 23:41:03     INFO -  (xpcshell/head.js) | test run_next_test 0 pending (2)
[task 2024-01-23T23:41:03.136Z] 23:41:03     INFO -  (xpcshell/head.js) | test MAIN run_test finished (2)
[task 2024-01-23T23:41:03.136Z] 23:41:03     INFO -  running event loop
[task 2024-01-23T23:41:03.136Z] 23:41:03     INFO -  toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | Starting setup setup
[task 2024-01-23T23:41:03.136Z] 23:41:03     INFO -  (xpcshell/head.js) | test setup pending (2)
[task 2024-01-23T23:41:03.138Z] 23:41:03     INFO -  (xpcshell/head.js) | test run_next_test 0 finished (2)
[task 2024-01-23T23:41:03.138Z] 23:41:03     INFO -  (xpcshell/head.js) | test run_next_test 1 pending (2)
[task 2024-01-23T23:41:03.138Z] 23:41:03     INFO -  (xpcshell/head.js) | test setup finished (2)
[task 2024-01-23T23:41:03.138Z] 23:41:03     INFO -  toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | Starting test_updateRecipes_activeExperiments
[task 2024-01-23T23:41:03.138Z] 23:41:03     INFO -  (xpcshell/head.js) | test test_updateRecipes_activeExperiments pending (2)
[task 2024-01-23T23:41:03.138Z] 23:41:03     INFO -  (xpcshell/head.js) | test run_next_test 1 finished (2)
[task 2024-01-23T23:41:03.138Z] 23:41:03     INFO -  TEST-PASS | toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | test_updateRecipes_activeExperiments - [test_updateRecipes_activeExperiments : 48] Should match active experiments - true == true
[task 2024-01-23T23:41:03.139Z] 23:41:03     INFO -  TEST-PASS | toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | test_updateRecipes_activeExperiments - [test_updateRecipes_activeExperiments : 50] Store should have no active enrollments - [] deepEqual []
[task 2024-01-23T23:41:03.139Z] 23:41:03     INFO -  TEST-PASS | toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | test_updateRecipes_activeExperiments - [test_updateRecipes_activeExperiments : 50] Store should have no inactive enrollments - [] deepEqual []
[task 2024-01-23T23:41:03.140Z] 23:41:03     INFO -  (xpcshell/head.js) | test run_next_test 2 pending (2)
[task 2024-01-23T23:41:03.140Z] 23:41:03     INFO -  (xpcshell/head.js) | test test_updateRecipes_activeExperiments finished (2)
[task 2024-01-23T23:41:03.140Z] 23:41:03     INFO -  toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | Starting test_updateRecipes_isFirstRun
[task 2024-01-23T23:41:03.140Z] 23:41:03     INFO -  (xpcshell/head.js) | test test_updateRecipes_isFirstRun pending (2)
[task 2024-01-23T23:41:03.141Z] 23:41:03     INFO -  (xpcshell/head.js) | test run_next_test 2 finished (2)
[task 2024-01-23T23:41:03.141Z] 23:41:03     INFO -  TEST-PASS | toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | test_updateRecipes_isFirstRun - [test_updateRecipes_isFirstRun : 69] Should match first run - true == true
[task 2024-01-23T23:41:03.141Z] 23:41:03     INFO -  TEST-PASS | toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | test_updateRecipes_isFirstRun - [test_updateRecipes_isFirstRun : 71] Store should have no active enrollments - [] deepEqual []
[task 2024-01-23T23:41:03.142Z] 23:41:03     INFO -  TEST-PASS | toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | test_updateRecipes_isFirstRun - [test_updateRecipes_isFirstRun : 71] Store should have no inactive enrollments - [] deepEqual []
[task 2024-01-23T23:41:03.142Z] 23:41:03     INFO -  (xpcshell/head.js) | test run_next_test 3 pending (2)
[task 2024-01-23T23:41:03.142Z] 23:41:03     INFO -  (xpcshell/head.js) | test test_updateRecipes_isFirstRun finished (2)
[task 2024-01-23T23:41:03.142Z] 23:41:03     INFO -  toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | Starting test_updateRecipes_invalidFeatureId
[task 2024-01-23T23:41:03.143Z] 23:41:03     INFO -  (xpcshell/head.js) | test test_updateRecipes_invalidFeatureId pending (2)
[task 2024-01-23T23:41:03.143Z] 23:41:03     INFO -  (xpcshell/head.js) | test run_next_test 3 finished (2)
[task 2024-01-23T23:41:03.143Z] 23:41:03     INFO -  PID 1324 | console.error: "Experiment 372c6736-8bd6-4fe7-8952-c830a025a6ca has unknown featureId: invalid-feature-id"
[task 2024-01-23T23:41:03.143Z] 23:41:03     INFO -  PID 1324 | console.error: "Experiment 372c6736-8bd6-4fe7-8952-c830a025a6ca has unknown featureId: invalid-feature-id"
[task 2024-01-23T23:41:03.144Z] 23:41:03     INFO -  TEST-PASS | toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | test_updateRecipes_invalidFeatureId - [test_updateRecipes_invalidFeatureId : 111] No recipes - true == true
<...>
[task 2024-01-23T23:41:03.323Z] 23:41:03     INFO -  TEST-PASS | toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | test_update_experiments_ordered_by_published_date - [test_update_experiments_ordered_by_published_date : 1673] Store should have no inactive enrollments - [] deepEqual []
[task 2024-01-23T23:41:03.323Z] 23:41:03     INFO -  (xpcshell/head.js) | test run_next_test 23 pending (2)
[task 2024-01-23T23:41:03.323Z] 23:41:03     INFO -  (xpcshell/head.js) | test test_update_experiments_ordered_by_published_date finished (2)
[task 2024-01-23T23:41:03.324Z] 23:41:03     INFO -  toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | Starting test_record_is_ready_no_value_for_nimbus_is_ready_feature
[task 2024-01-23T23:41:03.324Z] 23:41:03     INFO -  (xpcshell/head.js) | test test_record_is_ready_no_value_for_nimbus_is_ready_feature pending (2)
[task 2024-01-23T23:41:03.324Z] 23:41:03     INFO -  (xpcshell/head.js) | test run_next_test 23 finished (2)
[task 2024-01-23T23:41:03.325Z] 23:41:03  WARNING -  TEST-UNEXPECTED-FAIL | toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js | test_record_is_ready_no_value_for_nimbus_is_ready_feature - [test_record_is_ready_no_value_for_nimbus_is_ready_feature : 1692] 2 == 1
[task 2024-01-23T23:41:03.325Z] 23:41:03     INFO -  Z:/task_170604874410292/build/tests/xpcshell/tests/toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js:test_record_is_ready_no_value_for_nimbus_is_ready_feature:1692
[task 2024-01-23T23:41:03.326Z] 23:41:03     INFO -  Z:\task_170604874410292\build\tests\xpcshell\head.js:_do_main:245
[task 2024-01-23T23:41:03.326Z] 23:41:03     INFO -  Z:\task_170604874410292\build\tests\xpcshell\head.js:_execute_test:596
[task 2024-01-23T23:41:03.326Z] 23:41:03     INFO -  -e:null:1
[task 2024-01-23T23:41:03.326Z] 23:41:03     INFO -  exiting test
[task 2024-01-23T23:41:03.327Z] 23:41:03     INFO -  Unexpected exception NS_ERROR_ABORT:
[task 2024-01-23T23:41:03.327Z] 23:41:03     INFO -  _abort_failed_test@Z:\task_170604874410292\build\tests\xpcshell\head.js:869:20
[task 2024-01-23T23:41:03.327Z] 23:41:03     INFO -  do_report_result@Z:\task_170604874410292\build\tests\xpcshell\head.js:981:5
[task 2024-01-23T23:41:03.327Z] 23:41:03     INFO -  Assert<@Z:\task_170604874410292\build\tests\xpcshell\head.js:70:21
[task 2024-01-23T23:41:03.328Z] 23:41:03     INFO -  Assert.prototype.report@resource://testing-common/Assert.sys.mjs:244:10
[task 2024-01-23T23:41:03.328Z] 23:41:03     INFO -  equal@resource://testing-common/Assert.sys.mjs:286:8
[task 2024-01-23T23:41:03.329Z] 23:41:03     INFO -  test_record_is_ready_no_value_for_nimbus_is_ready_feature@Z:/task_170604874410292/build/tests/xpcshell/tests/toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js:1692:12
[task 2024-01-23T23:41:03.329Z] 23:41:03     INFO -  _do_main@Z:\task_170604874410292\build\tests\xpcshell\head.js:245:6
[task 2024-01-23T23:41:03.329Z] 23:41:03     INFO -  _execute_test@Z:\task_170604874410292\build\tests\xpcshell\head.js:596:5
[task 2024-01-23T23:41:03.329Z] 23:41:03     INFO -  @-e:1:1
[task 2024-01-23T23:41:03.329Z] 23:41:03     INFO -  exiting test
[task 2024-01-23T23:41:03.329Z] 23:41:03     INFO -  <<<<<<<
[task 2024-01-23T23:41:03.332Z] 23:41:03     INFO -  TEST-START | toolkit/components/nimbus/test/unit/test_RemoteSettingsExperimentLoader_updateRecipes.js

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

:charlie, since you are the author of the regressor, bug 1875510, could you take a look?

For more information, please visit BugBot documentation.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1875510

Comment 3

[Sandor Molnar[:smolnar]]

started failing on tier 1 jobs

Comment 4

[Charlie Humphreys [:charlie] [:jeddai]]

Attachment: Bug 1876186 - fix intermittently failing test in test_RemoteSettingsExperimentLoader_updateRecipes.js r?barret

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2fd6f68fee3a
fix intermittently failing test in test_RemoteSettingsExperimentLoader_updateRecipes.js r=barret

Comment 6

[Charlie Humphreys [:charlie] [:jeddai]]

Attachment: Bug 1876186 - fix intermittently failing test in test_RemoteSettingsExperimentLoader_updateRecipes.js r?barret

Original Revision: https://phabricator.services.mozilla.com/D199662

Comment 7

[Beth Rennie [:beth] (she/her)]

Attachment: Bug 1876186 - fix intermittently failing test in test_RemoteSettingsExperimentLoader_updateRecipes.js r?barret

Original Revision: https://phabricator.services.mozilla.com/D199662

Comment 8

[Phabricator Automation]

Uplift Approval Request

• String changes made/needed: None
• Risk associated with taking this patch: Low
• User impact if declined: We will not be able to evaluate the mechanism for long term holdback studies with a statistically signficant population before the feature hits release in 123
• Explanation of risk level: These patches essentially only add a new telemetry event. No user facing changes.
• Is Android affected?: no
• Fix verified in Nightly: no
• Code covered by automated testing: yes
• Steps to reproduce for manual QE testing: N / A
• Needs manual QE test: no

Comment 9

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/2fd6f68fee3a

Comment 10

[Intermittent Failures Robot]

9 failures in 4034 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 8
• mozilla-central: 1

Platform and build breakdown:

• windows11-32-2009-qr: 4
  • opt: 3
  • debug: 1
• windows11-64-2009-qr: 2
  • opt: 2
• linux1804-64-qr: 1
  • opt: 1
• windows11-64-2009-shippable-qr: 1
  • opt: 1
• windows11-32-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1876186&startday=2024-01-22&endday=2024-01-28&tree=all

Comment 11

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/0d874603541f