1763622 - Perma services/common/tests/unit/test_uptake_telemetry.js | xpcshell return code: 0 | | test_each_status_can_be_caught_in_snapshot - [test_each_status_can_be_caught_in_snapshot : 66] when Gecko 100 merges to release on 2022-04-25

Status: RESOLVED FIXED

(Cloud Services :: Firefox: Common, defect)

Description

[Marian-Vasile Laza]

Beta-as-release simulation
How to run these simulations
Failure log:

[task 2022-04-07T12:31:41.944Z] 12:31:41     INFO -  TEST-START | services/common/tests/unit/test_uptake_telemetry.js
[task 2022-04-07T12:31:42.078Z] 12:31:42  WARNING -  TEST-UNEXPECTED-FAIL | services/common/tests/unit/test_uptake_telemetry.js | xpcshell return code: 0
[task 2022-04-07T12:31:42.079Z] 12:31:42     INFO -  TEST-INFO took 133ms
[task 2022-04-07T12:31:42.080Z] 12:31:42     INFO -  >>>>>>>
[task 2022-04-07T12:31:42.087Z] 12:31:42     INFO -  (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2022-04-07T12:31:42.087Z] 12:31:42     INFO -  (xpcshell/head.js) | test run_next_test 0 pending (2)
[task 2022-04-07T12:31:42.087Z] 12:31:42     INFO -  (xpcshell/head.js) | test MAIN run_test finished (2)
[task 2022-04-07T12:31:42.088Z] 12:31:42     INFO -  running event loop
[task 2022-04-07T12:31:42.088Z] 12:31:42     INFO -  services/common/tests/unit/test_uptake_telemetry.js | Starting test_unknown_status_is_not_reported
[task 2022-04-07T12:31:42.088Z] 12:31:42     INFO -  (xpcshell/head.js) | test test_unknown_status_is_not_reported pending (2)
[task 2022-04-07T12:31:42.088Z] 12:31:42     INFO -  (xpcshell/head.js) | test run_next_test 0 finished (2)
[task 2022-04-07T12:31:42.089Z] 12:31:42     INFO -  TEST-PASS | services/common/tests/unit/test_uptake_telemetry.js | test_unknown_status_is_not_reported - [test_unknown_status_is_not_reported : 33] {} deepEqual {}
[task 2022-04-07T12:31:42.089Z] 12:31:42     INFO -  (xpcshell/head.js) | test run_next_test 1 pending (2)
[task 2022-04-07T12:31:42.090Z] 12:31:42     INFO -  (xpcshell/head.js) | test test_unknown_status_is_not_reported finished (2)
[task 2022-04-07T12:31:42.090Z] 12:31:42     INFO -  services/common/tests/unit/test_uptake_telemetry.js | Starting test_age_is_converted_to_string_and_reported
[task 2022-04-07T12:31:42.091Z] 12:31:42     INFO -  (xpcshell/head.js) | test test_age_is_converted_to_string_and_reported pending (2)
[task 2022-04-07T12:31:42.091Z] 12:31:42     INFO -  (xpcshell/head.js) | test run_next_test 1 finished (2)
[task 2022-04-07T12:31:42.091Z] 12:31:42     INFO -  TEST-PASS | services/common/tests/unit/test_uptake_telemetry.js | test_age_is_converted_to_string_and_reported - [test_age_is_converted_to_string_and_reported : 143] parent must be in snapshot. Has [parent]. - true == true
[task 2022-04-07T12:31:42.091Z] 12:31:42     INFO -  TEST-PASS | services/common/tests/unit/test_uptake_telemetry.js | test_age_is_converted_to_string_and_reported - [test_age_is_converted_to_string_and_reported : 182] After filtering we must have the expected number of events. - 1 == 1
[task 2022-04-07T12:31:42.092Z] 12:31:42     INFO -  TEST-PASS | services/common/tests/unit/test_uptake_telemetry.js | test_age_is_converted_to_string_and_reported - [test_age_is_converted_to_string_and_reported : 217] category in event uptake.remotecontent.result#uptake#remotesettings must match. - "uptake.remotecontent.result" matches "uptake.remotecontent.result"
[task 2022-04-07T12:31:42.092Z] 12:31:42     INFO -  TEST-PASS | services/common/tests/unit/test_uptake_telemetry.js | test_age_is_converted_to_string_and_reported - [test_age_is_converted_to_string_and_reported : 217] method in event uptake.remotecontent.result#uptake#remotesettings must match. - "uptake" matches "uptake"
[task 2022-04-07T12:31:42.093Z] 12:31:42     INFO -  TEST-PASS | services/common/tests/unit/test_uptake_telemetry.js | test_age_is_converted_to_string_and_reported - [test_age_is_converted_to_string_and_reported : 217] object in event uptake.remotecontent.result#uptake#remotesettings must match. - "remotesettings" matches "remotesettings"
[task 2022-04-07T12:31:42.093Z] 12:31:42     INFO -  TEST-PASS | services/common/tests/unit/test_uptake_telemetry.js | test_age_is_converted_to_string_and_reported - [test_age_is_converted_to_string_and_reported : 217] value in event uptake.remotecontent.result#uptake#remotesettings must match. - "success" matches "success"
[task 2022-04-07T12:31:42.093Z] 12:31:42     INFO -  TEST-PASS | services/common/tests/unit/test_uptake_telemetry.js | test_age_is_converted_to_string_and_reported - [test_age_is_converted_to_string_and_reported : 231] Actual event uptake.remotecontent.result#uptake#remotesettings expected to have extra. - true == true
[task 2022-04-07T12:31:42.094Z] 12:31:42     INFO -  TEST-PASS | services/common/tests/unit/test_uptake_telemetry.js | test_age_is_converted_to_string_and_reported - [test_age_is_converted_to_string_and_reported : 238] Expected key source must be in actual extra. Actual keys: [source,age]. - true == true
[task 2022-04-07T12:31:42.094Z] 12:31:42     INFO -  TEST-PASS | services/common/tests/unit/test_uptake_telemetry.js | test_age_is_converted_to_string_and_reported - [test_age_is_converted_to_string_and_reported : 244] extra[source] must match in event uptake.remotecontent.result#uptake#remotesettings. - "s" matches "s"
[task 2022-04-07T12:31:42.095Z] 12:31:42     INFO -  TEST-PASS | services/common/tests/unit/test_uptake_telemetry.js | test_age_is_converted_to_string_and_reported - [test_age_is_converted_to_string_and_reported : 238] Expected key age must be in actual extra. Actual keys: [source,age]. - true == true
[task 2022-04-07T12:31:42.095Z] 12:31:42     INFO -  TEST-PASS | services/common/tests/unit/test_uptake_telemetry.js | test_age_is_converted_to_string_and_reported - [test_age_is_converted_to_string_and_reported : 244] extra[age] must match in event uptake.remotecontent.result#uptake#remotesettings. - "42" matches "42"
[task 2022-04-07T12:31:42.095Z] 12:31:42     INFO -  (xpcshell/head.js) | test run_next_test 2 pending (2)
[task 2022-04-07T12:31:42.095Z] 12:31:42     INFO -  (xpcshell/head.js) | test test_age_is_converted_to_string_and_reported finished (2)
[task 2022-04-07T12:31:42.095Z] 12:31:42     INFO -  services/common/tests/unit/test_uptake_telemetry.js | Starting test_each_status_can_be_caught_in_snapshot
[task 2022-04-07T12:31:42.095Z] 12:31:42     INFO -  (xpcshell/head.js) | test test_each_status_can_be_caught_in_snapshot pending (2)
[task 2022-04-07T12:31:42.096Z] 12:31:42     INFO -  (xpcshell/head.js) | test run_next_test 2 finished (2)
[task 2022-04-07T12:31:42.096Z] 12:31:42  WARNING -  TEST-UNEXPECTED-FAIL | services/common/tests/unit/test_uptake_telemetry.js | test_each_status_can_be_caught_in_snapshot - [test_each_status_can_be_caught_in_snapshot : 66] check events for up_to_date - 1 == 0
[task 2022-04-07T12:31:42.096Z] 12:31:42     INFO -  /builds/worker/workspace/build/tests/xpcshell/tests/services/common/tests/unit/test_uptake_telemetry.js:test_each_status_can_be_caught_in_snapshot:66
[task 2022-04-07T12:31:42.096Z] 12:31:42     INFO -  /builds/worker/workspace/build/tests/xpcshell/head.js:_do_main:240
[task 2022-04-07T12:31:42.097Z] 12:31:42     INFO -  /builds/worker/workspace/build/tests/xpcshell/head.js:_execute_test:603
[task 2022-04-07T12:31:42.097Z] 12:31:42     INFO -  -e:null:1
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  exiting test
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  Unexpected exception NS_ERROR_ABORT:
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  _abort_failed_test@/builds/worker/workspace/build/tests/xpcshell/head.js:874:20
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  do_report_result@/builds/worker/workspace/build/tests/xpcshell/head.js:975:5
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  Assert<@/builds/worker/workspace/build/tests/xpcshell/head.js:75:21
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  proto.report@resource://testing-common/Assert.jsm:228:10
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  equal@resource://testing-common/Assert.jsm:270:8
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  checkUptakeTelemetry@/builds/worker/workspace/build/tests/xpcshell/tests/services/common/tests/unit/head_helpers.js:250:12
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  test_each_status_can_be_caught_in_snapshot@/builds/worker/workspace/build/tests/xpcshell/tests/services/common/tests/unit/test_uptake_telemetry.js:66:23
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  _do_main@/builds/worker/workspace/build/tests/xpcshell/head.js:240:6
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  _execute_test@/builds/worker/workspace/build/tests/xpcshell/head.js:603:5
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  @-e:1:1
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  exiting test
[task 2022-04-07T12:31:42.098Z] 12:31:42     INFO -  <<<<<<<
[task 2022-04-07T12:31:42.100Z] 12:31:42     INFO -  TEST-START | services/settings/test/unit/test_remote_settings.js

We think this is regressed by bug 1620175, Mathieu can you please take a look?

Comment 1

[Marian-Vasile Laza]

We think that X3 Failure Log and X2 Failure Log are also related to your latest changes.
Mathieu can you also please take a look at them? Thank you.

Comment 2

[Mathieu Leplatre [:leplatrem]]

Attachment: Bug 1763622 - Prevent telemetry to be sampled in uptake test r?gbeckley

Since Bug 1620175, we only send telemetry via events, which are sampled in ESR and Release.

When the test suite is ran via a Beta-as-release simulation, the telemetry is sampled and tests fail.
This patch fixes the failing test by forcing it to run as the Nightly channel

Comment 3

[Mathieu Leplatre [:leplatrem]]

We think that X3 Failure Log and X2 Failure Log are also related to your latest changes.

I will update my patch to fix the two others. It's the same root cause.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2ecf9367bb8e
Prevent telemetry to be sampled in uptake test r=gbeckley

Comment 5

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/2ecf9367bb8e

Comment 6

[Mathieu Leplatre [:leplatrem]]

Do you take care of uplift?

Comment 7

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Please include it in the next uplift to beta (a=testonly).

Comment 8

[Dianna Smith [:diannaS]]

https://hg.mozilla.org/releases/mozilla-beta/rev/a75aef8b1f61