Closed Bug 1488939 Opened Last year Closed 11 months ago

Perma-failing services/sync/tests/unit/test_browserid_identity_telemetry.js | xpcshell return code: 0 when the Gecko version number is bumped to 65 on 2018-10-22

Categories

(Firefox :: Sync, defect, P1)

defect

Tracking

()

VERIFIED FIXED
Firefox 64
Tracking Status
geckoview62 --- unaffected
firefox-esr60 --- unaffected
firefox62 --- unaffected
firefox63 --- unaffected
firefox64 + verified

People

(Reporter: apavel, Assigned: markh)

References

Details

Attachments

(1 file)

Central as Beta simulation

https://treeherder.mozilla.org/#/jobs?repo=try&revision=701dd8f52ed8236eb93a1bf297ef81997a391170&selectedJob=197617811

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=197617811&repo=try&lineNumber=1956

[task 2018-09-05T14:27:54.792Z] 14:27:54     INFO -  TEST-START | services/sync/tests/unit/test_browserid_identity_telemetry.js
[task 2018-09-05T14:27:55.857Z] 14:27:55  WARNING -  TEST-UNEXPECTED-FAIL | services/sync/tests/unit/test_browserid_identity_telemetry.js | xpcshell return code: 0
[task 2018-09-05T14:27:55.858Z] 14:27:55     INFO -  TEST-INFO took 1064ms
[task 2018-09-05T14:27:55.859Z] 14:27:55     INFO -  >>>>>>>
[task 2018-09-05T14:27:55.860Z] 14:27:55     INFO -  PID 10982 | [10982, Main Thread] WARNING: Couldn't get the user appdata directory. Crash events may not be produced.: file /builds/worker/workspace/build/src/toolkit/crashreporter/nsExceptionHandler.cpp, line 2727
[task 2018-09-05T14:27:55.864Z] 14:27:55     INFO -  PID 10982 | JavaScript strict warning: /builds/worker/workspace/build/tests/xpcshell/tests/services/sync/tests/unit/head_helpers.js -> resource://testing-common/sinon-2.3.2.js, line 8941: ReferenceError: reference to undefined property "iso-8859-8-i"
[task 2018-09-05T14:27:55.864Z] 14:27:55     INFO -  (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2018-09-05T14:27:55.864Z] 14:27:55     INFO -  (xpcshell/head.js) | test run_next_test 0 pending (2)
[task 2018-09-05T14:27:55.864Z] 14:27:55     INFO -  (xpcshell/head.js) | test MAIN run_test finished (2)
[task 2018-09-05T14:27:55.864Z] 14:27:55     INFO -  running event loop
[task 2018-09-05T14:27:55.864Z] 14:27:55     INFO -  "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "iso-8859-8-i"" {file: "/builds/worker/workspace/build/tests/xpcshell/tests/services/sync/tests/unit/head_helpers.js -> resource://testing-common/sinon-2.3.2.js" line: 8941}]"
[task 2018-09-05T14:27:55.865Z] 14:27:55     INFO -  services/sync/tests/unit/test_browserid_identity_telemetry.js | Starting head_setup
[task 2018-09-05T14:27:55.865Z] 14:27:55     INFO -  (xpcshell/head.js) | test head_setup pending (2)
[task 2018-09-05T14:27:55.865Z] 14:27:55     INFO -  (xpcshell/head.js) | test run_next_test 0 finished (2)
[task 2018-09-05T14:27:55.865Z] 14:27:55     INFO -  (xpcshell/head.js) | test run_next_test 1 pending (2)
[task 2018-09-05T14:27:55.865Z] 14:27:55     INFO -  (xpcshell/head.js) | test head_setup finished (2)
[task 2018-09-05T14:27:55.865Z] 14:27:55     INFO -  services/sync/tests/unit/test_browserid_identity_telemetry.js | Starting test_success
[task 2018-09-05T14:27:55.866Z] 14:27:55     INFO -  (xpcshell/head.js) | test test_success pending (2)
[task 2018-09-05T14:27:55.867Z] 14:27:55     INFO -  PID 10982 | JavaScript strict warning: resource://testing-common/services/sync/utils.js, line 290: ReferenceError: reference to undefined property "services.sync.sync_login_state_transitions"
[task 2018-09-05T14:27:55.867Z] 14:27:55  WARNING -  TEST-UNEXPECTED-FAIL | services/sync/tests/unit/test_browserid_identity_telemetry.js | test_success - [test_success : 12] "undefined" deepEqual {"SUCCESS":1}
[task 2018-09-05T14:27:55.868Z] 14:27:55     INFO -  /builds/worker/workspace/build/tests/xpcshell/tests/services/sync/tests/unit/test_browserid_identity_telemetry.js:test_success:12
[task 2018-09-05T14:27:55.868Z] 14:27:55     INFO -  /builds/worker/workspace/build/tests/xpcshell/head.js:_run_next_test:1459
[task 2018-09-05T14:27:55.869Z] 14:27:55     INFO -  /builds/worker/workspace/build/tests/xpcshell/head.js:run:692
[task 2018-09-05T14:27:55.869Z] 14:27:55     INFO -  /builds/worker/workspace/build/tests/xpcshell/head.js:_do_main:219
[task 2018-09-05T14:27:55.870Z] 14:27:55     INFO -  /builds/worker/workspace/build/tests/xpcshell/head.js:_execute_test:533
[task 2018-09-05T14:27:55.870Z] 14:27:55     INFO -  -e:null:1
[task 2018-09-05T14:27:55.871Z] 14:27:55     INFO -  exiting test
[task 2018-09-05T14:27:55.871Z] 14:27:55     INFO -  NS_ERROR_ABORT:
[task 2018-09-05T14:27:55.872Z] 14:27:55     INFO -  _abort_failed_test@/builds/worker/workspace/build/tests/xpcshell/head.js:746:9
[task 2018-09-05T14:27:55.872Z] 14:27:55     INFO -  do_report_result@/builds/worker/workspace/build/tests/xpcshell/head.js:853:5
[task 2018-09-05T14:27:55.873Z] 14:27:55     INFO -  Assert<@/builds/worker/workspace/build/tests/xpcshell/head.js:55:5
[task 2018-09-05T14:27:55.873Z] 14:27:55     INFO -  proto.report@resource://testing-common/Assert.jsm:214:5
[task 2018-09-05T14:27:55.873Z] 14:27:55     INFO -  deepEqual@resource://testing-common/Assert.jsm:281:3
[task 2018-09-05T14:27:55.874Z] 14:27:55     INFO -  test_success@/builds/worker/workspace/build/tests/xpcshell/tests/services/sync/tests/unit/test_browserid_identity_telemetry.js:12:3
[task 2018-09-05T14:27:55.874Z] 14:27:55     INFO -  _run_next_test@/builds/worker/workspace/build/tests/xpcshell/head.js:1459:11
[task 2018-09-05T14:27:55.875Z] 14:27:55     INFO -  run@/builds/worker/workspace/build/tests/xpcshell/head.js:692:9
[task 2018-09-05T14:27:55.875Z] 14:27:55     INFO -  _do_main@/builds/worker/workspace/build/tests/xpcshell/head.js:219:3
[task 2018-09-05T14:27:55.876Z] 14:27:55     INFO -  _execute_test@/builds/worker/workspace/build/tests/xpcshell/head.js:533:5
[task 2018-09-05T14:27:55.876Z] 14:27:55     INFO -  @-e:1:1
[task 2018-09-05T14:27:55.877Z] 14:27:55     INFO -  exiting test
[task 2018-09-05T14:27:55.877Z] 14:27:55     INFO -  (xpcshell/head.js) | test run_next_test 1 finished (2)
[task 2018-09-05T14:27:55.878Z] 14:27:55     INFO -  "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "services.sync.sync_login_state_transitions"" {file: "resource://testing-common/services/sync/utils.js" line: 290}]"
[task 2018-09-05T14:27:55.878Z] 14:27:55     INFO -  PID 10982 | [10982, Main Thread] WARNING: OOPDeinit() without successful OOPInit(): file /builds/worker/workspace/build/src/toolkit/crashreporter/nsExceptionHandler.cpp, line 3405
[task 2018-09-05T14:27:55.879Z] 14:27:55     INFO -  PID 10982 | nsStringStats
[task 2018-09-05T14:27:55.879Z] 14:27:55     INFO -  PID 10982 |  => mAllocCount:           8048
[task 2018-09-05T14:27:55.880Z] 14:27:55     INFO -  PID 10982 |  => mReallocCount:            0
[task 2018-09-05T14:27:55.880Z] 14:27:55     INFO -  PID 10982 |  => mFreeCount:            8048
[task 2018-09-05T14:27:55.880Z] 14:27:55     INFO -  PID 10982 |  => mShareCount:          12129
[task 2018-09-05T14:27:55.881Z] 14:27:55     INFO -  PID 10982 |  => mAdoptCount:            382
[task 2018-09-05T14:27:55.881Z] 14:27:55     INFO -  PID 10982 |  => mAdoptFreeCount:        382
[task 2018-09-05T14:27:55.882Z] 14:27:55     INFO -  PID 10982 |  => Process ID: 10982, Thread ID: 140447772512064
[task 2018-09-05T14:27:55.882Z] 14:27:55     INFO -  <<<<<<<

Histograms expire in firefox 65.
Flags: needinfo?(markh)
Summary: Perma-failing services/sync/tests/unit/test_browserid_identity_telemetry.js | xpcshell return code: 0 when Gecko merges to beta on 2018-10-22 → Perma-failing services/sync/tests/unit/test_browserid_identity_telemetry.js | xpcshell return code: 0 when Gecko 64 merges to beta on 2018-10-22
Summary: Perma-failing services/sync/tests/unit/test_browserid_identity_telemetry.js | xpcshell return code: 0 when Gecko 64 merges to beta on 2018-10-22 → Perma-failing services/sync/tests/unit/test_browserid_identity_telemetry.js | xpcshell return code: 0 when Gecko 64 merges to beta on 2018-10-15
In bug 1397610 we are still trying to understand what this telemetry is telling us (although I must admit it has slipped off our radar a little) - so I'm going to extend these probes for a few versions.
Assignee: nobody → markh
Status: NEW → ASSIGNED
Flags: needinfo?(markh)
Priority: -- → P1
See Also: → 1397610
MozReview-Commit-ID: AwFg6TVcVug
Summary: Perma-failing services/sync/tests/unit/test_browserid_identity_telemetry.js | xpcshell return code: 0 when Gecko 64 merges to beta on 2018-10-15 → Perma-failing services/sync/tests/unit/test_browserid_identity_telemetry.js | xpcshell return code: 0 when the Gecko version number is bumped to 65 on 208-10-22
The renewal of a data collection is subject to Data Collection Review: https://wiki.mozilla.org/Firefox/Data_Collection

It might be helpful for the reviewing Data Steward to be pointed to the originating bug 1375635, as it does have a review. (It's an older review, but it checks out).
Comment on attachment 9006754 [details]
Bug 1488939 - extend expiry of sync 'yellow state' users. r?gfritzsche

Georg Fritzsche [:gfritzsche] has approved the revision.
Attachment #9006754 - Flags: review+
Summary: Perma-failing services/sync/tests/unit/test_browserid_identity_telemetry.js | xpcshell return code: 0 when the Gecko version number is bumped to 65 on 208-10-22 → Perma-failing services/sync/tests/unit/test_browserid_identity_telemetry.js | xpcshell return code: 0 when the Gecko version number is bumped to 65 on 2018-10-22
Comment on attachment 9006754 [details]
Bug 1488939 - extend expiry of sync 'yellow state' users. r?gfritzsche

(In reply to Chris H-C :chutten from comment #3)
> The renewal of a data collection is subject to Data Collection Review:
> https://wiki.mozilla.org/Firefox/Data_Collection
> 
> It might be helpful for the reviewing Data Steward to be pointed to the
> originating bug 1375635, as it does have a review. (It's an older review,
> but it checks out).

Thanks Chris - given you are a data steward and are already on this bug, do you mind doing the honors?

* What questions will you answer with this data?

For how long and how often users appear to be disconnected from sync.

* Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

Establish baselines or measure changes in product or platform quality or performance.

* What alternative methods did you consider to answer these questions? Why were they not sufficient?

We investigated using server-side data but data collected there is not suffucient.

* Can current instrumentation answer these questions?

No

* List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories on the Mozilla wiki.

See bug 1375635 where this telemetry initially landed.

* How long will this data be collected?

Until Firefox 68

* If this data collection is default on, what is the opt-out mechanism for users?

Using the same histogram and scalar telemetry mechanisms

* Please provide a general description of how you will analyze this data.

via redash and/or jupyter

* Where do you intend to share the results of your analysis?

Internally, although we do not consider this data sensitive (eg, we occasionally comment on the data in public bugs)
Attachment #9006754 - Flags: review?(chutten)
Comment on attachment 9006754 [details]
Bug 1488939 - extend expiry of sync 'yellow state' users. r?gfritzsche

datareview+ for the same reasons as the original review response in bug 1375635 comment# 23
Attachment #9006754 - Flags: review?(chutten) → review+
Pushed by mhammond@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3bda9cc12fbf
extend expiry of sync 'yellow state' users. r=gfritzsche
Comment on attachment 9006754 [details]
Bug 1488939 - extend expiry of sync 'yellow state' users. r?gfritzsche

Chris H-C :chutten has been removed from the revision.
Attachment #9006754 - Flags: review+
https://hg.mozilla.org/mozilla-central/rev/3bda9cc12fbf
Status: ASSIGNED → RESOLVED
Closed: 11 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 64
You need to log in before you can comment on or make changes to this bug.