TEST-UNEXPECTED-FAIL | /test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js

RESOLVED FIXED in Thunderbird 29.0

Status

defect
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: mkmelin, Assigned: standard8)

Tracking

({intermittent-failure})

Dependency tree / graph

Thunderbird Tracking Flags

(thunderbird27 fixed, thunderbird28 fixed)

Details

Attachments

(1 attachment)

Reporter

Description

6 years ago
All platforms.

https://tbpl.mozilla.org/php/getParsedLog.php?id=32096380&tree=Thunderbird-Trunk&full=1


TEST-INFO | /builds/slave/test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js | running test ...
TEST-UNEXPECTED-FAIL | /builds/slave/test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js | test failed (with xpcshell return code: 0), see following log:
>>>>>>>
TEST-INFO | (xpcshell/head.js) | test MAIN run_test pending (1)
TEST-INFO | (xpcshell/head.js) | test run_next_test 0 pending (2)
TEST-INFO | (xpcshell/head.js) | test MAIN run_test finished (2)
TEST-INFO | (xpcshell/head.js) | running event loop
TEST-INFO | /builds/slave/test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js | Starting test_expired_pings_are_deleted
TEST-INFO | (xpcshell/head.js) | test test_expired_pings_are_deleted pending (2)
TEST-INFO | (xpcshell/head.js) | test run_next_test 0 finished (2)
TEST-UNEXPECTED-FAIL | /builds/slave/test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js | Found 5 unexpected saved pings. - See following stack:
JS frame :: /builds/slave/test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js :: assertNotSaved :: line 160
JS frame :: /builds/slave/test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js :: test_expired_pings_are_deleted :: line 229
JS frame :: resource://gre/modules/Task.jsm :: TaskImpl_run :: line 233
JS frame :: resource://gre/modules/Promise.jsm :: Handler.prototype.process :: line 767
JS frame :: resource://gre/modules/Promise.jsm :: this.PromiseWalker.walkerLoop :: line 531
native frame :: <unknown filename> :: <TOP_LEVEL> :: line 0
TEST-INFO | (xpcshell/head.js) | exiting test
TEST-UNEXPECTED-FAIL | resource://gre/modules/commonjs/sdk/core/promise.js | Unexpected exception 2147500036
TEST-INFO | (xpcshell/head.js) | exiting test
!!! error running onStopped callback: TypeError: callback is not a function
<<<<<<<
Node SPDY server shutting down ...
INFO | Result summary:
INFO | Passed: 2042
INFO | Failed: 1
INFO | Todo: 4
INFO | Retried: 2
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Assignee

Comment 6

6 years ago
mxr/hg say bug 863872 added this test. Normally with these issues, we've got some sort of preference that's differently set, however I don't see that here.
Blocks: 863872
Indeed - I'm the one who added this test. I'll look into this.
Assignee: nobody → mconley
Comment hidden (Legacy TBPL/Treeherder Robot)
So I'm having no luck running the telemetry xpcshell tests locally.

Here's what I'm executing / getting:

Mikes-MacBook-Pro-2:objdir-tb mikeconley$ make xpcshell-tests TEST_PATH=toolkit/components/telemetry/tests
make -C mozilla xpcshell-tests
Have you considered running xpcshell tests via |mach xpcshell-test|? mach is easier to use and has more features than make and it will eventually be the only way to run xpcshell tests. Please consider using mach today!
/Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_virtualenv/bin/python -u /Users/mikeconley/Projects/comm-central/mozilla/config/pythonpath.py \
	  -I./build \
	  -I/Users/mikeconley/Projects/comm-central/mozilla/build \
	  -I./_tests/mozbase/mozinfo \
	  /Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py \
	  --manifest=./_tests/xpcshell/xpcshell.ini \
	  --build-info-json=./mozinfo.json \
	  --no-logfiles \
	  --test-plugin-path='./dist/plugins' \
	  --tests-root-dir=/Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell \
	  --testing-modules-dir=/Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/modules \
	  --xunit-file=/Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/results.xml \
	  --xunit-suite-name=xpcshell \
          --symbols-path=./dist/crashreporter-symbols \
	  --test-path='toolkit/components/telemetry/tests'  \
	  /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/dist/bin/xpcshell
INFO | Using at most 32 threads.
TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryPing.js | running test ...
TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryStopwatch.js | running test ...
TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryLockCount.js | running test ...
TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryLateWrites.js | running test ...
TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_nsITelemetry.js | running test ...
TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryPing_idle.js | running test ...
TEST-INFO | Failed to remove directory: /var/folders/bq/n5qs5f650bl7kmj4h_fqsw2m0000gn/T/tmpZj9nQW. Waiting.
TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_ThirdPartyCookieProbe.js | running test ...
TEST-INFO | test_TelemetryLateWrites.js | Test failed or timed out, will retry.
TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryPingBuildID.js | running test ...
TEST-INFO | test_TelemetryLockCount.js | Test failed or timed out, will retry.
TEST-INFO | test_TelemetryPing.js | Test failed or timed out, will retry.
TEST-INFO | test_TelemetryPingBuildID.js | Test failed or timed out, will retry.
TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js | running test ...
TEST-INFO | test_TelemetryPing_idle.js | Test failed or timed out, will retry.
TEST-INFO | test_ThirdPartyCookieProbe.js | Test failed or timed out, will retry.
TEST-INFO | test_nsITelemetry.js | Test failed or timed out, will retry.
TEST-INFO | test_TelemetrySendOldPings.js | Test failed or timed out, will retry.
TEST-INFO | test_TelemetryStopwatch.js | Test failed or timed out, will retry.
Retrying tests that failed when run in parallel.
TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryLockCount.js | running test ...
INFO | Following exceptions were raised:
Traceback (most recent call last):
  File "/Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py", line 145, in run
    self.run_test()
  File "/Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py", line 604, in run_test
    stdout=self.pStdout, stderr=self.pStderr, env=self.env, cwd=test_dir)
  File "/Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py", line 224, in launchProcess
    env=env, cwd=cwd)
  File "/Users/mikeconley/Projects/comm-central/mozilla/python/psutil/psutil/__init__.py", line 855, in __init__
    self.__subproc = subprocess.Popen(*args, **kwargs)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1308, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Traceback (most recent call last):
  File "/Users/mikeconley/Projects/comm-central/mozilla/config/pythonpath.py", line 56, in <module>
    main(sys.argv[1:])
  File "/Users/mikeconley/Projects/comm-central/mozilla/config/pythonpath.py", line 48, in main
    execfile(script, frozenglobals)
  File "/Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py", line 1594, in <module>
    main()
  File "/Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py", line 1590, in main
    if not xpcsh.runTests(args[0], testdirs=args[1:], **options.__dict__):
  File "/Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py", line 1475, in runTests
    raise exceptions[0]
OSError: [Errno 2] No such file or directory
make[1]: *** [xpcshell-tests] Error 1
make: *** [xpcshell-tests] Error 2
Mikes-MacBook-Pro-2:objdir-tb mikeconley$


Any ideas how I can try running these tests locally?
Flags: needinfo?(mbanner)
Assignee

Comment 10

6 years ago
make xpcshell-tests TEST_PATH=toolkit/components/telemetry/tests is working for me here.

I did have to change my mozconfig file to get it to build:

mk_add_options MOZ_OBJDIR=/Users/moztest/comm/main/tb

(using non-relative dir)
Flags: needinfo?(mbanner)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
Assignee

Comment 14

6 years ago
Ok, I've worked this out. This fails because Thunderbird doesn't define:

MOZ_TELEMETRY_REPORTING

This means that telemetry doesn't really get enabled, and possible handle any data during the tests. I think it also means that we're not sending information generally though I'm struggling to confirm that at the moment.
Assignee

Comment 15

6 years ago
So the configuration variable was added ages ago, but bug 722240 actually introduced reliance on it for sending telemetry reports in Gecko 13.... so we've probably not been actually sending reports for quite a while :-(

This patch adds the configuration variable, and will get the tests passing, and we'll be sending results again.

It appears the dashboard doesn't support other apps currently, but I've filed an issue on it:

https://github.com/mozilla/telemetry-dashboard/issues/16

There's a try run with this patch here (windows disabled due to other issues):

https://tbpl.mozilla.org/?tree=Thunderbird-Try
Attachment #8354561 - Flags: review?(mconley)
Comment hidden (Legacy TBPL/Treeherder Robot)
Assignee

Updated

6 years ago
Assignee: mconley → mbanner
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment on attachment 8354561 [details] [diff] [review]
Enable Telemetry Reporting

Review of attachment 8354561 [details] [diff] [review]:
-----------------------------------------------------------------

Yes, this does the job! Thanks!
Attachment #8354561 - Flags: review?(mconley) → review+
Also, I figured out the XPCshell test issue - I had to alter my .mozconfig to place the objdir as a sibling of the srcdir (as opposed to being a child of the srcdir).
Assignee

Comment 20

6 years ago
https://hg.mozilla.org/comm-central/rev/edb466c19b8b
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 29.0
Comment hidden (Legacy TBPL/Treeherder Robot)
Seems we're still hitting this on Windows on c-c tip?
Comment hidden (Legacy TBPL/Treeherder Robot)
Assignee

Comment 24

6 years ago
(In reply to Ryan VanderMeulen [:RyanVM UTC-5] from comment #22)
> Seems we're still hitting this on Windows on c-c tip?

I suspect these were old builds that hadn't been clobbered. Seems fine now.
Assignee

Comment 25

6 years ago
Comment on attachment 8354561 [details] [diff] [review]
Enable Telemetry Reporting

[Approval Request Comment]
Regression caused by (bug #): 
User impact if declined: 
Testing completed (on c-c, etc.): 
Risk to taking this patch (and alternatives if risky):

[Triage Comment]

[Triage Comment]
Taking onto aurora and beta as bug 863872 is also there. Still thinking about esr.
Attachment #8354561 - Flags: approval-comm-esr24?
Attachment #8354561 - Flags: approval-comm-beta+
Attachment #8354561 - Flags: approval-comm-aurora+
Assignee

Comment 27

6 years ago
Comment on attachment 8354561 [details] [diff] [review]
Enable Telemetry Reporting

Decided not to take this for esr, we'll pick it up in the next ESR cycle.
Attachment #8354561 - Flags: approval-comm-esr24? → approval-comm-esr24-
Comment hidden (Legacy TBPL/Treeherder Robot)
Comment hidden (Legacy TBPL/Treeherder Robot)
You need to log in before you can comment on or make changes to this bug.