Closed Bug 951318 Opened 12 years ago Closed 12 years ago

TEST-UNEXPECTED-FAIL | /test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js

Categories

(Thunderbird :: General, defect)

Product:
Thunderbird
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(thunderbird27 fixed, thunderbird28 fixed)

Status:
RESOLVED FIXED
Milestone:
Thunderbird 29.0
Tracking Flags:
Tracking Status
thunderbird27 --- fixed
thunderbird28 --- fixed

People

(Reporter: mkmelin, Assigned: standard8)

References

Details

(Keywords: intermittent-failure)

Attachments

(1 file)

Enable Telemetry Reporting
3.82 KB, patch
mconley
: review+
standard8
: approval-comm-aurora+
standard8
: approval-comm-beta+
standard8
: approval-comm-esr24-
Details | Diff | Splinter Review
All platforms. https://tbpl.mozilla.org/php/getParsedLog.php?id=32096380&tree=Thunderbird-Trunk&full=1 TEST-INFO | /builds/slave/test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js | running test ... TEST-UNEXPECTED-FAIL | /builds/slave/test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js | test failed (with xpcshell return code: 0), see following log: >>>>>>> TEST-INFO | (xpcshell/head.js) | test MAIN run_test pending (1) TEST-INFO | (xpcshell/head.js) | test run_next_test 0 pending (2) TEST-INFO | (xpcshell/head.js) | test MAIN run_test finished (2) TEST-INFO | (xpcshell/head.js) | running event loop TEST-INFO | /builds/slave/test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js | Starting test_expired_pings_are_deleted TEST-INFO | (xpcshell/head.js) | test test_expired_pings_are_deleted pending (2) TEST-INFO | (xpcshell/head.js) | test run_next_test 0 finished (2) TEST-UNEXPECTED-FAIL | /builds/slave/test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js | Found 5 unexpected saved pings. - See following stack: JS frame :: /builds/slave/test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js :: assertNotSaved :: line 160 JS frame :: /builds/slave/test/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js :: test_expired_pings_are_deleted :: line 229 JS frame :: resource://gre/modules/Task.jsm :: TaskImpl_run :: line 233 JS frame :: resource://gre/modules/Promise.jsm :: Handler.prototype.process :: line 767 JS frame :: resource://gre/modules/Promise.jsm :: this.PromiseWalker.walkerLoop :: line 531 native frame :: <unknown filename> :: <TOP_LEVEL> :: line 0 TEST-INFO | (xpcshell/head.js) | exiting test TEST-UNEXPECTED-FAIL | resource://gre/modules/commonjs/sdk/core/promise.js | Unexpected exception 2147500036 TEST-INFO | (xpcshell/head.js) | exiting test !!! error running onStopped callback: TypeError: callback is not a function <<<<<<< Node SPDY server shutting down ... INFO | Result summary: INFO | Passed: 2042 INFO | Failed: 1 INFO | Todo: 4 INFO | Retried: 2
mxr/hg say bug 863872 added this test. Normally with these issues, we've got some sort of preference that's differently set, however I don't see that here.
Blocks: 863872
Indeed - I'm the one who added this test. I'll look into this.
Assignee: nobody → mconley
So I'm having no luck running the telemetry xpcshell tests locally. Here's what I'm executing / getting: Mikes-MacBook-Pro-2:objdir-tb mikeconley$ make xpcshell-tests TEST_PATH=toolkit/components/telemetry/tests make -C mozilla xpcshell-tests Have you considered running xpcshell tests via |mach xpcshell-test|? mach is easier to use and has more features than make and it will eventually be the only way to run xpcshell tests. Please consider using mach today! /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_virtualenv/bin/python -u /Users/mikeconley/Projects/comm-central/mozilla/config/pythonpath.py \ -I./build \ -I/Users/mikeconley/Projects/comm-central/mozilla/build \ -I./_tests/mozbase/mozinfo \ /Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py \ --manifest=./_tests/xpcshell/xpcshell.ini \ --build-info-json=./mozinfo.json \ --no-logfiles \ --test-plugin-path='./dist/plugins' \ --tests-root-dir=/Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell \ --testing-modules-dir=/Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/modules \ --xunit-file=/Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/results.xml \ --xunit-suite-name=xpcshell \ --symbols-path=./dist/crashreporter-symbols \ --test-path='toolkit/components/telemetry/tests' \ /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/dist/bin/xpcshell INFO | Using at most 32 threads. TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryPing.js | running test ... TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryStopwatch.js | running test ... TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryLockCount.js | running test ... TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryLateWrites.js | running test ... TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_nsITelemetry.js | running test ... TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryPing_idle.js | running test ... TEST-INFO | Failed to remove directory: /var/folders/bq/n5qs5f650bl7kmj4h_fqsw2m0000gn/T/tmpZj9nQW. Waiting. TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_ThirdPartyCookieProbe.js | running test ... TEST-INFO | test_TelemetryLateWrites.js | Test failed or timed out, will retry. TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryPingBuildID.js | running test ... TEST-INFO | test_TelemetryLockCount.js | Test failed or timed out, will retry. TEST-INFO | test_TelemetryPing.js | Test failed or timed out, will retry. TEST-INFO | test_TelemetryPingBuildID.js | Test failed or timed out, will retry. TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetrySendOldPings.js | running test ... TEST-INFO | test_TelemetryPing_idle.js | Test failed or timed out, will retry. TEST-INFO | test_ThirdPartyCookieProbe.js | Test failed or timed out, will retry. TEST-INFO | test_nsITelemetry.js | Test failed or timed out, will retry. TEST-INFO | test_TelemetrySendOldPings.js | Test failed or timed out, will retry. TEST-INFO | test_TelemetryStopwatch.js | Test failed or timed out, will retry. Retrying tests that failed when run in parallel. TEST-INFO | /Users/mikeconley/Projects/comm-central/objdir-tb/mozilla/_tests/xpcshell/toolkit/components/telemetry/tests/unit/test_TelemetryLockCount.js | running test ... INFO | Following exceptions were raised: Traceback (most recent call last): File "/Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py", line 145, in run self.run_test() File "/Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py", line 604, in run_test stdout=self.pStdout, stderr=self.pStderr, env=self.env, cwd=test_dir) File "/Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py", line 224, in launchProcess env=env, cwd=cwd) File "/Users/mikeconley/Projects/comm-central/mozilla/python/psutil/psutil/__init__.py", line 855, in __init__ self.__subproc = subprocess.Popen(*args, **kwargs) File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1308, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory Traceback (most recent call last): File "/Users/mikeconley/Projects/comm-central/mozilla/config/pythonpath.py", line 56, in <module> main(sys.argv[1:]) File "/Users/mikeconley/Projects/comm-central/mozilla/config/pythonpath.py", line 48, in main execfile(script, frozenglobals) File "/Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py", line 1594, in <module> main() File "/Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py", line 1590, in main if not xpcsh.runTests(args[0], testdirs=args[1:], **options.__dict__): File "/Users/mikeconley/Projects/comm-central/mozilla/testing/xpcshell/runxpcshelltests.py", line 1475, in runTests raise exceptions[0] OSError: [Errno 2] No such file or directory make[1]: *** [xpcshell-tests] Error 1 make: *** [xpcshell-tests] Error 2 Mikes-MacBook-Pro-2:objdir-tb mikeconley$ Any ideas how I can try running these tests locally?
Flags: needinfo?(mbanner)
Depends on: 952786
make xpcshell-tests TEST_PATH=toolkit/components/telemetry/tests is working for me here. I did have to change my mozconfig file to get it to build: mk_add_options MOZ_OBJDIR=/Users/moztest/comm/main/tb (using non-relative dir)
Flags: needinfo?(mbanner)
Ok, I've worked this out. This fails because Thunderbird doesn't define: MOZ_TELEMETRY_REPORTING This means that telemetry doesn't really get enabled, and possible handle any data during the tests. I think it also means that we're not sending information generally though I'm struggling to confirm that at the moment.
So the configuration variable was added ages ago, but bug 722240 actually introduced reliance on it for sending telemetry reports in Gecko 13.... so we've probably not been actually sending reports for quite a while :-( This patch adds the configuration variable, and will get the tests passing, and we'll be sending results again. It appears the dashboard doesn't support other apps currently, but I've filed an issue on it: https://github.com/mozilla/telemetry-dashboard/issues/16 There's a try run with this patch here (windows disabled due to other issues): https://tbpl.mozilla.org/?tree=Thunderbird-Try
Attachment #8354561 - Flags: review?(mconley)
Assignee: mconley → mbanner
Comment on attachment 8354561 [details] [diff] [review] Enable Telemetry Reporting Review of attachment 8354561 [details] [diff] [review]: ----------------------------------------------------------------- Yes, this does the job! Thanks!
Attachment #8354561 - Flags: review?(mconley) → review+
Also, I figured out the XPCshell test issue - I had to alter my .mozconfig to place the objdir as a sibling of the srcdir (as opposed to being a child of the srcdir).
https://hg.mozilla.org/comm-central/rev/edb466c19b8b
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 29.0
Seems we're still hitting this on Windows on c-c tip?
(In reply to Ryan VanderMeulen [:RyanVM UTC-5] from comment #22) > Seems we're still hitting this on Windows on c-c tip? I suspect these were old builds that hadn't been clobbered. Seems fine now.
Comment on attachment 8354561 [details] [diff] [review] Enable Telemetry Reporting [Approval Request Comment] Regression caused by (bug #): User impact if declined: Testing completed (on c-c, etc.): Risk to taking this patch (and alternatives if risky): [Triage Comment] [Triage Comment] Taking onto aurora and beta as bug 863872 is also there. Still thinking about esr.
Attachment #8354561 - Flags: approval-comm-esr24?
Attachment #8354561 - Flags: approval-comm-beta+
Attachment #8354561 - Flags: approval-comm-aurora+
Comment on attachment 8354561 [details] [diff] [review] Enable Telemetry Reporting Decided not to take this for esr, we'll pick it up in the next ESR cycle.
Attachment #8354561 - Flags: approval-comm-esr24? → approval-comm-esr24-
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: