1592860 - [META] Hidden issues of xpcshell test of C-C TB (hidden because they don't get printed if the test succeeds).

Status: NEW

(Thunderbird :: Testing Infrastructure, defect)

Description

[ISHIKAWA, Chiaki]

C-C TB xpcshell-tests only print out stdout/stderr output when the test fails.
This means many errors/warnings that don't somehow affect the outcome of each xcpshell tests do not get proper attention.

By passing |--verbose| option to |mach xpcshell-tests|, we can force the listing of stdout/stderr output.

This output was necessary for me to check for memory-related issues by running C-C TB (full debug version) during |mach xpcshell-tests --verbose|.
This is because valgrind warnings go into stderr (?) output.

So I ran |xpcshell-tests --verbose| under valgrind to find out what warnings if any from valgrind appears.

There WERE many valgrind warnings pointing to uninitialized memory access.
Not only that, I was staggered to find so many JavaScript strict warnings and JavaScript warnings in the dumped listing.

So I create this meta entry to keep track of the issues found during the execution of |mach xpcshell-tests --verbose| under valgrind on a local PC that runs linux 64-bit.
I believe most of the errors found are universal across different architectures, though.
That is why I set platform and OS to all each.

I am going to file initially the general list of issues:

• java strict warnings
• java warnings
• valgrind violations
  found in the test of M-C portion (not tested extensively due to the large volume of output produced. 110+ MB )
  found in the test of C-C portion.

In this bugzilla, I am going to upload the summary of errors/warnings found.
Each error/warning shall be addressed by a separate bugzilla entry.

Comment 1

[ISHIKAWA, Chiaki]

Attachment: 1st half of verbose log xpcshell-tests run for C-C portion of the code.

Comment 2

[ISHIKAWA, Chiaki]

Attachment: 2nd half of verbose log xpcshell-tests run for C-C portion of the code.

Comment 3

[ISHIKAWA, Chiaki]

listing the "JavaScript Strict Warning:" messages from C-C portion only:

Created by
grep -i "javascript strict warning" xaa xab | cut --delim=" " --fields=4- | sort | uniq -c | sort -nr

(xaa xab are the first and second half of the C-C xpcshell-test log.)

  7 JavaScript strict warning: resource://gre/modules/addons/XPIProvider.jsm, line 600: ReferenceError: reference to undefined property "version"
  7 JavaScript strict warning: resource://gre/modules/addons/XPIProvider.jsm, line 498: ReferenceError: reference to undefined property "currentModifiedTime"
  7 JavaScript strict warning: resource://gre/modules/addons/XPIInstall.jsm, line 551: ReferenceError: reference to undefined property "description"
  4 JavaScript strict warning: resource://gre/modules/Extension.jsm, line 2411: ReferenceError: reference to undefined property "cleanupFile"
  4 JavaScript strict warning: resource://gre/modules/AddonManager.jsm, line 4602: ReferenceError: reference to undefined property "source"
  2 JavaScript strict warning: resource://gre/modules/mozIntl.jsm, line 834: ReferenceError: reference to undefined property 0
  2 JavaScript strict warning: resource://gre/modules/Schemas.jsm, line 2626: ReferenceError: reference to undefined property "reference"
  2 JavaScript strict warning: resource://gre/modules/AddonManager.jsm, line 4671: ReferenceError: reference to undefined property "source"
  2 JavaScript strict warning: resource:///modules/xmpp-xml.jsm, line 435: ReferenceError: reference to undefined property "_node"
  2 JavaScript strict warning: resource:///components/logger.js, line 333: ReferenceError: reference to undefined property "incoming"
  1 JavaScript strict warning: resource://xpcshell-data/BootstrapMonitor.jsm, line 39: ReferenceError: reference to undefined property "install"
  1 JavaScript strict warning: resource://gre/modules/ExtensionSettingsStore.jsm, line 76: ReferenceError: reference to undefined property "version"
  1 JavaScript strict warning: resource://gre/modules/ExtensionParent.jsm, line 599: ReferenceError: reference to undefined property "envType"
  1 JavaScript strict warning: resource://gre/modules/ExtensionContent.jsm, line 602: ReferenceError: reference to undefined property "script"
  1 JavaScript strict warning: resource:///modules/jsaccount/JSAccountUtils.jsm, line 211: ReferenceError: reference to undefined property Symbol.toPrimitive
  1 JavaScript strict warning: /NEW-SSD/moz-obj-dir/objdir-tb3/_tests/xpcshell/comm/common/test/xpcshell/test_bootstrap.js, line 169: ReferenceError: reference to undefined property "newVersion"
  1 JavaScript strict warning: /NEW-SSD/moz-obj-dir/objdir-tb3/_tests/xpcshell/comm/common/test/xpcshell/test_bootstrap.js, line 164: ReferenceError: reference to undefined property "oldVersion"
  1 JavaScript strict warning: /NEW-SSD/moz-obj-dir/objdir-tb3/_tests/xpcshell/comm/chat/protocols/xmpp/test/test_xmppXml.js, line 79: ReferenceError: reference to undefined property "namespace"
  1 JavaScript strict warning: /NEW-SSD/moz-obj-dir/objdir-tb3/_tests/xpcshell/comm/chat/protocols/xmpp/test/test_parseJidAndNormalization.js, line 70: ReferenceError: reference to undefined property "resource"
  1 JavaScript strict warning: /NEW-SSD/moz-obj-dir/objdir-tb3/_tests/xpcshell/comm/chat/components/src/test/test_commands.js, line 214: ReferenceError: reference to undefined property "conv"

I see worse in the full test log (i.e., including xpcshell-tests in the M-C tree).

Comment 4

[ISHIKAWA, Chiaki]

Listing of "JavaScript Warning" messages from C-C portion only.

Created by
grep -i "javascript warning" xaa xab | cut --delim=" " --fields=4- | sort | uniq -c | sort -nr &
(xaa xab are the first and second half of the C-C xpcshell-test log.)

 12 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "Ignoring unrecognized chrome manifest directive 'interfaces'." {file: "file:///NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/components/httpd.manifest" line: 1}]"
 12 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "Ignoring unrecognized chrome manifest directive 'interfaces'." {file: "file:///NEW-SSD/NREF-COMM-CENTRAL/mozilla/netwerk/test/httpserver/httpd.manifest" line: 1}]"
  7 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "version"" {file: "resource://gre/modules/addons/XPIProvider.jsm" line: 600}]"
  7 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "description"" {file: "resource://gre/modules/addons/XPIInstall.jsm" line: 551}]"
  7 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "currentModifiedTime"" {file: "resource://gre/modules/addons/XPIProvider.jsm" line: 498}]"
  4 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "source"" {file: "resource://gre/modules/AddonManager.jsm" line: 4602}]"
  4 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "cleanupFile"" {file: "resource://gre/modules/Extension.jsm" line: 2411}]"
  2 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property 0" {file: "resource://gre/modules/mozIntl.jsm" line: 834}]"
  2 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "source"" {file: "resource://gre/modules/AddonManager.jsm" line: 4671}]"
  2 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "reference"" {file: "resource://gre/modules/Schemas.jsm" line: 2626}]"
  2 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "incoming"" {file: "resource:///components/logger.js" line: 333}]"
  2 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "_node"" {file: "resource:///modules/xmpp-xml.jsm" line: 435}]"
  1 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property Symbol.toPrimitive" {file: "resource:///modules/jsaccount/JSAccountUtils.jsm" line: 211}]"
  1 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "version"" {file: "resource://gre/modules/ExtensionSettingsStore.jsm" line: 76}]"
  1 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "script"" {file: "resource://gre/modules/ExtensionContent.jsm" line: 602}]"
  1 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "resource"" {file: "/NEW-SSD/moz-obj-dir/objdir-tb3/_tests/xpcshell/comm/chat/protocols/xmpp/test/test_parseJidAndNormalization.js" line: 70}]"
  1 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "oldVersion"" {file: "/NEW-SSD/moz-obj-dir/objdir-tb3/_tests/xpcshell/comm/common/test/xpcshell/test_bootstrap.js" line: 164}]"
  1 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "newVersion"" {file: "/NEW-SSD/moz-obj-dir/objdir-tb3/_tests/xpcshell/comm/common/test/xpcshell/test_bootstrap.js" line: 169}]"
  1 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "namespace"" {file: "/NEW-SSD/moz-obj-dir/objdir-tb3/_tests/xpcshell/comm/chat/protocols/xmpp/test/test_xmppXml.js" line: 79}]"
  1 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "install"" {file: "resource://xpcshell-data/BootstrapMonitor.jsm" line: 39}]"
  1 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "envType"" {file: "resource://gre/modules/ExtensionParent.jsm" line: 599}]"
  1 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "conv"" {file: "/NEW-SSD/moz-obj-dir/objdir-tb3/_tests/xpcshell/comm/chat/components/src/test/test_commands.js" line: 214}]"

I see worse in the full test log (i.e., including xpcshell-tests in the M-C tree).

Comment 5

[ISHIKAWA, Chiaki]

For valgrind warnings, due to the size of the log and the time it takes to produce warnings [often the test needs longer timeout and I may not have covered the whole test giving it enough time], I only list one M-C portion warning and
the so far known warnings from C-C portions.

Firstly, a bug from M-C portion.

880:06.75 pid:31803 ==31803== Conditional jump or move depends on uninitialised value(s)
880:06.75 pid:31803 ==31803==    at 0x9584FB6: nsPluginTag::GetEnabledState(unsigned int*) (nsPluginTags.cpp:522)
880:06.75 pid:31803 ==31803==    by 0x958ADAA: nsPluginTag::IsActive() (nsPluginTags.cpp:564)
880:06.75 pid:31803 ==31803==    by 0x957C618: nsPluginHost::AddPluginTag(nsPluginTag*) (nsPluginHost.cpp:1931)
880:06.75 pid:31803 ==31803==    by 0x957A1DF: nsPluginHost::ScanPluginsDirectory(nsIFile*, bool, bool*) (nsPluginHost.cpp:2110)
880:06.75 pid:31803 ==31803==    by 0x957A5CC: nsPluginHost::ScanPluginsDirectoryList(nsISimpleEnumerator*, bool, bool*) (nsPluginHost.cpp:2151)
880:06.75 pid:31803 ==31803==    by 0x957A979: nsPluginHost::FindPlugins(bool, bool*) (nsPluginHost.cpp:2353)
880:06.75 pid:31803 ==31803==    by 0x957B171: nsPluginHost::LoadPlugins() (nsPluginHost.cpp:2220)
880:06.75 pid:31803 ==31803==    by 0x957B517: nsPluginHost::nsPluginHost() (nsPluginHost.cpp:408)
880:06.75 pid:31803 ==31803==    by 0x957B5A4: nsPluginHost::GetInst() (nsPluginHost.cpp:423)
880:06.75 pid:31803 ==31803==    by 0x69388D5: mozilla::xpcom::CreateInstanceImpl(mozilla::xpcom::ModuleID, nsISupports*, nsID const&, void**) [clone .part.0] (StaticComponents.cpp:10270)
880:06.75 pid:31803 ==31803==    by 0x69535B4: nsComponentManagerImpl::GetServiceLocked((anonymous namespace)::MutexLock&, (anonymous namespace)::EntryWrapper&, nsID const&, void**) (nsComponentManager.cpp:224)
   ...

I will file a separate bugzilla to discuss this and upload a patch there.

Comment 6

[ISHIKAWA, Chiaki]

Attachment: excerpt from logs: created by grep -i1 "Conditional jump" log1093-xpcshell-memcheck.txt

So far known valgrind warnings from xpcshell-tests in C-C portion.

This was created by the following command line:
grep -i1 "Conditional jump" log1093-xpcshell-memcheck.txt

Comment 7

[ISHIKAWA, Chiaki]

I will file bugzilla entries that blocks this entry.

Comment 8

[ISHIKAWA, Chiaki]

Because the log for C-C only xpcshell-tests is already close to 20MB, I cannot easily upload it.

It would be great if somehow the xpcshell-tests of then current C-C xpcshell-test with --verbose flag is run, say, every week so that
it is easy to share the problems.

Comment 9

[ISHIKAWA, Chiaki]

My aim in filing this bugzilla is to reduce clutter in the log so that REAL BUGS/ISSUES can be found by inspecting the log dump. We want to eliminate frivolous messages.

(In reply to ISHIKAWA, Chiaki from comment #4)

Listing of "JavaScript Warning" messages from C-C portion only.

Created by
grep -i "javascript warning" xaa xab | cut --delim=" " --fields=4- | sort | uniq -c | sort -nr &
(xaa xab are the first and second half of the C-C xpcshell-test log.)

 12 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "Ignoring unrecognized chrome manifest directive 'interfaces'." {file: "file:///NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/components/httpd.manifest" line: 1}]"
 12 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "Ignoring unrecognized chrome manifest directive 'interfaces'." {file: "file:///NEW-SSD/NREF-COMM-CENTRAL/mozilla/netwerk/test/httpserver/httpd.manifest" line: 1}]"
  7 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "version"" {file: "resource://gre/modules/addons/XPIProvider.jsm" line: 600}]"
  7 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "description"" {file: "resource://gre/modules/addons/XPIInstall.jsm" line: 551}]"
  7 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "currentModifiedTime"" {file: "resource://gre/modules/addons/XPIProvider.jsm" line: 498}]"
  4 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "source"" {file: "resource://gre/modules/AddonManager.jsm" line: 4602}]"
  4 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property "cleanupFile"" {file: "resource://gre/modules/Extension.jsm" line: 2411}]"
  2 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "ReferenceError: reference to undefined property 0" {file: "resource://gre/modules/mozIntl.jsm" line: 834}]"

... 

Many of these and MORE in the full xpcshell-test (including M-C portion of the test) are related to references to undefined property.
It looks to me there are two exemplary causes.
case-1. A test creates a constant data structure, typically an array of similar data objects. Some of them have certain property defined while others don't. Later, the constant data structure is accessed and handled in a uniform manner by a function, and the function accesses the property which some data objects do not define.
While, we may argue that the approach is a bit slippery, I think the test author knows what he/she is doing and so we may simply
tide over the issue by checking whether the property is defined first and access it only when it is defined.
E.g. Rewriting variable.property to variable.hasOwnProperty("property") ? variable.property : undefined
|undefined| may be |null| depending on context.
This is awkward coding, but it works.

(Note: I REPEAT: The idea is to reduce clutter so that REAL BUGS/ISSUES can be found by inspecting the log dump. We want to eliminate frivolous messages.)

case-2: This is a bit problematic. The mozilla code often fails to check the sanity of input data construct and thus fails to create a
full data structure when the input data is ill-constructed.
As I looked at the code trying to eliminate the undefined references noticed above,
I realized some undefined references MAY be caused due to the nature of data read from external sources. The data may simply fails to define expected property OR the data is missing some info that lets the proper execution of program logic that would have define the property.
It is hard to tell.
HOWEVER, after eliminating all the references in C-C xpcshell-test portion by using the ugly remedy mentioned in case-1 above, I realize that
I probably should check
(a) if the C-C xpcshell test is properly configured to feed correct data to each test program, and
(b) whether C-C code checks the sanity of input data so that it won't misbehave.
So I reverted the ugly fixes for most of the cases.

I realized case-2 exists when I check the two strange messages of

12 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "Ignoring unrecognized chrome manifest directive 'interfaces'." {file: "file:///NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/components/httpd.manifest" line: 1}]"
12 "CONSOLE_MESSAGE: (warn) [JavaScript Warning: "Ignoring unrecognized chrome manifest directive 'interfaces'." {file: "file:///NEW-SSD/NREF-COMM-CENTRAL/mozilla/netwerk/test/httpserver/httpd.manifest" line: 1}]".

I will file a bugzilla (or two) about the couple of errors above.

Comment 10

[Wayne Mery (:wsmwk)]

Only one blocking bug remains.
Are more expected, or is it OK to close this?