Closed Bug 929985 Opened 8 years ago Closed 8 years ago

Reduce aggregate batch transaction stress test iterations in TestTXMgr.cpp for Android and ASAN

Categories

(Core :: DOM: Editor, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
mozilla28

People

(Reporter: dminor, Assigned: dminor)

References

Details

Attachments

(1 file, 1 obsolete file)

Based upon comments in bug 919595, it looks the solution to fixing TestTXMgr on Android and Linux ASAN is to rewrite the test.

I think it makes sense to stop building and packaging this test for those platforms now.
What things does this test currently block?  I'm fine with not building it on those platforms if it blocks something.
I'm working to get cpp unittests running from the test package. They are currently running on Cedar. This test fails consistently on both Android and Linux ASAN builds.

The alternative to disabling building this test would be to add them to a manifest file so that it is not run, but it seems wasteful to build and package it if there is no intention to fix it in its current form.
Try run for Android and ASAN is here: https://tbpl.mozilla.org/?tree=Try&rev=cfa927669cc0

Try run for remaining platforms is here: https://tbpl.mozilla.org/?tree=Try&rev=feff802debe1
Assignee: nobody → dminor
Status: NEW → ASSIGNED
Attachment #821686 - Flags: review?(ted)
Comment on attachment 821686 [details] [diff] [review]
Patch to disable test for Android and ASAN builds

Review of attachment 821686 [details] [diff] [review]:
-----------------------------------------------------------------

::: editor/txmgr/tests/moz.build
@@ +4,5 @@
>  # License, v. 2.0. If a copy of the MPL was not distributed with this
>  # file, You can obtain one at http://mozilla.org/MPL/2.0/.
>  
> +# Bug 919595
> +if CONFIG['MOZ_WIDGET_TOOLKIT'] != 'android' and CONFIG['MOZ_ASAN'] == '':

I think you can just say "not CONFIG['MOZ_ASAN']". I'm not sure if we have a moz.build style guide, but that's definitely more Pythonic.
Attachment #821686 - Flags: review?(ted) → review+
Comment on attachment 821686 [details] [diff] [review]
Patch to disable test for Android and ASAN builds

Actually, I changed my mind.  Before taking this patch, I'd like somebody to at least provide a stack trace or some debugging information on why this test fails.  I thought that I've seen that elsewhere but now I realize I haven't.  I don't want us to just mask a real bug here (the ASAN failure is specially worrying.)
Attachment #821686 - Flags: review-
Here is a log for the ASAN build. It appears to fail in "Aggregate Batch Transaction Stress Test". I tried reproducing this failure on my local system both from a local ASAN build and from the test package build without luck.

Before I request a test system from releng, I'll see if I can reproduce the Android failure locally to see if it will shed any light on the problem.

05:28:40     INFO -  cppunittests INFO | Running test TestTXMgr
05:28:40     INFO -  Running nsITransactionManager tests...
05:28:40     INFO -  -----------------------------------------------------
05:28:40     INFO -  - Begin Simple Transaction Test:
05:28:40     INFO -  -----------------------------------------------------
05:28:40     INFO -  TEST-PASS | Create transaction manager instance
05:28:40     INFO -  TEST-PASS | Call DoTransaction() with null transaction
05:28:40     INFO -  TEST-PASS | Call UndoTransaction() with empty undo stack
05:28:40     INFO -  TEST-PASS | Call RedoTransaction() with empty redo stack
05:28:40     INFO -  TEST-PASS | Call SetMaxTransactionCount(-1) with empty undo and redo stacks
05:28:40     INFO -  TEST-PASS | Call SetMaxTransactionCount(0) with empty undo and redo stacks
05:28:40     INFO -  TEST-PASS | Call SetMaxTransactionCount(10) with empty undo and redo stacks
05:28:40     INFO -  TEST-PASS | Call Clear() with empty undo and redo stack
05:28:40     INFO -  TEST-PASS | Call GetNumberOfUndoItems() with empty undo stack
05:28:40     INFO -  TEST-PASS | Call GetNumberOfRedoItems() with empty redo stack
05:28:40     INFO -  TEST-PASS | Call PeekUndoStack() with empty undo stack
05:28:40     INFO -  TEST-PASS | Call PeekRedoStack() with empty undo stack
05:28:40     INFO -  TEST-PASS | Call AddListener() with null listener
05:28:40     INFO -  TEST-PASS | Call RemoveListener() with null listener
05:28:40     INFO -  TEST-PASS | Test coalescing of transactions
05:28:40     INFO -  TEST-PASS | Execute 20 transactions
05:28:40     INFO -  TEST-PASS | Execute 20 transient transactions
05:28:40     INFO -  TEST-PASS | Undo 4 transactions
05:28:40     INFO -  TEST-PASS | Redo 2 transactions
05:28:40     INFO -  TEST-PASS | Check if new transactions prune the redo stack
05:28:40     INFO -  TEST-PASS | Undo 4 transactions then clear the undo and redo stacks
05:28:40     INFO -  TEST-PASS | Execute 5 transactions
05:28:40     INFO -  TEST-PASS | Test transaction DoTransaction() error
05:28:40     INFO -  TEST-PASS | Test transaction UndoTransaction() error
05:28:40     INFO -  TEST-PASS | Test transaction RedoTransaction() error
05:28:40     INFO -  TEST-PASS | Test max transaction count of zero
05:28:40     INFO -  TEST-PASS | Test SetMaxTransactionCount() greater than num stack items
05:28:40     INFO -  TEST-PASS | Test SetMaxTransactionCount() pruning undo stack
05:28:40     INFO -  TEST-PASS | Test SetMaxTransactionCount() pruning redo stack
05:28:40     INFO -  TEST-PASS | Release the transaction manager
05:28:40     INFO -  TEST-PASS | Number of transactions created and destroyed match
05:28:40     INFO -  TEST-PASS | 131 transactions processed during quick test
05:28:40     INFO -  -----------------------------------------------------
05:28:40     INFO -  - Begin Aggregate Transaction Test:
05:28:40     INFO -  -----------------------------------------------------
05:28:40     INFO -  TEST-PASS | Create transaction manager instance
05:28:40     INFO -  TEST-PASS | Call DoTransaction() with null transaction
05:28:40     INFO -  TEST-PASS | Call UndoTransaction() with empty undo stack
05:28:40     INFO -  TEST-PASS | Call RedoTransaction() with empty redo stack
05:28:40     INFO -  TEST-PASS | Call SetMaxTransactionCount(-1) with empty undo and redo stacks
05:28:40     INFO -  TEST-PASS | Call SetMaxTransactionCount(0) with empty undo and redo stacks
05:28:40     INFO -  TEST-PASS | Call SetMaxTransactionCount(10) with empty undo and redo stacks
05:28:40     INFO -  TEST-PASS | Call Clear() with empty undo and redo stack
05:28:40     INFO -  TEST-PASS | Call GetNumberOfUndoItems() with empty undo stack
05:28:40     INFO -  TEST-PASS | Call GetNumberOfRedoItems() with empty redo stack
05:28:40     INFO -  TEST-PASS | Call PeekUndoStack() with empty undo stack
05:28:40     INFO -  TEST-PASS | Call PeekRedoStack() with empty undo stack
05:28:40     INFO -  TEST-PASS | Call AddListener() with null listener
05:28:40     INFO -  TEST-PASS | Call RemoveListener() with null listener
05:28:40     INFO -  TEST-PASS | Test coalescing of transactions
05:28:40     INFO -  TEST-PASS | Execute 20 transactions
05:28:40     INFO -  TEST-PASS | Execute 20 transient transactions
05:28:40     INFO -  TEST-PASS | Undo 4 transactions
05:28:40     INFO -  TEST-PASS | Redo 2 transactions
05:28:40     INFO -  TEST-PASS | Check if new transactions prune the redo stack
05:28:40     INFO -  TEST-PASS | Undo 4 transactions then clear the undo and redo stacks
05:28:40     INFO -  TEST-PASS | Execute 5 transactions
05:28:40     INFO -  TEST-PASS | Test transaction DoTransaction() error
05:28:40     INFO -  TEST-PASS | Test transaction UndoTransaction() error
05:28:40     INFO -  TEST-PASS | Test transaction RedoTransaction() error
05:28:40     INFO -  TEST-PASS | Test max transaction count of zero
05:28:40     INFO -  TEST-PASS | Test SetMaxTransactionCount() greater than num stack items
05:28:40     INFO -  TEST-PASS | Test SetMaxTransactionCount() pruning undo stack
05:28:40     INFO -  TEST-PASS | Test SetMaxTransactionCount() pruning redo stack
05:28:40     INFO -  TEST-PASS | Release the transaction manager
05:28:40     INFO -  TEST-PASS | Number of transactions created and destroyed match
05:28:40     INFO -  TEST-PASS | 913 transactions processed during quick test
05:28:40     INFO -  -----------------------------------------------------
05:28:40     INFO -  - Begin Batch Transaction Test:
05:28:40     INFO -  -----------------------------------------------------
05:28:40     INFO -  TEST-PASS | Create transaction manager instance
05:28:40     INFO -  TEST-PASS | Test unbalanced EndBatch(false) with empty undo stack
05:28:40     INFO -  TEST-PASS | Test empty batch
05:28:40     INFO -  TEST-PASS | Execute 20 batched transactions
05:28:40     INFO -  TEST-PASS | Execute 20 batched transient transactions
05:28:40     INFO -  TEST-PASS | Test nested batched transactions
05:28:40     INFO -  TEST-PASS | Undo 2 batch transactions
05:28:40     INFO -  TEST-PASS | Redo 2 batch transactions
05:28:40     INFO -  TEST-PASS | Undo a batched transaction that was redone
05:28:40     INFO -  TEST-PASS | Test effect of unbalanced EndBatch(false) on undo and redo stacks
05:28:40     INFO -  TEST-PASS | Test effect of empty batch on undo and redo stacks
05:28:40     INFO -  TEST-PASS | Check if new batched transactions prune the redo stack
05:28:40     INFO -  TEST-PASS | Call undo
05:28:40     INFO -  TEST-PASS | Test transaction DoTransaction() error
05:28:40     INFO -  TEST-PASS | Test transaction UndoTransaction() error
05:28:40     INFO -  TEST-PASS | Test transaction RedoTransaction() error
05:28:40     INFO -  TEST-PASS | Test max transaction count of zero
05:28:40     INFO -  TEST-PASS | Release the transaction manager
05:28:40     INFO -  TEST-PASS | Number of transactions created and destroyed match
05:28:40     INFO -  TEST-PASS | 107 transactions processed during quick batch test
05:28:40     INFO -  -----------------------------------------------------
05:28:40     INFO -  - Begin Batch Aggregate Transaction Test:
05:28:40     INFO -  -----------------------------------------------------
05:28:40     INFO -  TEST-PASS | Create transaction manager instance
05:28:40     INFO -  TEST-PASS | Test unbalanced EndBatch(false) with empty undo stack
05:28:40     INFO -  TEST-PASS | Test empty batch
05:28:40     INFO -  TEST-PASS | Execute 20 batched transactions
05:28:40     INFO -  TEST-PASS | Execute 20 batched transient transactions
05:28:40     INFO -  TEST-PASS | Test nested batched transactions
05:28:40     INFO -  TEST-PASS | Undo 2 batch transactions
05:28:40     INFO -  TEST-PASS | Redo 2 batch transactions
05:28:40     INFO -  TEST-PASS | Undo a batched transaction that was redone
05:28:40     INFO -  TEST-PASS | Test effect of unbalanced EndBatch(false) on undo and redo stacks
05:28:40     INFO -  TEST-PASS | Test effect of empty batch on undo and redo stacks
05:28:40     INFO -  TEST-PASS | Check if new batched transactions prune the redo stack
05:28:40     INFO -  TEST-PASS | Call undo
05:28:40     INFO -  TEST-PASS | Test transaction DoTransaction() error
05:28:40     INFO -  TEST-PASS | Test transaction UndoTransaction() error
05:28:40     INFO -  TEST-PASS | Test transaction RedoTransaction() error
05:28:40     INFO -  TEST-PASS | Test max transaction count of zero
05:28:40     INFO -  TEST-PASS | Release the transaction manager
05:28:40     INFO -  TEST-PASS | Number of transactions created and destroyed match
05:28:40     INFO -  TEST-PASS | 745 transactions processed during quick batch test
05:28:40     INFO -  -----------------------------------------------------
05:28:40     INFO -  - Simple Transaction Stress Test:
05:28:40     INFO -  -----------------------------------------------------
05:28:44     INFO -  Stress test of 1500 iterations (may take a while) ... 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 passed
05:28:44     INFO -  TEST-PASS | 1125750 transactions processed during stress test
05:28:44     INFO -  -----------------------------------------------------
05:28:44     INFO -  - Aggregate Transaction Stress Test:
05:28:44     INFO -  -----------------------------------------------------
05:28:57     INFO -  Stress test of 500 iterations (may take a while) ... 100 200 300 400 500 passed
05:28:57     INFO -  TEST-PASS | 2630250 transactions processed during stress test
05:28:57     INFO -  -----------------------------------------------------
05:28:57     INFO -  - Aggregate Batch Transaction Stress Test:
05:28:57     INFO -  -----------------------------------------------------
05:29:08     INFO -  Stress test of 500 iterations (may take a while) ...
05:29:08     INFO -  cppunittests TEST-UNEXPECTED-FAIL | TestTXMgr | test failed with return code 9
Return code 9 means it was SIGKILLed, which probably just means it's being OOMkilled by the OS.
(In reply to comment #7)
> Return code 9 means it was SIGKILLed, which probably just means it's being
> OOMkilled by the OS.

The OS being Linux?  :-)
I ran this on my pandaboard this morning and it is being killed (likely OOM) while running the aggregation_batch_stress_test() case.
(In reply to comment #9)
> I ran this on my pandaboard this morning and it is being killed (likely OOM)
> while running the aggregation_batch_stress_test() case.

What about Linux ASAN builds?  That should not happen on Linux.
I can't reproduce the Linux ASAN test failure locally, so I've requested that the cpp unittests from test package be enabled on try (bug 931926). That seems to me to be the likeliest way to reproduce the issue.
Quick testing shows that 100 works and 250 fails (on Android). Once I have a way of duplicating the Linux ASAN failure I'll try it there as well. If so, I assume changing this for the two platforms with suitable #ifdefs would be preferable to disabling these completely.
Depends on: 934454
It appears 100 is a safe value for both Android and ASAN. I'll prepare a patch.
Summary: Stop building TestTXMgr.cpp for Android and Linux ASAN → Reduce aggregate batch transaction stress test iterations in TestTXMgr.cpp for Android and Linux ASAN
Attachment #821686 - Attachment is obsolete: true
Attachment #828794 - Flags: review?(ehsan)
Duplicate of this bug: 919595
Switch from Linux ASAN to all ASAN builds, as that is what the patch does.
Summary: Reduce aggregate batch transaction stress test iterations in TestTXMgr.cpp for Android and Linux ASAN → Reduce aggregate batch transaction stress test iterations in TestTXMgr.cpp for Android and ASAN
Comment on attachment 828794 [details] [diff] [review]
Switch from 500 to 100 runs on Android and ASAN builds

Review of attachment 828794 [details] [diff] [review]:
-----------------------------------------------------------------

::: editor/txmgr/tests/TestTXMgr.cpp
@@ +4579,5 @@
>    10
>  #else
>    //
>    // 500 iterations sends 2,630,250 transactions through the system!!
>    //

Please move this comment down to where the value 500 is used.
Attachment #828794 - Flags: review?(ehsan) → review+
https://hg.mozilla.org/mozilla-central/rev/780b232ec465
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla28
You need to log in before you can comment on or make changes to this bug.