Closed Bug 794070 Opened 8 years ago Closed 8 years ago

Intermittent TEST-UNEXPECTED-FAIL | testHistoryTab | First row has Today header - Yesterday

Categories

(Testing :: General, defect)

x86
Android
defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED
mozilla18

People

(Reporter: gbrown, Assigned: gbrown)

Details

Attachments

(1 file)

This can happen around midnight: If the day changes between when the test starts and ends.

FIRE PROC: '"MOZ_CRASHREPORTER=1,XPCOM_DEBUG_BREAK=stack,MOZ_CRASHREPORTER_NO_REPORT=1,NO_EM_RESTART=1,MOZ_PROCESS_LOG=/tmp/tmpspJ12Ypidlog,XPCOM_MEM_BLOAT_LOG=/mnt/sdcard/tests/profile/runtests_leaks.log" am instrument -w -e deviceroot /mnt/sdcard/tests -e class org.mozilla.fennec.tests.testHistoryTab org.mozilla.roboexample.test/org.mozilla.fennec.FennecInstrumentationTestRunner'
Robocop derived process name: org.mozilla.fennec
INFO | automation.py | Application pid: 0
0 INFO SimpleTest START
1 INFO TEST-START | testHistoryTab
2 INFO TEST-PASS | testHistoryTab | Awesomebar URL typed properly - http://mochi.test:8888/tests/robocop/robocop_big_link.html should equal http://mochi.test:8888/tests/robocop/robocop_big_link.html
3 INFO TEST-PASS | testHistoryTab | Awesomebar URL typed properly - http://mochi.test:8888/tests/robocop/robocop_blank_01.html should equal http://mochi.test:8888/tests/robocop/robocop_blank_01.html
4 INFO TEST-PASS | testHistoryTab | bookmark added sucessfully - true should equal true
5 INFO TEST-PASS | testHistoryTab | checking that history list exists - android.widget.ExpandableListView@485b8480
6 INFO TEST-PASS | testHistoryTab | history list has 3 children - 3 should equal 3
7 INFO TEST-PASS | testHistoryTab | TextView is filled in - Big Link
8 INFO TEST-PASS | testHistoryTab | TextView is filled in - http://mochi.test:8888/tests/robocop/robocop_big_link.html
9 INFO TEST-PASS | testHistoryTab | Correct number of ImageViews visible - 1 should equal 1
10 INFO TEST-PASS | testHistoryTab | TextView is filled in - Browser Blank Page 01
11 INFO TEST-PASS | testHistoryTab | TextView is filled in - http://mochi.test:8888/tests/robocop/robocop_blank_01.html
12 INFO TEST-PASS | testHistoryTab | Correct number of ImageViews visible - 2 should equal 2
13 INFO TEST-UNEXPECTED-FAIL | testHistoryTab | First row has Today header - Yesterday
Exception caught during test!
junit.framework.AssertionFailedError: 13 INFO TEST-UNEXPECTED-FAIL | testHistoryTab | First row has Today header - Yesterday
	at junit.framework.Assert.fail(Assert.java:47)
	at org.mozilla.fennec.FennecMochitestAssert._logMochitestResult(FennecMochitestAssert.java:107)
	at org.mozilla.fennec.FennecMochitestAssert.ok(FennecMochitestAssert.java:136)
	at org.mozilla.fennec.tests.testHistoryTab.testList(testHistoryTab.java:92)
	at org.mozilla.fennec.tests.testHistoryTab.testHistoryTab(testHistoryTab.java:60)
	at java.lang.reflect.Method.invokeNative(Native Method)
	at java.lang.reflect.Method.invoke(Method.java:521)
	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204)
	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194)
	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186)
	at org.mozilla.fennec.tests.BaseTest.runTest(BaseTest.java:95)
	at junit.framework.TestCase.runBare(TestCase.java:127)
	at junit.framework.TestResult$1.protect(TestResult.java:106)
	at junit.framework.TestResult.runProtected(TestResult.java:124)
	at junit.framework.TestResult.run(TestResult.java:109)
	at junit.framework.TestCase.run(TestCase.java:118)
	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:520)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447)
14 INFO TEST-UNEXPECTED-FAIL | testHistoryTab | Exception caught - junit.framework.AssertionFailedError: 13 INFO TEST-UNEXPECTED-FAIL | testHistoryTab | First row has Today header - Yesterday
15 INFO TEST-END | testHistoryTab | finished in 40962ms
16 INFO TEST-START | Shutdown
17 INFO Passed: 11
18 INFO Failed: 2
19 INFO Todo: 0
20 INFO SimpleTest FINISHED
INFO | automation.py | Application ran for: 0:00:44.956662
INFO | automation.py | Reading PID log: /tmp/tmpspJ12Ypidlog
getting files in '/mnt/sdcard/tests/profile/minidumps/'
WARNING | automationutils.processLeakLog() | refcount logging is off, so leaks can't be detected!
I checked the other robocop tests for similar vulnerabilities; I think only testHistoryTab is affected.
If the test starts "near midnight" (grossly approximated as any time satisfying 23:xx:xx), allow for a history heading of "Yesterday" instead of "Today".
Attachment #664649 - Flags: review?(jmaher)
Comment on attachment 664649 [details] [diff] [review]
allow 'Yesterday' instead of 'Today' if test run at 23:xx

Review of attachment 664649 [details] [diff] [review]:
-----------------------------------------------------------------

oh wow.  I can imagine future tests hitting this problem, so lets be careful on reviews.
Attachment #664649 - Flags: review?(jmaher) → review+
https://hg.mozilla.org/mozilla-central/rev/15cd63bd65d5
Status: NEW → RESOLVED
Closed: 8 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → mozilla18
You need to log in before you can comment on or make changes to this bug.