794070 - Intermittent TEST-UNEXPECTED-FAIL | testHistoryTab | First row has Today header - Yesterday

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Geoff Brown [:gbrown] (away Feb 17)]

This can happen around midnight: If the day changes between when the test starts and ends.

FIRE PROC: '"MOZ_CRASHREPORTER=1,XPCOM_DEBUG_BREAK=stack,MOZ_CRASHREPORTER_NO_REPORT=1,NO_EM_RESTART=1,MOZ_PROCESS_LOG=/tmp/tmpspJ12Ypidlog,XPCOM_MEM_BLOAT_LOG=/mnt/sdcard/tests/profile/runtests_leaks.log" am instrument -w -e deviceroot /mnt/sdcard/tests -e class org.mozilla.fennec.tests.testHistoryTab org.mozilla.roboexample.test/org.mozilla.fennec.FennecInstrumentationTestRunner'
Robocop derived process name: org.mozilla.fennec
INFO | automation.py | Application pid: 0
0 INFO SimpleTest START
1 INFO TEST-START | testHistoryTab
2 INFO TEST-PASS | testHistoryTab | Awesomebar URL typed properly - http://mochi.test:8888/tests/robocop/robocop_big_link.html should equal http://mochi.test:8888/tests/robocop/robocop_big_link.html
3 INFO TEST-PASS | testHistoryTab | Awesomebar URL typed properly - http://mochi.test:8888/tests/robocop/robocop_blank_01.html should equal http://mochi.test:8888/tests/robocop/robocop_blank_01.html
4 INFO TEST-PASS | testHistoryTab | bookmark added sucessfully - true should equal true
5 INFO TEST-PASS | testHistoryTab | checking that history list exists - android.widget.ExpandableListView@485b8480
6 INFO TEST-PASS | testHistoryTab | history list has 3 children - 3 should equal 3
7 INFO TEST-PASS | testHistoryTab | TextView is filled in - Big Link
8 INFO TEST-PASS | testHistoryTab | TextView is filled in - http://mochi.test:8888/tests/robocop/robocop_big_link.html
9 INFO TEST-PASS | testHistoryTab | Correct number of ImageViews visible - 1 should equal 1
10 INFO TEST-PASS | testHistoryTab | TextView is filled in - Browser Blank Page 01
11 INFO TEST-PASS | testHistoryTab | TextView is filled in - http://mochi.test:8888/tests/robocop/robocop_blank_01.html
12 INFO TEST-PASS | testHistoryTab | Correct number of ImageViews visible - 2 should equal 2
13 INFO TEST-UNEXPECTED-FAIL | testHistoryTab | First row has Today header - Yesterday
Exception caught during test!
junit.framework.AssertionFailedError: 13 INFO TEST-UNEXPECTED-FAIL | testHistoryTab | First row has Today header - Yesterday
	at junit.framework.Assert.fail(Assert.java:47)
	at org.mozilla.fennec.FennecMochitestAssert._logMochitestResult(FennecMochitestAssert.java:107)
	at org.mozilla.fennec.FennecMochitestAssert.ok(FennecMochitestAssert.java:136)
	at org.mozilla.fennec.tests.testHistoryTab.testList(testHistoryTab.java:92)
	at org.mozilla.fennec.tests.testHistoryTab.testHistoryTab(testHistoryTab.java:60)
	at java.lang.reflect.Method.invokeNative(Native Method)
	at java.lang.reflect.Method.invoke(Method.java:521)
	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204)
	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194)
	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186)
	at org.mozilla.fennec.tests.BaseTest.runTest(BaseTest.java:95)
	at junit.framework.TestCase.runBare(TestCase.java:127)
	at junit.framework.TestResult$1.protect(TestResult.java:106)
	at junit.framework.TestResult.runProtected(TestResult.java:124)
	at junit.framework.TestResult.run(TestResult.java:109)
	at junit.framework.TestCase.run(TestCase.java:118)
	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:520)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447)
14 INFO TEST-UNEXPECTED-FAIL | testHistoryTab | Exception caught - junit.framework.AssertionFailedError: 13 INFO TEST-UNEXPECTED-FAIL | testHistoryTab | First row has Today header - Yesterday
15 INFO TEST-END | testHistoryTab | finished in 40962ms
16 INFO TEST-START | Shutdown
17 INFO Passed: 11
18 INFO Failed: 2
19 INFO Todo: 0
20 INFO SimpleTest FINISHED
INFO | automation.py | Application ran for: 0:00:44.956662
INFO | automation.py | Reading PID log: /tmp/tmpspJ12Ypidlog
getting files in '/mnt/sdcard/tests/profile/minidumps/'
WARNING | automationutils.processLeakLog() | refcount logging is off, so leaks can't be detected!

Comment 1

[Geoff Brown [:gbrown] (away Feb 17)]

I checked the other robocop tests for similar vulnerabilities; I think only testHistoryTab is affected.

Comment 2

[Geoff Brown [:gbrown] (away Feb 17)]

Attachment: allow 'Yesterday' instead of 'Today' if test run at 23:xx

If the test starts "near midnight" (grossly approximated as any time satisfying 23:xx:xx), allow for a history heading of "Yesterday" instead of "Today".

Comment 3

[Joel Maher ( :jmaher ) (UTC-4) (back Feb 19th)]

Comment on attachment 664649 [details] [diff] [review]
allow 'Yesterday' instead of 'Today' if test run at 23:xx

Review of attachment 664649 [details] [diff] [review]:
-----------------------------------------------------------------

oh wow.  I can imagine future tests hitting this problem, so lets be careful on reviews.

Comment 4

[Geoff Brown [:gbrown] (away Feb 17)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/15cd63bd65d5

Comment 5

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/15cd63bd65d5