Closed Bug 1210466 Opened 6 years ago Closed 6 years ago

test_call_log_groups.py : AssertionError: '12/01/2014' != u'12/01/14'

Categories

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

ARM
Gonk (Firefox OS)
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: MaxIvanov, Assigned: martijn.martijn)

References

Details

Attachments

(2 files)

Description: Test case fails with "AssertionError"

Steps:
1. Update a flame device to 20151001030229
2. Run this test case http://lxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/functional/dialer/test_call_log_groups.py

Actual result:
"AssertionError"

Expected result:
Test completes successfully.

Repro Rate: Reproduced via Jenkins (0/5 passing)
http://jenkins1.qa.scl3.mozilla.com/job/flame-kk.ui.adhoc.bitbar/265/HTML_Report/

Traceback (most recent call last):
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/.env/lib/python2.7/site-packages/marionette_client-0.17-py2.7.egg/marionette/marionette_test.py", line 296, in run
testMethod()
File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/tests/python/gaia-ui-tests/gaiatest/tests/functional/dialer/test_call_log_groups.py", line 52, in test_call_log_groups
self.assertEqual('{:%m/%d/%Y}'.format(self.past_date_1), group_calls[2].header_text)
AssertionError: '12/01/2014' != u'12/01/14'
Attached image index.png
This started failing because of bug 1206306.
Assignee: nobody → martijn.martijn
Blocks: 1206306
QA Contact: martijn.martijn
Comment on attachment 8668932 [details] [review]
[gaia] mwargers:1210466 > mozilla-b2g:master

This fixes the failure for me. I also had to change the timezone for this test, otherwise I would get this failure:
TEST-UNEXPECTED-FAIL | test_call_log_groups.py TestCallLogGroups.test_call_log_groups | AssertionError: u'12/01/14' != u'11/30/14'
- 12/01/14
?  ^  -
+ 11/30/14
?  ^ +


Traceback (most recent call last):
  File "/Users/mwargers/.virtualenvs/zapion_test2/lib/python2.7/site-packages/marionette_client-0.17-py2.7.egg/marionette/marionette_test.py", line 296, in run
    testMethod()
  File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/tests/functional/dialer/test_call_log_groups.py", line 53, in test_call_log_groups
    self.assertEqual(u'{:%m/%d/%y}'.format(self.past_date_1), group_calls[2].header_text)
TEST-INFO took 93998ms


Not exactly sure why that is needed, but I guess it has something to do with the usage of UTC, which is then converted to a certain time on the phone.
Attachment #8668932 - Flags: review?(npark)
Attachment #8668932 - Flags: review?(jlorenzo)
Why this error did show up for me when I was working on the original patch? I would have investigate on my own. Sorry for that, 
Martijn!
Zibi, no problem. Gaia UI tests are not running on Treeherder, so there is no way you could have known. This has been going on for a while and is not an ideal situation. In the next quarter we want to get it back to report to Treeherder.
Comment on attachment 8668932 [details] [review]
[gaia] mwargers:1210466 > mozilla-b2g:master

git grep '{:%m/%d/%Y}' showed only the 2 lines you changed, which is good.

I agree with you regarding the timezone change. It seems related to bug 1206306#c8> In order to make the test less error-prone I think we can force the timezone to UTC.
Attachment #8668932 - Flags: review?(jlorenzo) → review+
I removed this part:
-        self.data_layer.set_setting('time.timezone', 'Europe/Istanbul')

And started an adhoc job to see if this works fine in the Mountain View lab:
http://jenkins1.qa.scl3.mozilla.com/view/Mozilla%20Lab/job/flame-kk.ui.adhoc/894/

I'm still surprised that I need this setting locally to get this test passing.
Setting the timezone to Europe/Paris also makes the test pass for me, but setting the timezone Europe/Amsterdam, I get the same failure.
Ok, the adhoc job is passing.
Comment on attachment 8668932 [details] [review]
[gaia] mwargers:1210466 > mozilla-b2g:master

LGTM
Attachment #8668932 - Flags: review?(npark) → review+
Merged: https://github.com/mozilla-b2g/gaia/commit/7ff515a65174cebf484cf0e17af82144103a73f0

Zibi, do you have an idea why I had to set the time.timezone setting to 'Europe/Istanbul' or 'Europe/Paris' to get not the failure as in comment 4?
Setting the time.timezone setting to 'Europe/Amsterdam' also gives me that failure.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Zibi, see question in comment 12.
Flags: needinfo?(gandalf)
My guess is that your test is timezone dependent so when it's run in one timezone it may show one day, and in another, another.

You should force all assertions to either be timezone dependent on both sides, or timezone independent on both sides (forced to UTC).
Flags: needinfo?(gandalf)
I don't see how this test is timezone dependant and even if it was how it can be that it works and not works in the cases mentioned in comment 14. There should have been no difference between 'Europe/Paris' and 'Europe/Amsterdam' timezone.
You need to log in before you can comment on or make changes to this bug.