1338040 - Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past | AssertionError: 0 != 1

Status: RESOLVED FIXED

(Testing :: Marionette, defect, P1)

Description

[Treeherder Bug Filer]

Filed by: philringnalda [at] gmail.com

https://treeherder.mozilla.org/logviewer.html#?job_id=75769840&repo=autoland

https://queue.taskcluster.net/v1/task/eD5au95CRbiTOnUION_0lg/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Henrik Skupin (:whimboo) [⌚️UTC+1]]

[task 2017-02-09T02:39:08.880335Z] 02:39:08  WARNING -  TEST-UNEXPECTED-FAIL | test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past | AssertionError: 0 != 1
[task 2017-02-09T02:39:08.880725Z] 02:39:08     INFO -  Traceback (most recent call last):
[task 2017-02-09T02:39:08.881063Z] 02:39:08     INFO -    File "/home/worker/workspace/build/venv/local/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 166, in run
[task 2017-02-09T02:39:08.881297Z] 02:39:08     INFO -      testMethod()
[task 2017-02-09T02:39:08.881693Z] 02:39:08     INFO -    File "/home/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py", line 38, in test_adding_a_cookie_that_expired_in_the_past
[task 2017-02-09T02:39:08.881867Z] 02:39:08     INFO -      self.assertEquals(0, len(cookies))

Comment 2

[Intermittent Failures Robot]

5 failures in 833 pushes (0.006 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* mozilla-inbound: 2
* mozilla-beta: 1
* graphics: 1
* autoland: 1

Platform breakdown:
* android-4-3-armv7-api15: 5

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2017-02-13&endday=2017-02-19&tree=all

Comment 3

[Intermittent Failures Robot]

1 failures in 814 pushes (0.001 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 1

Platform breakdown:
* android-4-3-armv7-api15: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2017-06-12&endday=2017-06-18&tree=all

Comment 4

[Intermittent Failures Robot]

1 failures in 720 pushes (0.001 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* android-4-3-armv7-api15: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2017-07-10&endday=2017-07-16&tree=all

Comment 5

[Intermittent Failures Robot]

1 failures in 1008 pushes (0.001 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* android-4-3-armv7-api15: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2017-07-24&endday=2017-07-30&tree=all

Comment 6

[Intermittent Failures Robot]

3 failures in 888 pushes (0.003 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* try: 1
* mozilla-inbound: 1
* autoland: 1

Platform breakdown:
* android-4-3-armv7-api15: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2017-07-31&endday=2017-08-06&tree=all

Comment 7

[Intermittent Failures Robot]

1 failures in 949 pushes (0.001 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* android-4-3-armv7-api15: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2017-08-14&endday=2017-08-20&tree=all

Comment 8

[Intermittent Failures Robot]

1 failures in 939 pushes (0.001 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 1

Platform breakdown:
* android-4-3-armv7-api16: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2017-08-28&endday=2017-09-03&tree=all

Comment 9

[Intermittent Failures Robot]

4 failures in 885 pushes (0.005 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 2
* mozilla-beta: 1
* autoland: 1

Platform breakdown:
* android-4-3-armv7-api16: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2017-09-25&endday=2017-10-01&tree=all

Comment 10

[Peter Major]

The tests are probably failing because the result of time.gmtime() is dependent on system clock. There is a slim chance that NTP adjusts the time between the creation and the addition of the cookie. A solution for this would be to set the expiration time a bit further in the past for the test.

Comment 11

[Intermittent Failures Robot]

1 failures in 824 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* android-4-3-armv7-api16: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2017-10-02&endday=2017-10-08&tree=all

Comment 12

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bugmasters#Intermittent_Test_Failure_Cleanup

Comment 13

[Intermittent Failures Robot]

2 failures in 792 pushes (0.003 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1
* mozilla-central: 1

Platform breakdown:
* android-4-3-armv7-api16: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2017-11-27&endday=2017-12-03&tree=all

Comment 14

[Bogdan Tara[:bogdan_tara]]

Reopening since it's happening again.

Recent log:
https://treeherder.mozilla.org/logviewer.html#?job_id=157285299&repo=mozilla-central&lineNumber=2006

Comment 15

[Intermittent Failures Robot]

3 failures in 657 pushes (0.005 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-central: 2
* mozilla-beta: 1

Platform breakdown:
* android-4-3-armv7-api16: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2018-01-15&endday=2018-01-21&tree=all

Comment 16

[Intermittent Failures Robot]

1 failures in 675 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* android-4-3-armv7-api16: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2018-02-19&endday=2018-02-25&tree=all

Comment 17

[Intermittent Failures Robot]

1 failures in 831 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-central: 1

Platform breakdown:
* android-4-3-armv7-api16: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2018-02-26&endday=2018-03-04&tree=all

Comment 18

[Intermittent Failures Robot]

1 failures in 814 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* android-4-3-armv7-api16: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338040&startday=2018-03-12&endday=2018-03-18&tree=all

Comment 19

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 20

[Intermittent Failures Robot]

1 failures in 792 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* android-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2018-04-09&endday=2018-04-15&tree=trunk

Comment 21

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 22

[Natalia Csoregi [:nataliaCs]]

https://treeherder.mozilla.org/logviewer.html#?job_id=178518713&repo=autoland&lineNumber=1852

Comment 23

[Intermittent Failures Robot]

1 failures in 639 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* android-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2018-05-14&endday=2018-05-20&tree=trunk

Comment 24

[Intermittent Failures Robot]

1 failures in 655 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* android-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2018-05-28&endday=2018-06-03&tree=trunk

Comment 25

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 26

[Narcis Beleuzu [:NarcisB]]

Fail reappeared on inbound:
https://treeherder.mozilla.org/logviewer.html#?job_id=186344931&repo=mozilla-inbound&lineNumber=1599

Comment 27

[Intermittent Failures Robot]

2 failures in 622 pushes (0.003 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1
* autoland: 1

Platform breakdown:
* android-em-4-3-armv7-api16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2018-07-02&endday=2018-07-08&tree=trunk

Comment 28

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 29

[Andreea Pavel [:apavel]]

Recent failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=197702810&repo=autoland&lineNumber=1591

[task 2018-09-05T21:31:15.674Z] 21:31:15     INFO -  TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past
[task 2018-09-05T21:31:21.236Z] 21:31:21  WARNING -  TEST-UNEXPECTED-FAIL | testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past | AssertionError: 0 != 1
[task 2018-09-05T21:31:21.237Z] 21:31:21     INFO -  Traceback (most recent call last):
[task 2018-09-05T21:31:21.238Z] 21:31:21     INFO -    File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 159, in run
[task 2018-09-05T21:31:21.238Z] 21:31:21     INFO -      testMethod()
[task 2018-09-05T21:31:21.239Z] 21:31:21     INFO -    File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py", line 40, in test_adding_a_cookie_that_expired_in_the_past
[task 2018-09-05T21:31:21.239Z] 21:31:21     INFO -      self.assertEquals(0, len(cookies))
[task 2018-09-05T21:31:21.240Z] 21:31:21     INFO -  TEST-INFO took 5563ms
[task 2018-09-05T21:31:21.577Z] 21:31:21     INFO -  TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py CookieTest.test_chrome_error

Comment 30

[Intermittent Failures Robot]

2 failures in 2667 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 2

Platform breakdown:
* android-em-4-3-armv7-api16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2018-09-03&endday=2018-09-09&tree=all

Comment 31

[Intermittent Failures Robot]

1 failures in 2773 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1

Platform breakdown:
* android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2018-09-24&endday=2018-09-30&tree=all

Comment 32

[Intermittent Failures Robot]

1 failures in 2708 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1

Platform breakdown:
* android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2018-10-01&endday=2018-10-07&tree=all

Comment 33

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 34

[Noemi Erli[:noemi_erli]]

Recent log failure: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=232496678&repo=autoland&lineNumber=2653

[task 2019-03-07T19:54:43.887Z] 19:54:43 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past
[task 2019-03-07T19:54:44.240Z] 19:54:44 WARNING - TEST-UNEXPECTED-FAIL | testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past | AssertionError: 0 != 1
[task 2019-03-07T19:54:44.240Z] 19:54:44 INFO - Traceback (most recent call last):
[task 2019-03-07T19:54:44.241Z] 19:54:44 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 159, in run
[task 2019-03-07T19:54:44.241Z] 19:54:44 INFO - testMethod()
[task 2019-03-07T19:54:44.241Z] 19:54:44 INFO - File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py", line 42, in test_adding_a_cookie_that_expired_in_the_past
[task 2019-03-07T19:54:44.242Z] 19:54:44 INFO - self.assertEquals(0, len(cookies))
[task 2019-03-07T19:54:44.242Z] 19:54:44 INFO - TEST-INFO took 353ms

Comment 35

[Henrik Skupin (:whimboo) [⌚️UTC+1]]

Geoff tried to fix that with bug 1526913 but as it looks like we can still hit this failure.

Comment 36

[Intermittent Failures Robot]

2 failures in 2818 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 1

Platform breakdown:

• android-em-7-0-x86_64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2019-03-04&endday=2019-03-10&tree=all

Comment 37

[Intermittent Failures Robot]

2 failures in 2771 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-inbound: 1
• autoland: 1

Platform breakdown:

• android-em-7-0-x86_64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2019-03-11&endday=2019-03-17&tree=all

Comment 38

[Intermittent Failures Robot]

4 failures in 2561 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4

Platform breakdown:

• android-em-7-0-x86_64: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2019-03-18&endday=2019-03-24&tree=all

Comment 39

[Intermittent Failures Robot]

10 failures in 3910 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 6
• mozilla-beta: 1
• try: 2
• mozilla-central: 1

Platform breakdown:

• android-em-7-0-x86_64: 10

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2019-03-25&endday=2019-03-31&tree=all

Comment 40

[Intermittent Failures Robot]

8 failures in 2843 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4
• mozilla-beta: 1
• mozilla-central: 2
• try: 1

Platform breakdown:

• android-em-7-0-x86_64: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2019-04-01&endday=2019-04-07&tree=all

Comment 42

[Intermittent Failures Robot]

26 failures in 3025 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-inbound: 4
• mozilla-beta: 6
• mozilla-central: 2
• autoland: 13
• try: 1

Platform breakdown:

• android-em-7-0-x86_64: 26

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2019-04-08&endday=2019-04-14&tree=all

Comment 43

[Dorel Luca [:dluca]]

This bug has failed 57 times in the last 7 days. Occurs on android-em-7-0-x86_64 opt and debug build types.

Recent log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=241181373&repo=autoland&lineNumber=2681

Comment 44

[Geoff Brown [:gbrown]]

Reviewing recent failures, almost all have android-performance.log artifacts corresponding to bug 1545308.

Comment 45

[Intermittent Failures Robot]

15 failures in 502 pushes (0.03 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 14
• mozilla-central: 1

Platform breakdown:

• android-em-7-0-x86_64: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2019-04-18&endday=2019-04-18&tree=all

Comment 46

[Intermittent Failures Robot]

20 failures in 354 pushes (0.056 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 17
• mozilla-inbound: 2
• mozilla-beta: 1

Platform breakdown:

• android-em-7-0-x86_64: 20

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2019-04-19&endday=2019-04-19&tree=all

Comment 47

[Intermittent Failures Robot]

78 failures in 2673 pushes (0.029 failures/push) were associated with this bug in the last 7 days.

This is the #27 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-beta: 6
• mozilla-inbound: 7
• try: 2
• autoland: 60
• mozilla-central: 3

Platform breakdown:

• android-em-7-0-x86_64: 78

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2019-04-15&endday=2019-04-21&tree=all

Comment 48

[Henrik Skupin (:whimboo) [⌚️UTC+1]]

Geoff, any idea when the situation on packet.net will become better? Otherwise we might just extend the time for this test to 1h or so, so that slowness doesn't affect us this much.

Comment 49

[Geoff Brown [:gbrown]]

I think there is a good chance we can resolve it soon. I've nudged :coop, https://bugzilla.mozilla.org/show_bug.cgi?id=1545308#c2.

Comment 50

[Intermittent Failures Robot]

15 failures in 499 pushes (0.03 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 12
• try: 2
• mozilla-inbound: 1

Platform breakdown:

• android-em-7-0-x86_64: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2019-04-23&endday=2019-04-23&tree=all

Comment 51

[Intermittent Failures Robot]

25 failures in 577 pushes (0.043 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 20
• mozilla-inbound: 5

Platform breakdown:

• android-em-7-0-x86_64: 25

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2019-04-25&endday=2019-04-25&tree=all

Comment 52

[Henrik Skupin (:whimboo) [⌚️UTC+1]]

To make the test more robust lets use a cookie which already expired a day ago and not just some seconds before.

Here a try build:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=831c0dde08f3416b84b3c8289f1495f8add926da

Comment 53

[Henrik Skupin (:whimboo) [⌚️UTC+1]]

Attachment: Bug 1338040 - [marionette] Use a cookie expired a day ago for test_adding_a_cookie_that_expired_in_the_past.

Comment 54

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/91925dfb4532
[marionette] Use a cookie expired a day ago for test_adding_a_cookie_that_expired_in_the_past. r=ato

Comment 55

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/91925dfb4532

Comment 56

[Intermittent Failures Robot]

66 failures in 3031 pushes (0.022 failures/push) were associated with this bug in the last 7 days.

This is the #29 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 49
• try: 3
• mozilla-inbound: 12
• mozilla-beta: 2

Platform breakdown:

• android-em-7-0-x86_64: 66

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1338040&startday=2019-04-22&endday=2019-04-28&tree=all

Comment 57

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/c139018f30d9