Closed Bug 1290850 Opened 8 years ago Closed 7 years ago

Intermittent test_software_update.py TestSoftwareUpdate.test_build_info | TypeError: argument of type 'NoneType' is not iterable

Categories

(Testing :: Marionette Client and Harness, defect)

Product:
Testing
Component:
Marionette Client and Harness
Version:
Version 3
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox54 fixed, firefox55 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla55
Tracking Flags:
Tracking Status
firefox54 --- fixed
firefox55 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: whimboo)

References

Details

(Keywords: intermittent-failure, pi-marionette-firefox-puppeteer, Whiteboard: [stockwell infra])

Attachments

(1 file)

Bug 1290850 - Force local update URL for software update unit tests.
59 bytes, text/x-review-board-request
impossibus
: review+
Details 
 10:25:40     INFO - TEST-UNEXPECTED-ERROR | test_software_update.py TestSoftwareUpdate.test_build_info | TypeError: argument of type 'NoneType' is not iterable

 10:25:40     INFO - Traceback (most recent call last):

 10:25:40     INFO -   File "/home/worker/workspace/build/venv/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 344, in run

 10:25:40     INFO -     testMethod()

 10:25:40     INFO -   File "/home/worker/workspace/build/tests/firefox-ui/tests/puppeteer/test_software_update.py", line 42, in test_build_info

 10:25:40     INFO -     self.assertIn('xml', build_info['update_snippet'])

 10:25:40     INFO -   File "/usr/lib/python2.7/unittest/case.py", line 798, in assertIn

10:25:40 INFO - if member not in container: 

Looks like we got an invalid snippet returned which was not an XML document. We better should tweak the test to use the local web server. In that case we would not depend on network.
this has failed 38 times today :(

https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1290850&startday=2017-04-10&endday=2017-04-13&tree=all

:whimboo, could you look into this or help find someone to look at this?
Flags: needinfo?(hskupin)
Whiteboard: [stockwell needswork]
I had a look at the test and that's what happened:

[task 2017-04-13T03:53:07.990215Z] 03:53:07     INFO -  1492055587985	Marionette	TRACE	22 -> [0,52,"executeScript",{"scriptTimeout":null,"newSandbox":true,"args":["https://aus5.mozilla.org/update/6/%PRODUCT%/%VERSION%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%SYSTEM_CAPABILITIES%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml"],"filename":"software_update.py","script":"\n          Components.utils.import(\"resource://gre/modules/UpdateUtils.jsm\")\n          return UpdateUtils.formatUpdateURL(arguments[0]);\n        ","sandbox":"default","line":383}]
[task 2017-04-13T03:53:08.005849Z] 03:53:08     INFO -  1492055588003	Marionette	TRACE	22 <- [1,52,null,{"value":"https://aus5.mozilla.org/update/6/Firefox/55.0a1/20170413031514/Linux_x86-gcc3/en-US/default/Linux%203.13.0-112-generic%20(GTK%203.18.9%2Clibpulse%208.0.0)/NA/default/default/update.xml"}]

First we retrieve the update URL from Firefox and then use a HTTP connection to retrieve the update snippet details. So as the above URL shows we access a remote site which is aus5.mozilla.org. I believe that there might have been an outage today, which caused this problem.

To solve this problem I suggest that we override the update URL with a local one as served via wptserve.

I'm out the next days and will be back on Tuesday. So I won't have time before, and hope that aus5.mozilla.org will deliver data fine again.
Flags: needinfo?(hskupin)
Ben, can you see some anomalies of aus5.mozilla.org in delivering data today?
Flags: needinfo?(bhearsum)
(In reply to Henrik Skupin (:whimboo) from comment #5)
> Ben, can you see some anomalies of aus5.mozilla.org in delivering data today?

We had a brief outage that caused a bunch of 500s from roughly  8:48 to about 9:05 PM PDT on April 12th - does that line up with the recent failures?
Flags: needinfo?(bhearsum)
this looks to be related exclusively to the outage.
Whiteboard: [stockwell needswork] → [stockwell infra]
Thanks Ben and Joel, for checking this. 

We should improve the assertion for the unit test, by simply forcing a local AUS URL. That would kill any failures due to a remote outage, and also is something we need for making this unit test a tier-1.

I will see what I can do this week.
Assignee: nobody → hskupin
Status: NEW → ASSIGNED
Component: Firefox UI Tests → Marionette
Keywords: ateam-marionette-firefox-puppeteer
QA Contact: hskupin
Comment on attachment 8860952 [details]
Bug 1290850 - Force local update URL for software update unit tests.

https://reviewboard.mozilla.org/r/132954/#review135920

::: testing/firefox-ui/tests/puppeteer/test_software_update.py:50
(Diff revision 1)
> -        self.assertTrue(build_info['locale'])
> +            self.assertTrue(build_info['locale'])
> -        self.assertIn('force=1', build_info['update_url'])
> +            self.assertIn('force=1', build_info['update_url'])
> -        self.assertIn('xml', build_info['update_snippet'])
> +            self.assertIn('xml', build_info['update_snippet'])
> -        self.assertEqual(build_info['channel'], self.software_update.update_channel)
> +            self.assertEqual(build_info['channel'], self.software_update.update_channel)
> +        finally:
> +            self.restart(clean=True)

I'm not sure why a restart is being added here in the context of this bug. Could you describe this in the commit message for future archaeologists?
Attachment #8860952 - Flags: review?(mjzffr) → review+
Comment on attachment 8860952 [details]
Bug 1290850 - Force local update URL for software update unit tests.

https://reviewboard.mozilla.org/r/132954/#review135920

> I'm not sure why a restart is being added here in the context of this bug. Could you describe this in the commit message for future archaeologists?

I don't think a comment in the commit message is useful here. If we need one it has to be in the code right before this line. I will do that before pushing.
Comment on attachment 8860952 [details]
Bug 1290850 - Force local update URL for software update unit tests.

https://reviewboard.mozilla.org/r/132954/#review135920

> I don't think a comment in the commit message is useful here. If we need one it has to be in the code right before this line. I will do that before pushing.

So in short, we have to reset the custom update url which we set via the default preferences. But I see now that I should move the line which sets the URL into this test, because it's not needed somewhere else.
Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d467cc23f675
Force local update URL for software update unit tests. r=maja_zf
https://hg.mozilla.org/mozilla-central/rev/d467cc23f675
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
status-firefox55: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
A test-only change which stabilizes the firefox puppeteer unit tests. Please uplift to mozilla-beta. Thanks.
status-firefox54: --- → affected
Whiteboard: [stockwell infra] → [stockwell infra][checkin-needed-beta]
https://hg.mozilla.org/releases/mozilla-beta/rev/b310658e4537
status-firefox54: affected → fixed
Whiteboard: [stockwell infra][checkin-needed-beta] → [stockwell infra]
Depends on: 1361849
Product: Testing → Remote Protocol
Moving bug to Testing::Marionette Client and Harness component per bug 1815831.
Component: Marionette → Marionette Client and Harness
Product: Remote Protocol → Testing
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: