Closed
Bug 1173933
Opened 9 years ago
Closed 9 years ago
We should allow for more than one tab
Categories
(Testing :: Firefox UI Tests, defect)
Tracking
(firefox38 fixed, firefox39 fixed, firefox40 fixed, firefox41 fixed, firefox-esr38 fixed)
RESOLVED
FIXED
mozilla41
People
(Reporter: armenzg, Assigned: chmanchester)
References
Details
Attachments
(2 files)
I believe the reason is because after the update we get the pocket window. This update goes from 38.0 to 38.0.5 See attached log to see the real issue. Also another issue is that we can't see this error in the mozharness log [1] There is a bug in the mozharness code that looks for gecko.txt instead of gecko.log. You can reproduce this failure with this command: python scripts/firefox_ui_updates.py --firefox-ui-branch mozilla-beta --installer-url http://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/38.0-candidates/build3/mac/en-US/Firefox%2038.0.dmg --cfg developer_config.py You cannot reproduce the issue for the 39 betas (since there is no pocket tab): python scripts/firefox_ui_updates.py --firefox-ui-branch mozilla-beta --installer-url http://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/39.0b3-candidates/build1/mac/en-US/Firefox%2039.0b3.dmg --cfg developer_config.py --run-tests [1] 1:59.98 TEST_END: MainThread ERROR, expected PASS Traceback (most recent call last): File "/Users/armenzg/repos/hg/mozharness/build/venv/lib/python2.7/site-packages/marionette/marionette_test.py", line 326, in run self.tearDown() File "/Users/armenzg/repos/hg/mozharness/build/venv/lib/python2.7/site-packages/firefox_ui_tests/update/fallback/test_fallback_update.py", line 17, in tearDown UpdateTestCase.tearDown(self) File "/Users/armenzg/repos/hg/mozharness/build/venv/lib/python2.7/site-packages/firefox_ui_harness/testcases/update.py", line 235, in tearDown FirefoxTestCase.tearDown(self) File "/Users/armenzg/repos/hg/mozharness/build/venv/lib/python2.7/site-packages/firefox_ui_harness/testcases/base.py", line 75, in tearDown (self._start_handle_count, win_count)) AssertionError: A test must not leak window handles. This test started the browser with 1 open top level browsing contexts, but ended with 2. [2] 15:15:10 INFO - Current package versions: 15:15:10 INFO - manifestparser == 1.1 15:15:10 INFO - wheel == 0.24.0 15:15:10 INFO - mozhttpd == 0.7 15:15:10 INFO - mozprofile == 0.23 15:15:10 INFO - marionette-transport == 0.4 15:15:10 INFO - marionette-client == 0.12 15:15:10 INFO - mozcrash == 0.14 15:15:10 INFO - mozfile == 1.1 15:15:10 INFO - firefox-ui-tests == 0.2 15:15:10 INFO - mozlog == 2.10 15:15:10 INFO - moznetwork == 0.24 15:15:10 INFO - mozInstall == 1.12 15:15:10 INFO - mozdevice == 0.44 15:15:10 INFO - marionette-driver == 0.7 15:15:10 INFO - blessings == 1.5.1 15:15:10 INFO - mozversion == 1.1 15:15:10 INFO - mozprocess == 0.21 15:15:10 INFO - mozrunner == 6.6 15:15:10 INFO - moztest == 0.7 15:15:10 INFO - mozinfo == 0.8 ... 15:16:41 INFO - Copy/paste: /Users/armenzg/repos/hg/mozharness/build/venv/bin/firefox-ui-update --installer /Users/armenzg/repos/hg/mozharness/build/Firefox%2038.0.dmg --gecko-log=/Users/armenzg/repos/hg/mozharness/build/gecko.log ... 15:16:41 INFO - 0:00.01 LOG: MainThread INFO Installing application "/Users/armenzg/repos/hg/mozharness/build/Firefox%2038.0.dmg" to "/var/folders/wl/l70t_q8n6gg64vr7bfzjrqg80000gn/T/tmpuobWcN" 15:17:05 INFO - 0:23.09 LOG: MainThread INFO Creating a copy of the application at "/var/folders/wl/l70t_q8n6gg64vr7bfzjrqg80000gn/T/tmpQSUcBt.binary-update-tests". 15:17:09 INFO - 0:27.05 LOG: MainThread INFO starting httpd 15:17:09 INFO - 0:27.06 LOG: MainThread INFO running httpd on http://127.0.0.1:52793/ 15:17:09 INFO - 0:27.06 LOG: MainThread mozversion INFO application_buildid: 20150508094354 15:17:09 INFO - 0:27.06 LOG: MainThread mozversion INFO application_changeset: 4c4dc6640c7e 15:17:09 INFO - 0:27.06 LOG: MainThread mozversion INFO application_display_name: Firefox 15:17:09 INFO - 0:27.06 LOG: MainThread mozversion INFO application_id: {ec8030f7-c20a-464f-9b0e-13a3a9e97384} 15:17:09 INFO - 0:27.06 LOG: MainThread mozversion INFO application_name: Firefox 15:17:09 INFO - 0:27.06 LOG: MainThread mozversion INFO application_remotingname: firefox 15:17:09 INFO - 0:27.06 LOG: MainThread mozversion INFO application_repository: https://hg.mozilla.org/releases/mozilla-release 15:17:09 INFO - 0:27.06 LOG: MainThread mozversion INFO application_vendor: Mozilla 15:17:09 INFO - 0:27.06 LOG: MainThread mozversion INFO application_version: 38.0 15:17:09 INFO - 0:27.06 LOG: MainThread mozversion INFO platform_buildid: 20150508094354 15:17:09 INFO - 0:27.06 LOG: MainThread mozversion INFO platform_changeset: 4c4dc6640c7e 15:17:09 INFO - 0:27.06 LOG: MainThread mozversion INFO platform_repository: https://hg.mozilla.org/releases/mozilla-release 15:17:09 INFO - 0:27.06 LOG: MainThread mozversion INFO platform_version: 38.0 15:17:09 INFO - 0:27.06 SUITE_START: MainThread 1 15:17:09 INFO - 0:27.06 TEST_START: MainThread test_direct_update.py TestDirectUpdate.test_update 15:17:16 INFO - 0:34.53 TEST_END: MainThread ERROR, expected PASS 15:17:16 INFO - Traceback (most recent call last): ^@15:17:16 INFO - File "/Users/armenzg/repos/hg/mozharness/build/venv/lib/python2.7/site-packages/marionette/marionette_test.py", line 296, in run 15:17:16 INFO - testMethod() 15:17:16 INFO - File "/Users/armenzg/repos/hg/mozharness/build/venv/lib/python2.7/site-packages/firefox_ui_tests/update/direct/test_direct_update.py", line 26, in test_update 15:17:16 INFO - self._test_update() 15:17:16 INFO - File "/Users/armenzg/repos/hg/mozharness/build/venv/lib/python2.7/site-packages/firefox_ui_tests/update/direct/test_direct_update.py", line 20, in _test_update 15:17:16 INFO - self.download_and_apply_available_update(force_fallback=False) 15:17:16 INFO - File "/Users/armenzg/repos/hg/mozharness/build/venv/lib/python2.7/site-packages/firefox_ui_harness/testcases/update.py", line 147, in download_and_apply_available_update 15:17:16 INFO - about_window = self.browser.open_about_window() 15:17:16 INFO - File "/Users/armenzg/repos/hg/mozharness/build/venv/lib/python2.7/site-packages/firefox_puppeteer/ui/windows.py", line 576, in open_about_window 15:17:16 INFO - return BaseWindow.open_window(self, callback, AboutWindow) 15:17:16 INFO - File "/Users/armenzg/repos/hg/mozharness/build/venv/lib/python2.7/site-packages/firefox_puppeteer/ui/windows.py", line 352, in open_window 15:17:16 INFO - Wait(self.marionette).until(lambda _: window.focused == expect_focus) 15:17:16 INFO - File "/Users/armenzg/repos/hg/mozharness/build/venv/lib/python2.7/site-packages/marionette_driver/wait.py", line 143, in until 15:17:16 INFO - cause=last_exc) 15:17:16 INFO - TimeoutException: TimeoutException: Timed out after 5.1 seconds 15:17:16 INFO - 0:34.56 LOG: MainThread INFO Update test results: 15:17:16 INFO - [{'build_post': None, 15:17:16 INFO - 'build_pre': {'buildid': u'20150508094354', 15:17:16 INFO - 'channel': u'release', 15:17:16 INFO - 'disabled_addons': None, 15:17:16 INFO - 'locale': u'en-US', 15:17:16 INFO - 'mar_channels': set([u'firefox-mozilla-release']), 15:17:16 INFO - 'url_aus': u'https://aus4.mozilla.org/update/3/Firefox/38.0/20150508094354/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/Darwin%2014.3.0/default/default/update.xml?force=1', 15:17:16 INFO - 'user_agent': u'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Firefox/38.0', 15:17:16 INFO - 'version': u'38.0'}, 15:17:16 INFO - 'fallback': False, 15:17:16 INFO - 'patch': {}, 15:17:16 INFO - 'success': False}] 15:17:16 INFO - 0:34.63 LOG: MainThread INFO 15:17:16 INFO - SUMMARY 15:17:16 INFO - ------- 15:17:16 INFO - 0:34.63 LOG: MainThread INFO passed: 0 15:17:16 INFO - 0:34.63 LOG: MainThread INFO failed: 1 15:17:16 INFO - 0:34.64 LOG: MainThread INFO todo: 0 15:17:16 INFO - 0:34.64 LOG: MainThread INFO 15:17:16 INFO - FAILED TESTS 15:17:16 INFO - ------- 15:17:16 INFO - 0:34.64 LOG: MainThread INFO test_direct_update.py test_direct_update.TestDirectUpdate.test_update 15:17:16 INFO - 0:34.69 SUITE_END: MainThread 15:17:16 INFO - Summary 15:17:16 INFO - ======= 15:17:16 INFO - Ran 1 tests 15:17:16 INFO - Expected results: 0 15:17:16 INFO - Unexpected results: 1 (ERROR: 1) 15:17:16 INFO - Unexpected Results
Assignee | ||
Comment 1•9 years ago
|
||
As discussed on irc, what we'd like to do is close open tabs in teardown so we might be able to add assertions later that these promotional tabs are appearing as expected.
Assignee: nobody → cmanchester
Comment 2•9 years ago
|
||
That's correct. Something we have to do in the future is to also check for the what's new page if necessary. We cannot limit Firefox to only use a single tab.
Assignee | ||
Comment 3•9 years ago
|
||
Attachment #8621237 -
Flags: review?(armenzg)
Reporter | ||
Comment 4•9 years ago
|
||
Comment on attachment 8621237 [details] [review] PR with fix I've tested that this works. This can be reproduced like this: 1) Run with --no-run-tests python scripts/firefox_ui_updates.py --firefox-ui-branch mozilla-beta --installer-url http://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/38.0-candidates/build3/mac/en-US/Firefox%2038.0.dmg --no-run-tests --cfg developer_config.py 2) Apply the patch on build/firefox_ui_tests 3) Re-create venv & run-tests rm -rf build/venv && python scripts/firefox_ui_updates.py --installer-path `pwd`/build/Firefox%2038.0.dmg --create-virtualenv --run-tests --cfg developer_config.py
Attachment #8621237 -
Flags: review?(armenzg) → review+
Reporter | ||
Comment 5•9 years ago
|
||
We need the change on all firefox_ui_test branches.
Assignee | ||
Comment 6•9 years ago
|
||
https://github.com/mozilla/firefox-ui-tests/commit/dd810bde586d58cb239fc579ace4d0be1b0a8589 https://github.com/mozilla/firefox-ui-tests/commit/dd31762a18743dcba19db278117e4cc3a35eba7c https://github.com/mozilla/firefox-ui-tests/commit/e693f4fa832585a100de11397a5c85ea2cd61538 https://github.com/mozilla/firefox-ui-tests/commit/4161b07a3295ebd93fcfc784043b28f88149409d https://github.com/mozilla/firefox-ui-tests/commit/26353bd57b28feaa69e7fbcdef48acd24152d701
status-firefox39:
--- → fixed
status-firefox40:
--- → fixed
status-firefox41:
--- → fixed
status-firefox-esr38:
--- → fixed
Version: Version 3 → Firefox 41
Assignee | ||
Updated•9 years ago
|
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Updated•9 years ago
|
Target Milestone: --- → Firefox 41
Version: Firefox 41 → Firefox 38
Updated•9 years ago
|
status-firefox38:
--- → fixed
Updated•8 years ago
|
Product: Mozilla QA → Testing
You need to log in
before you can comment on or make changes to this bug.
Description
•