917750 - mirror mozbase packages to m-c for ongoing harness refactoring

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[Jeff Hammel]

mozbase packages should be mirrored to m-c for (ongoing) porting of
the mochitest and reftest harness to mozbase (bug 746243 and
bug 915865 ).

Changes in the following packages are directly needed:

mozfile
mozprocess
mozprofile
mozrunner

There have also been changed in these directories which I'd elect to
version bump and mirror:

ManifestDestiny
mozdevice
mozlog

This blocks bug 915866 regarding mozrunner wait as well as smooths
over several ongoing workarounds.  Several bugs and API deficiencies
were discovered in the course of this work.  A partial list:

= TODO =

* https://bugzilla.mozilla.org/show_bug.cgi?id=912285
  [mozprocess] ProcessHandler should front-end more subprocess API

* https://bugzilla.mozilla.org/show_bug.cgi?id=913152
  [mozprofile] add interpolation function and fix screwy m-c interpolation

* Bug 913217 -
  [mozrunner] debuggers and debugger_args should be be added to __all__

* https://bugzilla.mozilla.org/show_bug.cgi?id=913970
  [mozrunner] mozrunner cleanup from mochitest on mozbase

* bug 916512
  [mozrunner] mozrunner should use -foreground at least for mac

* https://bugzilla.mozilla.org/show_bug.cgi?id=916517
  [mozprofile] rectify DEFAULT_PORTS to be actual default ports, not
  proxied ports


= FIXED =

* Bug 912338 - [mozprofile] Profile.pop_preferences() has wrongly hardc
  https://github.com/mozilla/mozbase/commit/97ffebb7629fb2076d545c36528b7be86e04beff

* [FIXED] https://bugzilla.mozilla.org/show_bug.cgi?id=911218
  [mozprofile] AddonManager should only optionally cleanup on __del__

* [FIXED] https://bugzilla.mozilla.org/show_bug.cgi?id=912338
  [mozprofile] Profile.pop_preferences() has wrongly hardcoded user.js

See also: https://etherpad.mozilla.org/great-automation-refactor

Only a few of these are currently fixed, so additional mirroring will
be needed.

Comment 1

[Jeff Hammel]

Attachment: /home/jhammel/mozilla/mozbase/bug-746243/mirroring/mirror.20130918012330.diff

outstanding changes to m-c; tip of manifestdestiny mozdevice mozfile mozlog moznetwork mozprocess mozprofile mozrunner; try run: https://tbpl.mozilla.org/?tree=Try&rev=c20c67ab4e69

Comment 2

[Jeff Hammel]

Several test failures exist on the try run of
https://bugzilla.mozilla.org/show_bug.cgi?id=917750#c1 :

- inability to import mozfile.tree on Android 4.0 Opt ;
  AIUI, this is due to using mozfile from whereever mozharness uses it
  from, not from the in-tree modules.  Again AIUI, this should be
  changed here:
  http://hg.mozilla.org/build/mozharness/file/d080fb4225a3/scripts/android_panda.py#l131

- There was the addition of $name to webapps.py :

  http://hg.mozilla.org/mozilla-central/diff/668bb7634d04/testing/mozbase/mozprofile/mozprofile/webapps.py#l1.12

- manifestparser.convert tests fail on OSX build,
  https://bugzilla.mozilla.org/show_bug.cgi?id=902610 :

"""
  AssertionError: Lists differ: ['../../../../../../../../var/... !=
  ['../bar', '../fleem', '../foo...
make[1]: *** [run-test.py] Error 1
make: *** [check] Error 2
"""

- for the b2g ICS emulator,

"""
16:36:15     INFO -  INFO | automation.py | Launching:
/builds/slave/test/build/xre/bin/xpcshell -g
/builds/slave/test/build/xre/bin -v 170 -f
/builds/slave/test/build/xre/bin/components/httpd.js -e "const
_PROFILE_PATH = '/tmp/tmpC5TPNH';const _SERVER_PORT = '8888'; const
_SERVER_ADDR ='10.0.2.2';" -f
/builds/slave/test/build/tests/reftest/server.js
16:36:15     INFO -  INFO | remotereftests.py | Server pid: 2301
16:36:21     INFO -  Traceback (most recent call last):
16:36:21     INFO -    File "runreftestb2g.py", line 566, in
run_remote_reftests
16:36:21     INFO -      retVal = reftest.runTests(manifest, options,
cmdlineArgs)
16:36:21     INFO -    File
"/builds/slave/test/build/tests/reftest/runreftest.py", line 159, in
runTests
16:36:21     INFO -      self.cleanup(profileDir)
16:36:21     INFO -    File "runreftestb2g.py", line 247, in cleanup
16:36:21     INFO -      extensionDir = os.path.join(profileDir,
'extensions', 'staged')
16:36:21     INFO -    File
"/builds/slave/test/build/venv/lib/python2.7/posixpath.py", line 68,
in join
16:36:21     INFO -      elif path == '' or path.endswith('/'):
16:36:21     INFO -  AttributeError: 'NoneType' object has no
attribute 'endswith'
16:36:21     INFO -  Automation Error: Exception caught while running
tests
16:36:21    ERROR - Return code: 256
"""

- mochitest other fails on all platforms:

"""
TypeError: aPossiblyBadFilenameString is undefined
10492 ERROR TEST-UNEXPECTED-FAIL |
chrome://mochitests/content/chrome/dom/tests/mochitest/localstorage/test_app_uninstall.html
| Test timed out.
Bug 843649 - Intermittent
content/chrome/dom/tests/mochitest/localstorage/test_app_uninstall.html
| application timed out after 330 seconds with no output 10495 ERROR
TEST-UNEXPECTED-FAIL |
chrome://mochitests/content/chrome/dom/tests/mochitest/localstorage/test_clear_browser_data.html
| no data - got bar, expected null
Bug 826924 - Intermittent test_clear_browser_data.html | application
timed out after 330 seconds with no output 10497 ERROR
TEST-UNEXPECTED-FAIL |
chrome://mochitests/content/chrome/dom/tests/mochitest/localstorage/test_clear_browser_data.html
| no data - got bar, expected null
Bug 826924 - Intermittent test_clear_browser_data.html | application
timed out after 330 seconds with no output 10499 ERROR
TEST-UNEXPECTED-FAIL |
chrome://mochitests/content/chrome/dom/tests/mochitest/localstorage/test_clear_browser_data.html
| Test timed out.
Bug 826924 - Intermittent test_clear_browser_data.html | application
timed out after 330 seconds with no output 10711 ERROR
TEST-UNEXPECTED-FAIL |
chrome://mochitests/content/chrome/dom/tests/mochitest/webapps/test_install_app.xul
| getInstalled() returns nothing - got 1, expected 0
Bug 848958 - Intermittent
test_install_errors.xul,test_getNotInstalled.xul,test_install_receipts.xul,test_install_app.xul,test_launch_paths.xul,test_install_utf8.xul
| Assertion count 1 is greater than expected range (mTempFile not
equal to mTargetFile: 'Error')
Bug 855765 - Intermittent test_bug_795164.xul | Test timed out,
test_cross_origin.xul | Test timed out, test_getNotInstalled.xul |
Test timed out, test_install_app.xul | Test timed out, | 4 test
timeouts, giving up, | Skipping 468 remaining tests
Bug 854975 - Intermittent TEST-UNEXPECTED-FAIL |
chrome://mochitests/content/chrome/dom/tests/mochitest/webapps/test_launch_paths.xul,test_install_app.xul
| install time is after install() call OR install time is before
install success
Bug 833205 - Intermittent test_install_app.xul | application timed out
after 330 seconds with no output
Bug 856356 - Intermittent webapps/test_install_app.xul | install time
is after install() call
Bug 879210 - Intermittent webapps/test_install_app.xul | install time
is before install success 10758 ERROR TEST-UNEXPECTED-FAIL |
chrome://mochitests/content/chrome/dom/tests/mochitest/webapps/test_launch_paths.xul
| getInstalled() returns nothing - got 1, expected 0
Bug 848958 - Intermittent
test_install_errors.xul,test_getNotInstalled.xul,test_install_receipts.xul,test_install_app.xul,test_launch_paths.xul,test_install_utf8.xul
| Assertion count 1 is greater than expected range (mTempFile not
equal to mTargetFile: 'Error')
Bug 854975 - Intermittent TEST-UNEXPECTED-FAIL |
chrome://mochitests/content/chrome/dom/tests/mochitest/webapps/test_launch_paths.xul,test_install_app.xul
| install time is after install() call OR install time is before
install success
Bug 858382 - Intermittent test_launch_paths.xul | application timed
out after 330 seconds with no output 11903 ERROR TEST-UNEXPECTED-FAIL
|
chrome://mochitests/content/chrome/extensions/cookie/test/test_app_uninstall_cookies.html
| Test timed out.
Bug 857824 - Intermittent test_app_uninstall_cookies.html | uncaught
exception - uncaught exception: Error in onStartRequest: Could not get
contentLength at :0 | Error in onStopRequest: Failed to load URL:
804b001e at :0 | etc...
Bug 862065 - Intermittent test_app_uninstall_cookies.html | uncaught
exception - uncaught exception: Error in onStartRequest: Could not get
contentLength at :0 | Error in onStopRequest: Failed to load URL:
804b001e at :0 | etc... 11906 ERROR TEST-UNEXPECTED-FAIL |
chrome://mochitests/content/chrome/extensions/cookie/test/test_app_uninstall_permissions.html
| Test timed out.
11907 ERROR TEST-UNEXPECTED-FAIL | (SimpleTest/TestRunner.js) | 4 test
timeouts, giving up.
11908 ERROR TEST-UNEXPECTED-FAIL | (SimpleTest/TestRunner.js) |
Skipping 468 remaining tests.
11909 ERROR TEST-UNEXPECTED-FAIL | (SimpleTest/TestRunner.js) |
chrome://mochitests/content/chrome/extensions/cookie/test/test_app_uninstall_permissions.html
finished in a non-clean fashion, probably because it didn't call SimpleTest.finish()
"""

These errors should all be fixed and the mirror re-attempted.

Comment 3

[Jonathan Griffin (:jgriffin)]

Attachment: sync.diff

New mirroring patch: https://tbpl.mozilla.org/?tree=Try&rev=ac05c57a3b93

Comment 4

[Jonathan Griffin (:jgriffin)]

This try run is totally green except for Android 4.0:

16:56:22     INFO -  Traceback (most recent call last):
16:56:22     INFO -    File "/builds/panda-0376/test/build/tests/mochitest/runtestsremote.py", line 17, in <module>
16:56:22     INFO -      from automation import Automation
16:56:22     INFO -    File "/builds/panda-0376/test/build/tests/mochitest/automation.py", line 34, in <module>
16:56:22     INFO -      from mozprofile import Profile, Preferences
16:56:22     INFO -    File "/builds/panda-0376/test/build/tests/mozbase/mozprofile/mozprofile/__init__.py", line 14, in <module>
16:56:22     INFO -      from cli import *
16:56:22     INFO -    File "/builds/panda-0376/test/build/tests/mozbase/mozprofile/mozprofile/cli.py", line 17, in <module>
16:56:22     INFO -      from profile import FirefoxProfile
16:56:22     INFO -    File "/builds/panda-0376/test/build/tests/mozbase/mozprofile/mozprofile/profile.py", line 18, in <module>
16:56:22     INFO -      from mozfile import tree
16:56:22     INFO -  ImportError: cannot import name tree

I'll dig into that a bit today; I think we should have this wrapped up soon.

Comment 5

[Jeff Hammel]

This should be https://bugzilla.mozilla.org/show_bug.cgi?id=921596

Comment 6

[Jonathan Griffin (:jgriffin)]

bug 921596 has landed; I'm waiting for try to reopen so I can verify that this allows the panda tests to run green

Comment 7

[Jonathan Griffin (:jgriffin)]

new try run: https://tbpl.mozilla.org/?tree=Try&rev=ec981b3ab542

Comment 8

[Jonathan Griffin (:jgriffin)]

Attachment: versionbump.diff

We need to bump the versions of several packages before this mirror; this is the complete set, according to versionbump.py, if we want to bump mozprofile/mozrunner/moznetwork/mozfile.

I think for simplicity's sake, I'll handle mozprocess separately after this lands.

Comment 9

[Jonathan Griffin (:jgriffin)]

(In reply to Jonathan Griffin (:jgriffin) from comment #7)
> new try run: https://tbpl.mozilla.org/?tree=Try&rev=ec981b3ab542

This is very green.  After the versionbump patch is r+'d, I'll land it in github and do another try run with the result, since it includes a few more packages than the above.

Comment 10

[Jonathan Griffin (:jgriffin)]

versionbump: https://github.com/mozilla/mozbase/commit/c41bb24efc38c463826def9fd6826ee312ee57c4

Comment 11

[Jonathan Griffin (:jgriffin)]

Attachment: sync.diff

new try run: https://tbpl.mozilla.org/?tree=Try&rev=52d80bc8c1ff

Comment 12

[Jonathan Griffin (:jgriffin)]

Comment on attachment 818725 [details] [diff] [review]
sync.diff

Review of attachment 818725 [details] [diff] [review]:
-----------------------------------------------------------------

This is completely green on try so far, so I'm going to go ahead and flag for review.

Comment 13

[Jonathan Griffin (:jgriffin)]

Oops, flagged an obsolete patch for review...

Comment 14

[Jonathan Griffin (:jgriffin)]

(In reply to Jonathan Griffin (:jgriffin) from comment #11)
> Created attachment 820412 [details] [diff] [review]
> sync.diff
> 
> new try run: https://tbpl.mozilla.org/?tree=Try&rev=52d80bc8c1ff

This is basically finished with only 1 orange!  I think this patch will fix all the things!

Comment 15

[Andrew Halberstadt [:ahal]]

Comment on attachment 820412 [details] [diff] [review]
sync.diff

Review of attachment 820412 [details] [diff] [review]:
-----------------------------------------------------------------

Don't see anything suspicious after a cursory glance. If it passes on try then wfm!

Comment 16

[Jonathan Griffin (:jgriffin)]

This was one of the greenest try runs I've ever seen.  :)

https://hg.mozilla.org/integration/mozilla-inbound/rev/e199bf0b3257

Comment 17

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/e199bf0b3257