658964 - cfx testcfx fails on test_xpi: self.failUnlessEqual(sorted(used_files), sorted(expected))

Status: RESOLVED FIXED

(Add-on SDK Graveyard :: General, defect)

Description

[Alexandre Poirot [:ochameau]]

This failure is specific to windows:
FAIL: test_contents (cuddlefish.tests.test_xpi.SmallXPI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tests
\test_xpi.py", line 103, in test_contents
    self.failUnlessEqual(sorted(used_files), sorted(expected))
AssertionError: Lists differ: [u'C:\\Users\\Coup2Bambou\\Des... != ['C:\\Users\\
Coup2Bambou\\Desk...

First differing element 0:
C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tests\linker-
files\three-deps\three-a\lib/main.js
C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tests\linker-
files\three-deps\three-a\lib\main.js

Diff is 1835 characters long. Set self.maxDiff to None to see it.

I think it's because of this line:
https://github.com/mozilla/addon-sdk/blob/master/python-lib/cuddlefish/manifest.py#L331

That will end up returning a path with "/" separator in get_used_files:
https://github.com/mozilla/addon-sdk/blob/master/python-lib/cuddlefish/manifest.py#L229

And test will fail as it uses os.path.join, that use windows separator "\":
https://github.com/mozilla/addon-sdk/blob/master/python-lib/cuddlefish/tests/test_xpi.py#L103

Comment 1

[Brian Warner [:warner :bwarner]]

Attachment: handle windows paths for main="./lib/main"

I *think* this is the source of the non-windows pathnames. When we parse the "main" property in package.json, it's allowed (in fact required) to use "/", but we need to convert this into a platform-specific separator (i.e. "\" on windows) before actually using it as a filename. Python is pretty lenient, and will accept either sort of slash, but the unit tests were using two different codepaths and the differences are causing tests to fail.

Comment 2

[Myk Melez [:myk] [@mykmelez]]

Before:

$ cfx testcfx
..............................................................E...........F........................................
======================================================================
ERROR: test_create_example_doc (cuddlefish.tests.test_webdocs.WebDocTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\myk\Projects\addon-sdk\python-lib\cuddlefish\tests\test_webdocs.py", line 87, in test_create_example_doc
    doc = web_docs.create_examples_page()
  File "c:\Users\myk\Projects\addon-sdk\python-lib\cuddlefish\webdocs.py", line 94, in create_examples_page
    examples_content = self._create_examples_listing(examples_dir)
  File "c:\Users\myk\Projects\addon-sdk\python-lib\cuddlefish\webdocs.py", line 195, in _create_examples_listing
    for dirname in os.listdir(examples_dir):
WindowsError: [Error 3] The system cannot find the path specified: 'c:\\Users\\myk\\Projects\\addon-sdk/python-lib/cuddlefish/tests/static-files\\examples/*.*'

======================================================================
FAIL: test_contents (cuddlefish.tests.test_xpi.SmallXPI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\myk\Projects\addon-sdk\python-lib\cuddlefish\tests\test_xpi.py", line 103, in test_contents
    self.failUnlessEqual(sorted(used_files), sorted(expected))
AssertionError: [u'c:\\Users\\myk\\Projects\\addon-sdk\\python-lib\\cuddlefish\\tests\\linker-files\\three-deps\\three-a\\lib/main.js', u'c:\\Users\\myk\\Projects\\addon-sdk\\python-lib\\cuddlefish\\tests\\linker-files\\three-deps\\three-b\\lib/main.js', u'c:\\Users\\myk\\Projects\\addon-sdk\\python-lib\\cuddlefish\\tests\\linker-files\\three-deps\\three-c\\lib/main.js', u'c:\\Users\\myk\\Projects\\addon-sdk\\python-lib\\cuddlefish\\tests\\linker-files\\three\\lib\\main.js'] != ['c:\\Users\\myk\\Projects\\addon-sdk\\python-lib\\cuddlefish\\tests\\linker-files\\three-deps\\three-a\\lib\\main.js', 'c:\\Users\\myk\\Projects\\addon-sdk\\python-lib\\cuddlefish\\tests\\linker-files\\three-deps\\three-b\\lib\\main.js', 'c:\\Users\\myk\\Projects\\addon-sdk\\python-lib\\cuddlefish\\tests\\linker-files\\three-deps\\three-c\\lib\\main.js', 'c:\\Users\\myk\\Projects\\addon-sdk\\python-lib\\cuddlefish\\tests\\linker-files\\three\\lib\\main.js']

----------------------------------------------------------------------
Ran 115 tests in 6.744s

FAILED (failures=1, errors=1)


After:

$ cfx testcfx
..............................................................E....................................................
======================================================================
ERROR: test_create_example_doc (cuddlefish.tests.test_webdocs.WebDocTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\myk\Projects\addon-sdk\python-lib\cuddlefish\tests\test_webdocs.py", line 87, in test_create_example_doc
    doc = web_docs.create_examples_page()
  File "c:\Users\myk\Projects\addon-sdk\python-lib\cuddlefish\webdocs.py", line 94, in create_examples_page
    examples_content = self._create_examples_listing(examples_dir)
  File "c:\Users\myk\Projects\addon-sdk\python-lib\cuddlefish\webdocs.py", line 195, in _create_examples_listing
    for dirname in os.listdir(examples_dir):
WindowsError: [Error 3] The system cannot find the path specified: 'c:\\Users\\myk\\Projects\\addon-sdk/python-lib/cuddlefish/tests/static-files\\examples/*.*'

----------------------------------------------------------------------
Ran 115 tests in 4.932s

FAILED (errors=1)
(addon-sdk)
myk@MYKPAD ~/Projects/addon-sdk

Comment 3

[Alexandre Poirot [:ochameau]]

The failing assertion now pass with your patch applied.

But there may be others failure. I have this very rare intermitent failure:
# cfx testcfx
.................................................E..............................
..................................
======================================================================
ERROR: test_generate_static_docs_does_not_smoke (cuddlefish.tests.test_server.Se
rverTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tests
\test_server.py", line 13, in test_generate_static_docs_does_not_smoke
    server.generate_static_docs(env_root, tgz_filename=filename)
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\serve
r.py", line 424, in generate_static_docs
    shutil.rmtree(staging_dir)
  File "C:\Python27\lib\shutil.py", line 253, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\Python27\lib\shutil.py", line 251, in rmtree
    os.rmdir(path)
WindowsError: [Error 145] Folder is not empty: 'C:\\Users\\Coup2Bambou\
\Desktop\\mozilla\\jetpack\\addon-sdk-docs'

----------------------------------------------------------------------
Ran 114 tests in 8.671s

FAILED (errors=1)

Comment 4

[Brian Warner [:warner :bwarner]]

Attachment: handle windows paths for main= and require(a/b/c)

I found a second codepath which would make the same mistake. If I'm analyzing it correctly, then require("a/b/c") on windows could result in a js_filename of "...\a\b/c.js", which Python would handle correctly, but some other tool or unit test that pays attention to the filename might get messed up. I don't think this one would matter, but fixing it seems like a good idea. The fact that it wasn't caught by tests before worries me a bit.. I'm going to look into how we might improve coverage of this path.

Comment 5

[Brian Warner [:warner :bwarner]]

Attachment: handle windows paths for main= and require(a/b/c)

ok, found a way to test it. If you were to apply the changes to linker-files/ and test_xpi.py, and the first change to manifest.py but not the second, then you should see test_xpi.SmallXPI.test_contents fail on windows (missing the lib/sub/foo.js entry). Then applying the full patch ought to make the test pass.

Comment 6

[Alexandre Poirot [:ochameau]]

(In reply to comment #5)
> Created attachment 534631 [details] [diff] [review] [review]
> handle windows paths for main= and require(a/b/c)

You may have forgot to attach new file?

# cfx testcfx
.........................E...............................................E......
..................................
======================================================================
ERROR: test_main_in_deps (cuddlefish.tests.test_linker.Basic)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tests
\test_linker.py", line 68, in test_main_in_deps
    "P/", scan_tests=False)
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\manif
est.py", line 566, in build_manifest
    mxt.build(scan_tests)
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\manif
est.py", line 190, in build
    self.top_uri = self.process_module(self.find_top(self.target_cfg))
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\manif
est.py", line 375, in process_module
    raise err
ModuleNotFoundError: ModuleNotFoundError: unable to satisfy require(three-c/sub/
foo) from C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tes
ts\linker-files\three\lib\main.js .
Looked for it in:
 C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tests\linker
-files\three-deps\three-c\lib\sub\foo.js
 C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tests\linker
-files\three\lib\three-c\sub\foo.js
 C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\packages\addon-kit\lib\three-c\sub
\foo.js
 C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\packages\api-utils\lib\three-c\sub
\foo.js
 C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tests\linker
-files\three\lib\three-c\sub\foo.js


======================================================================
ERROR: test_contents (cuddlefish.tests.test_xpi.SmallXPI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tests
\test_xpi.py", line 91, in test_contents
    "P/", scan_tests=False)
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\manif
est.py", line 566, in build_manifest
    mxt.build(scan_tests)
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\manif
est.py", line 190, in build
    self.top_uri = self.process_module(self.find_top(self.target_cfg))
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\manif
est.py", line 375, in process_module
    raise err
ModuleNotFoundError: ModuleNotFoundError: unable to satisfy require(three-c/sub/
foo) from C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tes
ts\linker-files\three\lib\main.js .
Looked for it in:
 C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tests\linker
-files\three-deps\three-c\lib\sub\foo.js
 C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tests\linker
-files\three\lib\three-c\sub\foo.js
 C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\packages\addon-kit\lib\three-c\sub
\foo.js
 C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\packages\api-utils\lib\three-c\sub
\foo.js
 C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tests\linker
-files\three\lib\three-c\sub\foo.js


----------------------------------------------------------------------
Ran 114 tests in 8.160s

FAILED (errors=2)

Comment 7

[Brian Warner [:warner :bwarner]]

Attachment: handle windows paths for main= and require(a/b/c)

oops, yup, I missed a file. Give this one a try.

Comment 8

[Alexandre Poirot [:ochameau]]

Seems good.

I have this error whatever I apply this patch or not, should be another regression:

.................................................E.......................F......
..................................
======================================================================
ERROR: test_generate_static_docs_does_not_smoke (cuddlefish.tests.test_server.Se
rverTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\tests
\test_server.py", line 13, in test_generate_static_docs_does_not_smoke
    server.generate_static_docs(env_root, tgz_filename=filename)
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\serve
r.py", line 357, in generate_static_docs
    package_doc_html = web_docs.create_package_page(src_dir)
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\webdo
cs.py", line 89, in create_package_page
    package_content = self._create_package_detail(package_name)
  File "C:\Users\Coup2Bambou\Desktop\mozilla\jetpack\python-lib\cuddlefish\webdo
cs.py", line 175, in _create_package_detail
    raise IOError(errno.ENOENT, 'Package not found')
IOError: [Errno 2] Package not found

Comment 9

[Brian Warner [:warner :bwarner]]

ok, patch landed in https://github.com/mozilla/addon-sdk/commit/680e544ca24688fc1baf454bd8b7808ccbca5c4f . Myk said that since this bug is targeted for 1.0, and since it fixes a test failure, the patch didn't need a specific r= .

I've opened Bug 659478 for the static-docs failure.