Closed
Bug 636693
Opened 13 years ago
Closed 13 years ago
cfx docs throws "TypeError: 'int' object is not iterable"
Categories
(Add-on SDK Graveyard :: General, defect)
Add-on SDK Graveyard
General
Tracking
(Not tracked)
VERIFIED
FIXED
People
(Reporter: irakli, Assigned: warner)
References
Details
(Whiteboard: [cherry-pick-1.0b3])
Attachments
(1 file, 1 obsolete file)
It seems that cfx docs is broken. Also its seems that following commit caused the breakage: https://github.com/mozilla/addon-sdk/commit/5512c7a143bfeac2cc9bbcffda744e8cd4c5cbd4#diff-28 P.S: Cherry picking really messes up history and it was hard to find the commit that caused the problem.
Reporter | ||
Comment 1•13 years ago
|
||
This is an error output: (addon-sdk)➜ api-utils (5512c7a...) cfx docs One moment. Traceback (most recent call last): File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/server.py", line 447, in <module> httpd, port = fault_tolerant_make_httpd(env_root) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/server.py", line 262, in fault_tolerant_make_httpd httpd = make_httpd(env_root, host, port, quiet=True) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/server.py", line 250, in make_httpd web_docs = webdocs.WebDocs(env_root) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/webdocs.py", line 63, in __init__ self.base_page = self._create_base_page(root, base_url) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/webdocs.py", line 130, in _create_base_page self._create_package_summaries(self.packages_json, is_low_level) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/webdocs.py", line 112, in _create_package_summaries text += self._create_module_list(package_json) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/webdocs.py", line 92, in _create_module_list package_json['files']['lib']) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/webdocs.py", line 30, in get_documented_modules modules = get_modules(modules_json) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/webdocs.py", line 24, in get_modules sub_modules = get_modules(modules_json[module]) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/webdocs.py", line 24, in get_modules sub_modules = get_modules(modules_json[module]) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/webdocs.py", line 20, in get_modules for module in modules_json: TypeError: 'int' object is not iterable
Comment 2•13 years ago
|
||
(In reply to comment #0) > It seems that cfx docs is broken. Hmm, I can't reproduce the problem on Mac OS X 10.6.6 with Python 2.6.6, Ubuntu Linux 10.04 with Python 2.6.5, nor Windows 7 with Python 2.6.5. With which OSes and Python versions are you noticing the problem? > P.S: Cherry picking really messes up history and it was hard to find the commit > that caused the problem. How so?
Summary: cfx docs is broken → cfx docs throws "TypeError: 'int' object is not iterable"
Comment 3•13 years ago
|
||
Are there any extra packages/modules in your tree? The code there uses the JSON structure returned from 'packaging.py' to build the list of modules. It assumes that in the 'lib' list in that JSON, items are either modules or are themselves lists of modules, i.e.: 'lib': { 'securable-module.js': {'size': 25326}, 'memory.js': {'size': 4754}, 'tab-browser.js': {'size': 25192}, 'content': { 'symbiont.js': {'size': 6334}, 'worker.js': {'size': 12798}, 'loader.js': {'size': 6691} }, This assumption is true for the packages and modules in the SDK, I think. If something in there caused this assumption to fail you might see an error like that. I'm not very familiar with the code in 'packaging.py' though, so I'm not sure what exactly would cause that.
Reporter | ||
Comment 4•13 years ago
|
||
So I have noticed that if I make a fresh clone it works fine again for some time but then it get's broken again :( Also I noticed that cloning this repo: https://github.com/Gozala/jetpack-net into packages folder breaks cfx docs regardless of how fresh my clone is. Mac OS X 10.6.6 Python 2.6.1
Reporter | ||
Comment 5•13 years ago
|
||
Also I do have additional packages but if I go back to the version mentioned above everything works fine again. Also removing all extra packages does not solves an issue. Only fresh clone seems to work.
Comment 6•13 years ago
|
||
(In reply to comment #4) > So I have noticed that if I make a fresh clone it works fine again for some > time but then it get's broken again :( Is there any obvious trigger which breaks it again, after you get a fresh clone? > Also I noticed that cloning this repo: https://github.com/Gozala/jetpack-net > into packages folder breaks cfx docs regardless of how fresh my clone is. So, to be clear, this sequence of commands: ~ > cd mozilla/ ~/mozilla > git clone git://github.com/mozilla/addon-sdk.git ~/mozilla > cd addon-sdk ~/mozilla/addon-sdk(master) > source bin/activate (addon-sdk)~/mozilla/addon-sdk/packages(master) > git clone git://github.com/Gozala/jetpack-net.git (addon-sdk)~/mozilla/addon-sdk/packages(master) > cfx docs ? This works for me. (At least: it's not showing the modules inside the 'net' package, but the doc browser launches, and everything else seems like it's working.)
Comment 7•13 years ago
|
||
(In reply to comment #6) > (In reply to comment #4) > > So I have noticed that if I make a fresh clone it works fine again for some > > time but then it get's broken again :( > > Is there any obvious trigger which breaks it again, after you get a fresh > clone? > > > Also I noticed that cloning this repo: https://github.com/Gozala/jetpack-net > > into packages folder breaks cfx docs regardless of how fresh my clone is. > > So, to be clear, this sequence of commands: > > ~ > cd mozilla/ > ~/mozilla > git clone git://github.com/mozilla/addon-sdk.git > ~/mozilla > cd addon-sdk > ~/mozilla/addon-sdk(master) > source bin/activate + (addon-sdk)~/mozilla/addon-sdk(master) > cd packages/ > (addon-sdk)~/mozilla/addon-sdk/packages(master) > git clone > git://github.com/Gozala/jetpack-net.git > (addon-sdk)~/mozilla/addon-sdk/packages(master) > cfx docs >
Comment 8•13 years ago
|
||
One thing that might help me, if it's not too much trouble, would be to see the package index. You can generate it using cuddlefish.packaging:
(addon-sdk)~/mozilla/addon-sdk(master) > python
>>> import cuddlefish.packaging, os, simplejson
>>> pkg_cfg = cuddlefish.packaging.build_pkg_cfg(os.getcwd())
>>> pkg_index = cuddlefish.packaging.build_pkg_index(pkg_cfg)
>>> print simplejson.dumps(pkg_index, indent=2)
Then I could compare them to see if there are any interesting differences.
Reporter | ||
Comment 9•13 years ago
|
||
Output was huge so I created this gist out of it. https://gist.github.com/52cb8a55cf25be538d22 Also I guess that json does not really needs to contain all the files from the nested git repositories or *.un~ files that are generated by VIM.
Assignee | ||
Comment 10•13 years ago
|
||
I found it: to reproduce on a fresh addon-sdk checkout, just do: touch packages/addon-kit/lib/IAMNOTJAVASCRIPT cfx docs The problem is that the test in get_modules() assumes that the JSON structure contains only javascript files (*.js) and directories, and mistakenly concludes that an entry that doesn't have ".js" in the name must be a directory. The JSON structure is basically just a recursive walk of the filesystem, so any random file (editor temp files, the funny .DS_Store file that OS-X likes to leave around, .gitignore files) will show up there. We need to tolerate lots of random files in those directories. I'll write up a patch.
Assignee | ||
Comment 11•13 years ago
|
||
This patch fixes the crash, and adds some tests.
Attachment #515197 -
Flags: review?(rFobic)
Reporter | ||
Comment 12•13 years ago
|
||
Brian, Thanks for the patch. I think there is a different issue now, it looks like `keywords` property in package.json causes some troubles. Here is my output: Traceback (most recent call last): File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/server.py", line 447, in <module> httpd, port = fault_tolerant_make_httpd(env_root) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/server.py", line 262, in fault_tolerant_make_httpd httpd = make_httpd(env_root, host, port, quiet=True) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/server.py", line 250, in make_httpd web_docs = webdocs.WebDocs(env_root) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/webdocs.py", line 66, in __init__ self.base_page = self._create_base_page(root, base_url) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/webdocs.py", line 129, in _create_base_page self._create_package_summaries(self.packages_json, is_high_level) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/webdocs.py", line 109, in _create_package_summaries if not include(package_json): File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/webdocs.py", line 53, in is_high_level return not is_low_level(package_json) File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/webdocs.py", line 56, in is_low_level return 'jetpack-low-level' in package_json['keywords'] File "/Users/gozala/Projects/addon-sdk/python-lib/cuddlefish/bunch.py", line 13, in __getitem__ item = dict.__getitem__(self, key) KeyError: 'keywords'
Reporter | ||
Comment 13•13 years ago
|
||
Also removing packages that contain keywords: ["foo", "bar"] in package.json everything works. I think we should tolerate all the properties in package.json regardless if we understand them or not. Also I've noticed that API-utils no longer shows up in docs and I'm not sure if that's intentional.
Comment 14•13 years ago
|
||
(In reply to comment #13) > Also removing packages that contain keywords: ["foo", "bar"] in package.json > everything works. I think we should tolerate all the properties in package.json > regardless if we understand them or not. Can you file that as a separate bug?
Assignee | ||
Comment 15•13 years ago
|
||
here's an updated patch, which also tolerates the lack of a 'keywords' property in the package.json, which seems to be the problem in that traceback
Attachment #515197 -
Attachment is obsolete: true
Attachment #515197 -
Flags: review?(rFobic)
Attachment #515670 -
Flags: review?(rFobic)
Updated•13 years ago
|
Attachment #515670 -
Flags: review?(rFobic) → review+
Updated•13 years ago
|
Whiteboard: [cherry-pick-needed]
Assignee | ||
Comment 16•13 years ago
|
||
Landed in https://github.com/mozilla/addon-sdk/commit/345b68ec7e443f3aef85dc65b285a22ab229de75
Assignee | ||
Comment 17•13 years ago
|
||
ok, I think that closes this one. I think we *do* tolerate all properties in package.json .. the problem was that webdocs.py was relying upon having a 'keywords' property (whereas for almost everything else it touches, it checks to see if the property is present before using it). The only remaining requirement is the ".name" property: if a package doesn't have that, webdocs.py will probably crash, but I think it's reasonable to require a .name property on all packages.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Comment 18•13 years ago
|
||
Thanks for the fix! Cherry-pick commit: https://github.com/mozilla/addon-sdk/commit/e47587d7fa845ea2074f15df4d020899e1ecb7be
Assignee: nobody → warner-bugzilla
Status: RESOLVED → VERIFIED
OS: Mac OS X → All
Hardware: x86 → All
Whiteboard: [cherry-pick-needed] → [cherry-pick-1.0b3]
You need to log in
before you can comment on or make changes to this bug.
Description
•