678622 - "packages" array value in package.json causes error

Status: RESOLVED WONTFIX

(Add-on SDK Graveyard :: General, defect)

Description

[James Burke [:jrburke]]

This is with Addon-SDK 1.1a2, but also happened in 1.0:

I want to use some packages that are not in my addon directory, the addon directory is a subdirectory of a larger project.

This page:
https://addons.mozilla.org/en-US/developers/docs/sdk/1.0/dev-guide/addon-development/package-spec.html

implies that a "packages" property can be an array of directory names that will be used to find packages. However, when I put an array value for "packages" in my package.json, I get the following dump when trying to run "cfx test":

Traceback (most recent call last):
  File "/Users/jr/tools/addon-sdk-1.1a2/bin/cfx", line 29, in <module>
    cuddlefish.run()
  File "/Users/jr/tools/addon-sdk-1.1a2/python-lib/cuddlefish/__init__.py", line 536, in run
    target_cfg = packaging.get_config_in_dir(options.pkgdir)
  File "/Users/jr/tools/addon-sdk-1.1a2/python-lib/cuddlefish/packaging.py", line 183, in get_config_in_dir
    set_section_dir(base_json, 'packages', path, ['packages'])
  File "/Users/jr/tools/addon-sdk-1.1a2/python-lib/cuddlefish/packaging.py", line 117, in set_section_dir
    resolved = compute_section_dir(base_json, base_path, dirnames, allow_root)
  File "/Users/jr/tools/addon-sdk-1.1a2/python-lib/cuddlefish/packaging.py", line 129, in compute_section_dir
    return os.path.join(base_path, base_json[dirname])
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/posixpath.py", line 65, in join
    if b.startswith('/'):
AttributeError: 'list' object has no attribute 'startswith'

It seems like the code in packaging.py is trying to normalize some paths, but does not work for array values, assumes string values only.

I would not mind having a stricter mapping, where I would specify "find this package over here at this path" for the exterior packages that are in use, but just offering that as an alternative suggestion to directory scanning. But directory scanning is fine too, just some mechanism for it.

If there is another way to accomplish what I am aiming to do, pointers in that direction are appreciated.

Comment 1

[Myk Melez [:myk] [@mykmelez]]

I can't recall if we intentionally removed the ability for this to be an array; if not, this is a regression, or at least a bug in the docs; either way, we should address it for 1.1.

Note also the --package-path option to cfx.

Comment 2

[Brian Warner [:warner :bwarner]]

"packages" is kind of weird. I've never quite understood how it's supposed to
work, especially the singular-vs-plural aspects: One doc claims it can be a
list, and "packages" (the property name) is plural. But the default value is
a single string, But that string is a plural name, "packages" (the directory
name). Is it supposed to point to packages? Or to a directory containing
packages? Or to a list of directories containing packages? Or is it supposed
to identify packages by *name* (as opposed to directoryname)?

So there's lots of opportunity for bugs. I'd rather simplify and offer fewer
choices than more, so my preference would be to change the docs to suggest
something more conservative. I'm not sure what that would be.. maybe only
offer a single string? Or just not document it at all?

But as Myk said, yeah, I think --package-path might be what you want. It's
not stored in the package.json, though, so it's not an exact replacement (you
have to add it to your Makefile or some other tool to control how 'cfx xpi'
is executed).

Comment 3

[Myk Melez [:myk] [@mykmelez]]

(In reply to Brian Warner [:warner :bwarner] from comment #2)
> But as Myk said, yeah, I think --package-path might be what you want. It's
> not stored in the package.json, though, so it's not an exact replacement (you
> have to add it to your Makefile or some other tool to control how 'cfx xpi'
> is executed).

You could use a command-line configuration:

https://addons.mozilla.org/en-US/developers/docs/sdk/1.0/dev-guide/addon-development/cfx-tool.html#configurations

That'd make invoking cfx with --package-path a bit easier, although still not as easy as the "packages" property.

Comment 4

[James Burke [:jrburke]]

--package-path works better, although it would be good to specify in the docs/command line that there needs to be a separate --package-path=path for each path you want to add.

I also did not see the option on this page:
https://addons.mozilla.org/en-US/developers/docs/sdk/1.0/dev-guide/addon-development/cfx-tool.html

but I do see it on the command line for 1.1b1.

So with the multiple --package-path args I can avoid any bootstrap errors, but then when trying to a require('common/test/unit-gendb') it cannot find the module.

I have a --package-path=../../common and it has a package.json in it, with the file living at common/test/unit-gendb.js. I have tried different combinations of "lib": ".", and "test" and tried to adjust the require name to "common/test/unit-genb" and "common/unit-gendb" respectively, but still could not find the module.

The weirdness may be because there is an additional --package-path=../../common/lib/rdcommon for another package. So, there is a package inside the "common" package.

This is not a blocker, we have a way around it either via symlinks or copying files directly into lib, just giving extra feedback.

Comment 5

[Andrew Sutherland [:asuth] (he/him)]

It appears the other magic ingredient is naming the module via "--extra-packages" when using "--package-path".  (Or already establishing the dependency elsewhere.)  This seems to be working for me now in a shell script:

cfx $EXTRA \
 --package-path ../../common \
 --extra-packages "addon-kit,rdcommon" \
 "$@"

Comment 6

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

(Pushing all open bugs to the --- milestone for the new triage system)

Comment 7

[Tomislav Jovanovic :zombie]

i edited the MDN page to specify it's a single directory.

so since this is a (3 year old) cfx issue, and this property doesn't make sense with native jetpack, and modules management is going to get handled via other means with jpm, i'm going to WONTFIX this.