592270 - Non-ASCII characters are not possible in author field

Status: RESOLVED DUPLICATE of bug 567660

(Add-on SDK Graveyard :: General, defect)

Description

[Mathias Panzenböck]

User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.8) Gecko/20100724 Remi/fc12 Firefox/3.6.8
Build Identifier: 0.7

If you enter a non-ascii character in the author field cfx xpi crashes. Therefore I have to write "Mathias Panzenboeck" instead of my real name "Mathias Panzenböck". I tried using a simple UTF-8 "ö" as well as "\u00f6". Both give:

Exporting extension to view-image-plus.xpi.
Traceback (most recent call last):
  File "/home/panzi/src/firefox-add-ons/jetpack-sdk-0.7/bin/cfx", line 6, in <module>
    cuddlefish.run()
  File "/home/panzi/src/firefox-add-ons/jetpack-sdk-0.7/python-lib/cuddlefish/__init__.py", line 561, in run
    xpts=xpts)
  File "/home/panzi/src/firefox-add-ons/jetpack-sdk-0.7/python-lib/cuddlefish/xpi.py", line 10, in build_xpi
    open('.install.rdf', 'w').write(str(manifest))
  File "/home/panzi/src/firefox-add-ons/jetpack-sdk-0.7/python-lib/cuddlefish/rdf.py", line 11, in __str__
    self.dom.writexml(buf, encoding="utf-8")
  File "/usr/lib64/python2.6/xml/dom/minidom.py", line 1749, in writexml
    node.writexml(writer, indent, addindent, newl)
  File "/usr/lib64/python2.6/xml/dom/minidom.py", line 817, in writexml
    node.writexml(writer,indent+addindent,addindent,newl)
  File "/usr/lib64/python2.6/xml/dom/minidom.py", line 817, in writexml
    node.writexml(writer,indent+addindent,addindent,newl)
  File "/usr/lib64/python2.6/xml/dom/minidom.py", line 817, in writexml
    node.writexml(writer,indent+addindent,addindent,newl)
  File "/usr/lib64/python2.6/xml/dom/minidom.py", line 1036, in writexml
    _write_data(writer, "%s%s%s"%(indent, self.data, newl))
  File "/usr/lib64/python2.6/xml/dom/minidom.py", line 303, in _write_data
    writer.write(data)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 15: ordinal not in range(128)

I've attached a patch that works for me. Don't ask me why it works when you do it like in the patch but not like you did. It seems to be a bug in minidom.

Reproducible: Always

Steps to Reproduce:
1. Write a unicode character into the autor field in package.json
2. run `cfx xpi`
Actual Results:  
UnicodeEncodeError

Expected Results:  
Correct XPI containing a UTF-8 install.rdf

Comment 1

[Mathias Panzenböck]

Attachment: fix unicode charcaters in package.json

Comment 2

[Drew Willcoxon :adw]

Hi panzi, thanks for reporting this, and for the patch.  There's already a bug filed for a similar issue: bug 567660.  It looks like the same issue actually.  There's a patch attached to that bug that hasn't been checked in yet but will be soon, so could you try to apply it to see if it fixes your problem?

Comment 3

[Drew Willcoxon :adw]

Speak of the devil, bug 567660 was just checked in, so please pull the repo again and see if the problem is fixed.  I think it should be.

Comment 4

[Mathias Panzenböck]

Yes this also fixes the issue.

However, are you sure that the different behaviour between StringIO and cStringIO is really intentional and can be expected to stay that way? Well I guess if anything changes cStringIO might in future behave more like StringIO, but is that guaranteed? Is there a problem with dom.toxml(encoding="utf-8")? It is simpler than the StringIO handling.

Comment 5

[Drew Willcoxon :adw]

I don't know, could you ask the people who worked on the patch in bug 567660?

Comment 6

[Myk Melez [:myk] [@mykmelez]]

The Add-on SDK is no longer a Mozilla Labs experiment and has become a big enough project to warrant its own Bugzilla product, so the "Add-on SDK" product has been created for it, and I am moving its bugs to that product.

To filter bugmail related to this change, filter on the word "looptid".