504930 - Encoding error in JSON error message when uploading invalid xpi

Status: VERIFIED FIXED

(addons.mozilla.org Graveyard :: Developer Pages, defect)

Description

[Joachim Herb]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.1) Gecko/20090624 Firefox/3.5
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.1) Gecko/20090624 Firefox/3.5

I tried to upload an extension (getpartialmessages-0.1.xpi). I guess the frist trail somehow did not work. When I try to upload the same file, I get an error message:
This add-on-ID already exists...

So I renamed the file to getpartialmessages-0.1.1.xpi and tried to upload again:


Hoppla! Mit dieser Datei scheint es ein Problem zu geben...

Diese Add-on-ID ({2d19fa6f-da0b-4677-b73f-53934866de8a}) existiert bereits in der Datenbank. Falls dies Ihr Add-on ist, möchten Sie wahrscheinlich eine neue Version hochladen.

Bitte beheben Sie dieses Problem und laden Sie Ihre Datei erneut hoch.

(sorry for the German version. I were not able to get an English error message. By the way there seems to be a problem with the encoding)

Now I tried tied the update link ("neue Version"). This resulted in this error message (access denied):

Zugriff verweigert

Sie haben nicht die erforderlichen Berechtigungen, um diese Seite angezeigt zu bekommen.


Reproducible: Always

Comment 1

[Joachim Herb]

Attachment: This is the add-on I try to upload.

Comment 2

[Fred Wenzel [:wenzel]]

We've seen this kind of error a few times now. It seems, when an add-on is broken a specific way, it gets created yet the author does not get any access rights to it. This is a bug in the Dev CP and we should figure out where it occurs, so an add-on is either not created or if it is, the author can access it properly in spite of the error.

Naturally, we also need to help Herb with his particular add-on.

Comment 3

[fcp]

There are several related problems.

Your install.rdf contains <em:homepageURL> elements twice.  From some experimenting, I am sure that that is the reason your first attempt failed; or more precisely, half-succeeded and half-failed.  That is worse than a complete failure because now the ID of your extension has been registered on addons.mozilla.org but you cannot access it.  This is the reason the subsequent attempts failed.  Because of this, you will not probably be able to upload it even after you remove the second <em:homepageURL> element from your install.rdf.

The easiest workaround that I can think of is to change an ID.  Make sure you change all the occurrences of the ID, not only in install.rdf.

Comment 4

[fcp]

Just to make sure, “ID” in my comment 3 refers to the add-on ID, which is {2d19fa6f-da0b-4677-b73f-53934866de8a} in your case.

Comment 5

[fcp]

(In reply to comment #3)
> The easiest workaround that I can think of is to change an ID.  Make sure you
> change all the occurrences of the ID, not only in install.rdf.

This might not be a good idea if you have already been distributing this add-on somewhere else, because existing users cannot update the add-on easily if you change the add-on ID.

Comment 6

[fcp]

(In reply to comment #2)
> Naturally, we also need to help Herb with his particular add-on.
I agree.

(In reply to comment #3)
> Your install.rdf contains <em:homepageURL> elements twice.  From some
> experimenting, I am sure that that is the reason your first attempt failed;
It seems that this was caused not by having two <em:homepageURL> elements, but by having an empty <em:homepageURL> element.

Comment 7

[Joachim Herb]

(In reply to comment #6)
> (In reply to comment #3)
> > Your install.rdf contains <em:homepageURL> elements twice.  From some
> > experimenting, I am sure that that is the reason your first attempt failed;
> It seems that this was caused not by having two <em:homepageURL> elements, but
> by having an empty <em:homepageURL> element.
Thank you for your help: I removed the empty <em:homepageURL> element and now it worked. 

So it works for me.

Nevertheless an error message would have been helpful. Even if it only stated that the install.rdf contains invalid information.

(And just a reminder about the encoding problem of the original error message.)

Comment 8

[Fred Wenzel [:wenzel]]

Did you change the ID? Otherwise I can probably indeed give you access to the other, orphaned add-on.

Comment 9

[Fred Wenzel [:wenzel]]

(In reply to comment #7)
> Nevertheless an error message would have been helpful. Even if it only stated
> that the install.rdf contains invalid information.

Absolutely, that is what's usually supposed to happen, but apparently in this case it doesn't.

> (And just a reminder about the encoding problem of the original error message.)

I cannot confirm the encoding issue here: Our page is completely UTF-8, and the string seems to be fine. Have you set up your browser to enforce ISO8859-1 encoding?

Comment 10

[Joachim Herb]

(In reply to comment #8)
> Did you change the ID? Otherwise I can probably indeed give you access to the
> other, orphaned add-on.

I did change the ID.

(In reply to comment #9)
> I cannot confirm the encoding issue here: Our page is completely UTF-8, and 
> the string seems to be fine. Have you set up your browser to enforce 
> ISO8859-1 encoding?

I tried to upload the same version of the addon again, to reproduce the encoding error and I got the same result. I will add an attachment with a screenshot. Below is the source code of the relevant part of the page. As you can see only the text in the box seems to use the wrong encoding. I tested the upload with Firefox 3.5.1/Windows Vista and Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.11) Gecko/2009062818 Gentoo Firefox/3.0.11 - Build ID: 2009062818



<div id="file-upload">
            <h3>Datei hochladen</h3>
            <p>Laden Sie die Datei für Ihr Add-on mit dem unten stehenden Formular hoch. Falls Sie mehrere, plattform-spezifische Dateien haben, wählen Sie zunächst eine der Dateien und laden Sie die übrigen auf der "Versionen und Dateien"-Seite hoch.            <br>
            </p><form id="upload-form" method="post" enctype="multipart/form-data" action="/de/developers/json/fileupload/update" target="upload-frame" onsubmit="return upload.uploadFile();">
            <div class="hsession"><input name="sessionCheck" value="7cf86da3a4d4f90b27ab77d82c82480d" type="hidden"></div>            <input name="data[Addon][id]" value="13131" type="hidden">                        <label>Add-on-Datei: <input id="upload-field" name="file" type="file"></label><br>
        Unterstützte Betriebssysteme:&nbsp;&nbsp;<label><input name="platformtype" onclick="upload.platformAll();" checked="checked" type="radio">&nbsp;Alle</label>&nbsp;&nbsp;

            <label><input name="platformtype" onclick="upload.platformSpecific();" type="radio">&nbsp;Spezifisch:</label>
            &nbsp;&nbsp;<label class="specific-platforms disabled"><input name="data[File][platform_id][]" value="4" disabled="disabled" type="checkbox">BSD</label>&nbsp;&nbsp;<label class="specific-platforms disabled"><input name="data[File][platform_id][]" value="2" disabled="disabled" type="checkbox">Linux</label>&nbsp;&nbsp;<label class="specific-platforms disabled"><input name="data[File][platform_id][]" value="3" disabled="disabled" type="checkbox">MacOSX</label>&nbsp;&nbsp;<label class="specific-platforms disabled"><input name="data[File][platform_id][]" value="6" disabled="disabled" type="checkbox">Solaris</label>&nbsp;&nbsp;<label class="specific-platforms disabled"><input name="data[File][platform_id][]" value="5" disabled="disabled" type="checkbox">Windows</label>            <div class="submit">
                <input value="Datei hochladen" class="require-privs" type="submit">
            </div>
            <div style="display: none;" id="upload-loading"><img src="/img/ajax_loading.gif">&nbsp;Datei wird hochgeladen...</div>

            </form>
            <div style="display: block;" id="upload-error" class="redbox-tr"><div class="redbox-tl">
                <div id="upload-error-content">
                    <h4>Hoppla! Mit dieser Datei scheint es ein Problem zu geben...</h4>
                    <p id="upload-error-text" class="smallmargin">Die hochgeladene Version (0.1.2) existiert bereits für dieses Add-on. Wenn Sie eine weitere Datei zu dieser Version hinzufügen möchten, <a href="/de/firefox/developers/versions/addfile/73756">klicken Sie hier</a>.</p>
                    <p class="smallmargin">Bitte beheben Sie dieses Problem und laden Sie Ihre Datei erneut hoch.</p>

                </div>
                <div class="redbox-br"><div class="redbox-bl"></div></div>
            </div></div>
        </div>

Comment 11

[Joachim Herb]

Attachment: Screenshot with encoding error in error message.

Comment 12

[Joachim Herb]

Attachment: Another error message with wrong encoding.

Comment 13

[Wil Clouser [:clouserw]]

I'm assigning this to Fred just to figure out the encoding problems.  If that's something only in production, we can file an IT bug.  If it's on herb's machine we can wontfix, or if there is something we can do he can fix it.

The error messages are another bug (should be rolled into our add-on parser) and don't block closing this bug.

Comment 14

[Fred Wenzel [:wenzel]]

I have a patch ready for this, using json_encode() to encode our JSON properly and not screw up utf-8 anymore. This requires PHP 5.2.0 or later.

Comment 16

[Fred Wenzel [:wenzel]]

Attachment: Patch, rev. 1

This fixes the JSON encoding and it requires PHP 5.2.0. To test, upload a broken .xpi (invalid install.rdf will do the trick) in, for example, German, French or Japanese and note the non-ASCII characters not being screwed up.

Comment 17

[Justin Scott [:fligtar]]

Comment on attachment 392896 [details] [diff] [review]
Patch, rev. 1

>-    upload.response = JSON.parse(document.getElementById('upload-frame').contentWindow.document.getElementById('json').innerHTML);
>+    upload.response = JSON.parse($('#upload-frame').contents().find('#json').text());

The response often includes HTML, like links.

Comment 18

[Fred Wenzel [:wenzel]]

(In reply to comment #17)
> (From update of attachment 392896 [details] [diff] [review])
> >-    upload.response = JSON.parse(document.getElementById('upload-frame').contentWindow.document.getElementById('json').innerHTML);
> >+    upload.response = JSON.parse($('#upload-frame').contents().find('#json').text());
> 
> The response often includes HTML, like links.

That should be fine, considering the HTML is wrapped inside a JSON object (I am treating the entire object as plain text here).

Actually using the html content (as html) happens a few lines later:
-        $('#upload-error-text').html(urldecode(upload.response.error_message));
+        $('#upload-error-text').html(upload.response.error_message);

Comment 19

[Joachim Herb]

I don't know why, but I tried to add another addon and the original problem happened again. After the upload could not add an project description.

I do not see the addon in my list. When I tried to upload the file again, I got an errormessage, that the version already exists and I should add a new version with this link:
https://addons.mozilla.org/de/firefox/developers/versions/add/13546
But there I get an error message saying that the access was denied.

Can somebody with the propper rights please reset this addon, so I can upload it again. If possible I would like to avail to change the addon id.

Here the error message:

Hoppla! Mit dieser Datei scheint es ein Problem zu geben...

Diese Add-on-ID ([e-mail]) existiert bereits in der Datenbank. Falls dies Ihr Add-on ist, möchten Sie wahrscheinlich eine neue Version hochladen.

Bitte beheben Sie dieses Problem und laden Sie Ihre Datei erneut hoch.


Oops! There seems to be a problem with this file...

This add-on ID ([e-mail]) already exists in the database. If this is your add-on, you can upload a new version.

Please correct this problem and upload your file again.

--
Access Denied

You are not authorized to view this page.
--

Comment 20

[Joachim Herb]

Attachment: Addon cannot be uploaded.

Comment 21

[Wil Clouser [:clouserw]]

Let's not commit this until PHP is upgraded

Comment 22

[fcp]

(In reply to comment #19)
> Can somebody with the propper rights please reset this addon, so I can upload
> it again. If possible I would like to avail to change the addon id.

> This add-on ID ([e-mail]) already exists in the database. If
> this is your add-on, you can upload a new version.

I do not know who are allowed to use add-on IDs with the mozilla.org domain.  Assuming that you are, and assuming that no one with the necessary permissions has responded yet, I suggest you to either
- ask on irc.mozilla.org (I am not sure which channel, but probably #amo, possibly #addons), or
- file a separate bug,
because this bug has been morphed specifically to deal with the problem with the encoding of the error message.

Comment 23

[Joachim Herb]

I opened a new bug 509211 for the problem of uploading a broken install.rdf and the following blockage of the addon ID.

Comment 24

[Wil Clouser [:clouserw]]

wenzel: PHP is upgraded on preview, let's land this.

Comment 25

[Fred Wenzel [:wenzel]]

r49807

Comment 26

[Stephen Donner [:stephend] Not actively reading bugmail]

Herb, is this something you could help us out with by verifying on https://preview.addons.mozilla.org?  Thanks!

Comment 27

[Joachim Herb]

WFM

Comment 28

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED per comment 27.