Very large string literals in XPI package JavaScript files cause validator to fail unexpectedly


Status Graveyard
Add-on Validation
7 years ago
2 years ago


(Reporter: Zachary Johnson, Unassigned)



(Whiteboard: [streaming-json-parser], URL)


(4 attachments)



7 years ago
User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_7; en-us) AppleWebKit/533.21.1 (KHTML, like Gecko) Version/5.0.5 Safari/533.21.1
Build Identifier: current website

I'm using the Addon SDK (formerly JetPack) to make a JavaScript based plugin for Firefox 4.  The plugin runs perfectly from the command line with 'cfx run' and I can package it with 'cfx xpi'.

When I go to submit my XPI to the website ( the Validation steps fails with "Unexpected server error while validating."

I proceeded to delete all my custom JS from my plugin, recreate an XPI, and upload again, and then the validation was successful -- so I knew this was not a temporary server issue.

Through a process of commenting out certain parts of my plugin JS code, recreating the XPI, and reuploading/revalidating I narrowed the problem down to some very large string literals in my plugin JS code.

I have code in the form of:

var x = 'foobarfoobarfoobarfoobarfoobarfoobarfoobarfoobar' +
'foobarfoobarfoobarfoobarfoobarfoobarfoobarfoobar' +
'foobarfoobarfoobarfoobarfoobarfoobarfoobarfoobar' +
...many more lines line that ...;

If the concatenation of the strings being saved into the variable x exceed something greater than 25-35KB then the XPI validator on will fail with "Unexpected server error while validating."

However, if I break the > 30KB string into a few < 20KB strings... there are no problems.  So I can do:

var x = 'foo' + 'foo' + 'foo' + ... 15 more KB;
x += 'foo' + 'foo' + 'foo' + ... 15 more KB;
x += 'foo' + 'foo' + 'foo' + ... 15 more KB;

That works fine.  Again, however, if I just do:

var x = 'foo' + 'foo' + 'foo' + ... 40 more KB;

Now the validator will fail.

Reproducible: Always

Steps to Reproduce:
1. Create a plugin with the add-on SDK
2. Use the page-mod API to load in a JS file from your plugin's data folder with contentScriptFile
3. Add a JS string variable to the JS code as I described in the Details field of this bug report
4. Package as an XPI and submit to

Actual Results:  
The XPI uploads and validates, but you get the error:
"Unexpected server error while validating."

Expected Results:  
I should have received a less generic error message about where/how the validator failed.  The validator also should not have failed in the first place.

Comment 1

7 years ago
Zachary, do you mind attaching the add-on which generated this error? Thanks!

Comment 2

7 years ago
Created attachment 540077 [details]
Full source of my add-on

In this version of the source I've applied my fix to get past the validation failure.

Comment 3

7 years ago
Created attachment 540078 [details]
XPI with no problems

This XPI already fixed

Comment 4

7 years ago
Created attachment 540080 [details]
XPI WITH Problem

This XPI will create unexpected validation failure.
Worked fine for me on the command line:

$ ./addon-validator firebomb-plugin.xpi 

Detected type: Extension/Multi-Extension
Test failed! Errors:

Warning: Jetpack module hash unknown
Warning: Jetpack module hash unknown
Warning: Hidden files and folders flagged

Comment 6

7 years ago
This may be an issue with the prod python setup. The validator JSON output from the bad xpi has the following error

File \"/usr/lib/python2.6/json/\", line 55, in iterscan
    rval, next_pos = action(m, context)
File \"/usr/lib/python2.6/json/\", line 68, in JSONNumber
    res = fn(integer)
RuntimeError: maximum recursion depth exceeded while calling a Python object

Output cleaned up a little to make readable
Ever confirmed: true

Comment 7

7 years ago
Created attachment 540101 [details]
output from bad xpi
My output was from khan which is supposed to mirror production
Component: Developer Pages → Add-on Validation
QA Contact: developers → add-on-validation

Comment 9

6 years ago
This problem should have been fixed when we switched from `json` to `simplejson`. If it hasn't been, then this is another reason why we need a streaming JSON parser.
Last Resolved: 6 years ago
Resolution: --- → FIXED
Whiteboard: [streaming-json-parser]
Don't json and simplejson trivially parse JSON from streams? Or do you mean something SAX-like?

Comment 11

6 years ago
SAX-like. We need something that's not based on recursion.


2 years ago
Product: → Graveyard
You need to log in before you can comment on or make changes to this bug.