Analyze binary add-ons from




JavaScript Engine
6 years ago
6 years ago


(Reporter: billm, Assigned: billm)


Firefox Tracking Flags

(Not tracked)



(3 attachments)



6 years ago
Dave Herman gave me a big tarball with all the addons. The high-level breakdown in as follows:

  Valid: 9129, non-zip: 56, corrupted: 20, binary: 51

This means that out of 9129 addons, 51 are binary (0.56%). I looked at all the Linux and Windows binaries, and it looks like none of them use JSAPI! To make sure I didn't make a mistake, I did the same analysis on the Kikin addon, which is not on AMO and which is known to use JSAPI. It does indeed import a bunch of JSAPI entry points.

I'll post more detailed results here. Also, I have a zip file with all the binaries extracted if anyone wants to examine them.
Good to hear. It sounds like for the most part when we change JSAPI we just need to put out the word ahead of time on m.d.t.js-engine and the AMO channels.

Comment 2

6 years ago
Created attachment 547298 [details]
script to find binary addons

This is the script I used to process all the addons. The first argument is a path to the directory containing all the addons. The second argument is a fresh directory where it will put all the dlls it finds.

It turns out that a bunch of them are messed up, so most of the code deals with that. For some reason some of the .xpi files actually contain HTML. Also, some of the zip files are reported as corrupted by unzip. I skipped over all these, although the ones in the latter category could probably be recovered.
Assignee: general → wmccloskey

Comment 3

6 years ago
Created attachment 547300 [details]
output of Python script

This is the output of the analysis script. It just shows which xpi files were skipped. Note that some of them have references in their chrome manifest to files that don't exist. Mostly this seems to happen when a Mac addon claims to have a .dll or a .so, but it really only has a .dylib. Or some variation of that.

Comment 4

6 years ago
Created attachment 547302 [details]
list of symbols imported by each addon

This is the result of dumpbin.exe -IMPORTS foo.dll (thanks khuey). As can be seen, mozjs.dll does not appear.

I also ran a Windows version of strings on each dll and grepped for service names, of the form "@xyz;1". A lot of addons used some sort of xpcom proxying service, which seems to have something to do with threads. One addon (cooliris) wanted the XPCOM context stack service. I don't know enough to say if this matters. I can upload the list of anyone is interested. (I left it on my Windows partition by accident.)
9129 seems small to me -- I thought there were somewhere between 15k and 20k addons. Also, the XPI's that contained HTML might possibly be the result of a failed wget in Wil Clouser's script that collected all the addons from AMO.

CC'ing Wil and Matt Basta, who does the AMO validator. Wil & Matt: do you think this suggests that the tarball might not be good? Could you guys help Bill and me make sure we can get an accurate snapshot of the full set of AMO addons?


Comment 6

6 years ago
I'll run this through our stats tool, Grizwald, and get back to you tomorrow.

Comment 7

6 years ago
Another odd thing is that, out of ~12,000 directories in the tarball, only about 9,000 have xpi files in them.

Comment 8

6 years ago
Themes are packaged in JAR files. That might account for the other chunk.

Comment 9

6 years ago
The failed wget seems quite possible. How can we figure that out?
There is no wget.  I ran the script on a box with the add-ons on an NFS mount.  The script is super simple, it just makes a best guess on what the latest file is and copies it:

Comment 11

6 years ago
I'm going to mark this as FIXED, I guess, since the work is done.
Last Resolved: 6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.