1064610 - Try to detect library files using file names and headers

Status: RESOLVED INCOMPLETE

(addons.mozilla.org Graveyard :: Add-on Validation, defect, P2)

Description

[Jorge Villalobos [:jorgev] (he/him)]

We currently use checksums to detect known JS libraries, but that leads to many problems because files can easily have their encoding changed, breaking the checksum and leading us to reject add-ons can cause developers lots of wasted time and grief. We can do better.

Library files tend to have recognizable names that can be caught with regular expressions, and predictable headers even in their minified forms, like

/*! jQuery v1.11.1 | (c) 2005, 2014 jQuery Foundation, Inc. | jquery.org/license */

//     Underscore.js 1.7.0

I propose we try to detect known library versions by they file names and/or headers, and then compare a hash of their *source code* against the original. That means we would store a second hash of the contents of the library files so we can perform this comparison.

Comment 1

[erosman [:erosman]]

IMHO parsing the files (often 10,000+ lines) for names and other data would greatly complicate the process.

The hash system works. It needs a bit of tweaking to get it optimized.

Comment 2

[Jorge Villalobos [:jorgev] (he/him)]

We already parse the files for more complex patterns. The problem of developers or programs changing the encoding of these files is frequent enough that I think it warrants a more robust detection mechanism.