688759 - Bulk compatibility tool ignored binary XPCOM components in 6.* -> 7.* bump

Status: VERIFIED FIXED

(addons.mozilla.org Graveyard :: Compatibility Tools, defect, P1)

Description

[Jorge Villalobos [:jorgev] (he/him)]

We have received notifications from 2 add-on developers (WebRunner and Input Language Assistant) that say that their add-ons were incorrectly upgraded in the latest compatibility bump.

They both have binary XPCOM components in them, which is something the compatibility tool should be checking for, and I thought worked correctly in previous compatibility bumps.

We need this fixed ASAP. We are planning a 7 -> 8 bump for next week and a 8 -> 9 bump for the following week.

Comment 1

[Andrew Williamson [:eviljeff]]

From a quick look at the thunderbird log [https://addons.mozilla.org/z/en-US/admin/email_preview/validation_job-39-success.csv] and seeing an addon on it (FireTray - which I recognised as containing a XPCOM binary) I'd guess the Thunderbird run was incorrect also.

Comment 2

[Mark Banner (:standard8)]

That's a surprise, because I believe that Lightning was detected as not compatible - unless there was something else that it had been incompatible for. Unfortunately the logs don't tell you the results for the addons, and unless you're the add-on author you can't access the specific results.

Comment 3

[Jorge Villalobos [:jorgev] (he/him)]

(In reply to Mark Banner (:standard8) from comment #2)
> That's a surprise, because I believe that Lightning was detected as not
> compatible - unless there was something else that it had been incompatible
> for. Unfortunately the logs don't tell you the results for the addons, and
> unless you're the add-on author you can't access the specific results.

I can see Lightning in the failure log, which means it was validated even though it has binary components. You can see the validation link in the log. Here it is: https://addons.mozilla.org/en-US/developers/addon/lightning/validation-result/68848

Comment 4

[Mark Banner (:standard8)]

(In reply to Jorge Villalobos [:jorgev] from comment #3)
> I can see Lightning in the failure log, which means it was validated even
> though it has binary components.

Ah right.

> You can see the validation link in the log.
> Here it is:
> https://addons.mozilla.org/en-US/developers/addon/lightning/validation-result/68848

I get nothing when I try to load that link, even though I'm logged in.

Comment 5

[Andrei [:sancus]]

I can fix this for you guys, I know what the problem is. One sec.

Comment 6

[Andrei [:sancus]]

I have patch for this https://github.com/Sancus/amo-validator/commit/69046421bccd1f0b894296b2335dace7e7eeaa88 that should fix it forever, but I'm not too sure how to write a test for it, or alter the existing test so that it accounts for version strings. I emailed Matt Basta and hopefully he'll respond with some insight over the weekend.

Comment 7

[Kumar McMillan [:kumar]]

The fix is in:

validator: https://github.com/mozilla/amo-validator/commit/5a382ea9622521dbfdf5cbd558ffde787f28cfdd
zamboni: https://github.com/jbalogh/zamboni/commit/10263e9

Comment 8

[krupa raj[:krupa]]

In job #38, add-ons with binary components fail validation as expected. See

https://addons-dev.allizom.org/en-US/developers/addon/dock-progress/validation-result/67969
https://addons-dev.allizom.org/en-US/developers/addon/snagit-autoscroll-helper/validation-result/68658

We filed bug 690435

NOTE: I haven't verified all the failures. I just picked a few samples.