839691 - Add-on Validator spurious CSS resource warnings

Status: RESOLVED INCOMPLETE

(addons.mozilla.org Graveyard :: Add-on Validation, defect, P4)

Description

[Aris]

Attachment: noia4v178.xpi & noia4v179.xpi

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0
Build ID: 20130206083616

Steps to reproduce:

Download Noia4 v1.7.8 and v1.7.9 (attachment) and upload both to "Add-ons Validator".

v1.7.8 (non-jared theme file) fails.

v1.7.9 (jared theme file) passes.




Actual results:

If the uploaded file is structured like

noia4v178.xpi\noia4.xpi\chrome\ ... theme files
noia4v178.xpi\noia4tc.xpi\ ... extension files

uploading multi package (noia4v178.xpi: theme + add-on) shows "server" error,
uploading main theme file from multi package (noia4.xpi) shows "@namespace..." warnings.


If the uploaded file is structured like

noia4v179.xpi\noia4.jar\chrome\noia4.jar\ ... theme files
noia4v179.xpi\noia4tc.xpi\ ... extension files

uploading multi package (noia4v179.xpi: theme + add-on) shows no "server" error,
uploading main theme file from multi package (first noia4.jar) shows no "@namespace..." warnings


Expected results:

If the uploaded file is structured like

noia4v178.xpi\noia4.xpi\chrome\ ... theme files
noia4v178.xpi\noia4tc.xpi\ ... extension files

uploading multi package (noia4v178.xpi: theme + add-on) should not show "server" errors,
uploading main theme file from multi package (noia4.xpi) should not show "@namespace..." warnings.

Comment 1

[Jorge Villalobos [:jorgev] (he/him)]

This does appear to be a regression in the validator. See this file, for example: https://addons.mozilla.org/developers/addon/silvermel/file/190726/validation

It passed validation a few days ago when it was submitted. When I ran validation during review today, it failed with a server error. It works now that I just tested it, so it might be a timeout problem.

Comment 2

[Matt Basta [:basta]]

This doesn't seem to be a problem with the validator code itself, as both of those XPIs validate fine locally. Neither takes more than a couple seconds to run, so I doubt it's a timeout issue. It could be an issue with the celery servers, though.

CCing jason and oremj.

Comment 3

[Kris Maglione [:kmag]]

Uploading it to the standalone validator, it spins for a while and then returns with "There was a problem contacting the server." Which is odd.

Comment 4

[support]

The "cannot contact to the server" is likely a red herring.  Sharebird was able to update his theme without a problem http://forums.mozillazine.org/viewtopic.php?p=12648827#p12648827 The problem seems to have started with https://bugzilla.mozilla.org/show_bug.cgi?id=664823 

This is the thread at MozillaZine that brought this to light:  http://forums.mozillazine.org/viewtopic.php?f=18&t=2654789 Please read this in the light that there is good information here but the themers are **** off at Mozilla for good reason.

Comment 5

[support]

"... the themers are annoyed at Mozilla for good reason"  Sorry about the P word.

Comment 6

[Matt Basta [:basta]]

If validation is nondeterministic, it is a server problem (I.e. timeouts, memory issues, queue/rabbit issues). The validation process itself is side-effect free (and single-threaded), so it's not possible for multiple runs on the same file to produce different results.

With regard to the @namespace issue, that's an edge-case that I missed in writing that test case. Please reopen the bug if you're feeling scandalized by it. I'll have a look this weekend one way or the other.

I've been having some server issues myself, but all of the problems that I've faced have been on Dev and have been somewhat temporary. On Monday, one of the ops can have a look and see if there's anything that's immediately obvious.

Comment 7

[support]

I don't feel scandalized about it at all. Sorry, I was simply pointing out that there was a good reason to think that the bug fix caused the problem.  I honestly didn't realize that I was getting into a touchy subject here.  My apologizes.

Comment 8

[Kris Maglione [:kmag]]

I'm not aware of anyone getting into a touchy subject.

Bug 664823 was file at the request of a theme developer who noticed that a theme he was testing was loading remote images, which everyone present agreed was a bad idea. It was certainly not in any way designed to persecute theme developers. There also isn't any evidence to suggest that it's related to this bug.

Comment 9

[Kris Maglione [:kmag]]

*filed

Comment 10

[Matt Basta [:basta]]

I can't reproduce this on -dev or prod. Attributing this to celery/rabbit issues from two weeks ago, please reopen if the problem persists.

Comment 11

[Aris]

Problem with multipacked files was solved about 10 days ago. Only "@namespace..." warnings still exist, but all themes are affected by them. Any chance this get fixed in the future?

Comment 12

[Matt Basta [:basta]]

Can you provide an example snippet? The following tests are showing that it should be ignored now, but if that's not the case, then I'm probably looking at the wrong examples:

https://github.com/mozilla/amo-validator/commit/c4ab6760c0af0210f76babb065ba488c16e1083b#L0R56

Comment 13

[Jorge Villalobos [:jorgev] (he/him)]

I saw those false positives for most themes I looked at today. See for example:
https://addons.mozilla.org/en-US/developers/addon/noia-4/file/194038/validation

The file is the latest version of this theme:
https://addons.mozilla.org/en-US/firefox/addon/noia-4/

Comment 14

[Matt Basta [:basta]]

I don't have permissions to look at those results. Can you pastebin it?

Comment 15

[Aris]

They look like this: http://pastebin.com/kAMiVTTd

Comment 16

[Kris Maglione [:kmag]]

(In reply to Matt Basta [:basta] from comment #14)
> I don't have permissions to look at those results. Can you pastebin it?

The results are the same on -dev: https://addons-dev.allizom.org/en-US/developers/addon/noia-4/file/180154/validation

Comment 18

[Jorge Villalobos [:jorgev] (he/him)]

I suppose this was meant to be reopened at some point.

Comment 19

[Jorge Villalobos [:jorgev] (he/him)]

Here's a new sample, in case the other ones aren't working: https://addons.mozilla.org/en-US/developers/addon/firefox-2-theme-for-firefox-3x/file/274368/validation