1191604 - Spawn validation jobs for unvalidated files as soon as a review page is open

Status: VERIFIED FIXED

(addons.mozilla.org Graveyard :: Admin/Editor Tools, defect)

Description

[Kris Maglione [:kmag]]

If a reviewer opens a review page, they're almost certainly going to need validation results. We currently clear cached validation results every time we push, which means they're often not ready by the time an add-on is reviewed.

We should spawn validation jobs for an add-on as soon as its review page is open, so reviewers have to deal with less lag before starting a review.

Comment 1

[[github robot]]

Commit pushed to master at https://github.com/mozilla/olympia

https://github.com/mozilla/olympia/commit/ba17429432d7cb7283ed242b336360cfef328a97
Bug 1191604: Start validation jobs for unvalidated files when loading an add-on review page.

Comment 2

[Valentina Virlics [:vvalentina], Ecosystem QA]

I think maybe Bug 1193741 is a regression from this fix.
What do you think, Kris Maglione?

Comment 3

[Mathieu Agopian [:magopian]]

It is Valentina, I came up with a fix: https://github.com/mozilla/olympia/pull/656

Comment 4

[Kris Maglione [:kmag]]

This is a bit difficult to verify. STR are essentially:

1) Open the review page for an add-on without cached validation results.
2) Wait long enough for the validation to complete.
3) Open validation and file viewer pages.
4) Verify that validation results are loaded instantly, with no spinners in the validation page.

And, for safety:

1) Open the review page for another add-on without cached validation results.
2) Do not wait long enough for the validation to complete.
3) Open the validation and file viewer pages.
4) Verify that the validation page initially shows spinners, but loads the results once they complete.

The main problem is that I'm not sure how to find an add-on without cached validation results. We currently clear them on every push to prod, but I'm not sure about dev or stage. When testing locally, I just checked my DB to verify. I'm not sure what to do on our testing servers.

Comment 5

[Victor Carciu]

(In reply to Kris Maglione [:kmag] from comment #4)
> This is a bit difficult to verify. STR are essentially:
> 
> 1) Open the review page for an add-on without cached validation results.
> 2) Wait long enough for the validation to complete.
> 3) Open validation and file viewer pages.
> 4) Verify that validation results are loaded instantly, with no spinners in
> the validation page.
> 
> And, for safety:
> 
> 1) Open the review page for another add-on without cached validation results.
> 2) Do not wait long enough for the validation to complete.
> 3) Open the validation and file viewer pages.
> 4) Verify that the validation page initially shows spinners, but loads the
> results once they complete.
> 
> The main problem is that I'm not sure how to find an add-on without cached
> validation results. We currently clear them on every push to prod, but I'm
> not sure about dev or stage. When testing locally, I just checked my DB to
> verify. I'm not sure what to do on our testing servers.

I have the same issue, I am not sure how to find a an add-on without cached validation result :(

Comment 6

[Kris Maglione [:kmag]]

I've found a solution to that problem.

Look at the developer versions page for the add-on. Any version without cached validation results will have a 'Validate now' link.

Comment 7

[Victor Carciu]

(In reply to Kris Maglione [:kmag] from comment #6)
> I've found a solution to that problem.
> 
> Look at the developer versions page for the add-on. Any version without
> cached validation results will have a 'Validate now' link.

Hi, can you please give an example , because I can see this "Validate now" link everywhere

Comment 8

[Mathieu Agopian [:magopian]]

Not sure if it's relevant or linked, but I just saw this "validate now" link for my "tabsmack" addon in production, which was there for quite some time now. Clicking the link seems to validate it (again? or just display the cached validation?).

Comment 9

[Kris Maglione [:kmag]]

The 'Validate now' link will be on every existing version just after we push. When a new version is uploaded, it should wind up with cached validation, so no validate now links. I think it would suffice to open a review page, verify that the validate now link has disappeared, and then look at the review page for a different add-on to verify the changed behavior as in comment 4.

If you're having too much trouble testing this, I'd be OK marking it resolved as long as you're not seeing new issues opening any of those pages (aside from the server error in the validate now links, which I've confirmed isn't new, and am still working on). The unit tests are pretty thorough.

Comment 10

[Victor Carciu]

Verified as fixed based on previous comment(Validate now link has disappeared ). 
The server error discussed in comment #9 is still reproducible , but this is another fix.