746313 - Bulk Validation should finish a validation job even if there are exceptions

Status: RESOLVED WONTFIX

(addons.mozilla.org Graveyard :: Compatibility Tools, defect)

Description

[Jorge Villalobos [:jorgev] (he/him)]

It is now very common for the bulk validation process to stop short of finishing because of unexpected exceptions. See bug 728951 for a recent example. This requires me to poke Kumar or Wil to force-finish the process, which isn't very optimal.

There are 2 possible solutions I see for this:

1) Make the bulk validation job finish even if there are exceptions. They are reported in that page anyway.

2) Add a Force Finish button that shows up on unfinished jobs. This button would effectively kill the job if there's anything pending and activate the links in the Failing and Passing columns.

Comment 1

[Kumar McMillan [:kumar]]

This is kind of hard because currently each task event is responsible for completing the full run. When there is an exception we could maybe catch that early on or make the completion tally smarter. Alternatively, as mentioned above, we could provide a "mark as complete" button.

Until then, someone with DB access has to do: update validation_job set completed=NOW() where id=NN

Comment 2

[Wil Clouser [:clouserw]]

Is this still a bug with d2c?  I haven't done any poking for a long time.

Comment 3

[Jorge Villalobos [:jorgev] (he/him)]

Yes, we've had to manually enable the bump because of exceptions that prevented the job from finishing. The most recent was for Firefox 22, ~4 months ago.

Comment 4

[Kumar McMillan [:kumar]]

I think [maybe] the fix is to switch to a TaskTree, like this: https://github.com/mozilla/zamboni/blob/master/mkt/api/resources.py#L136-L138

Comment 5

[Rob Hudson [:robhudson]]

(In reply to Kumar McMillan [:kumar] from comment #4)
> I think [maybe] the fix is to switch to a TaskTree, like this:
> https://github.com/mozilla/zamboni/blob/master/mkt/api/resources.py#L136-L138

Except with Celery 3, which we're on now, we can build complex tasks with celery primitives and stop using TaskTree. See http://docs.celeryproject.org/en/latest/userguide/canvas.html

Comment 6

[Wil Clouser [:clouserw]]

Thanks for filing this.  Due to resource constraints we are closing bugs which we won't realistically be able to fix.  If you have a patch that applies to this bug please reopen.

For more info see http://micropipes.com/blog/2014/09/24/the-great-add-on-bug-triage/