Closed Bug 1030972 Opened 5 years ago Closed 5 years ago

Add ability to auto-import tasks from Bugzilla

Categories

(Mozilla QA Graveyard :: One and Done, defect, P1)

Version 2
x86
macOS
defect

Tracking

(Not tracked)

VERIFIED FIXED
Milestone 2

People

(Reporter: rbillings, Assigned: maja_zf)

References

Details

Attachments

(3 files, 1 obsolete file)

Add ability to import tasks from a bugzilla query, that can add tasks to One and Done from a cron job or something similar.
Blocks: 1032881
Liz and I came up with the following requirements.
- A form in which administrator can specify:
-- Data source (for now, the source is just Bugzilla and admin should specify a query; future sources include Moztrap)
-- Task expiration criteria (e.g. a task should become automatically unavailable when the bug it refers to is no longer unconfirmed and untriaged)
-- All the other fields that are needed to create a task. 

Based on this information, when the admin clicks a "Create Tasks" button, the tool automatically creates one task for each bug in the Bugzilla query at that moment. Every task will have largely the same attributes, as specified in the form; each task title should be different (e.g. "Triage Bug [Bug number] [Bug Summary]).

Other notes:
- Admin should be able to refer to some attributes of the bugzilla bug in task fields so they are automatically populated with the right info from the bug (bug summary, bug number)
- There should be a limit on the number of tasks per batch: we want to shield users from chaos and stale information as much as possible and therefore encourage admins to select tasks with care.
- For the same reason, a task end date should be *required* for autogenerated tasks.
- In our meeting, we called this a "batch" of tasks. There's probably a better name. Suggestions?
-- In the future (separate bug), we might allow admins to  process such a batch of tasks once a week, for example to populate One and Done with a few of the newest untriaged Firefox bugs.)
It might be useful to have a preview/confirmation step.

"The following 10 tasks will be created. Are you sure?"
I'm not sure I agree that a task end date should be required. How can the end date be selected for a bug query? Is it an arbitrary date [ex: 3 months from creation]? I understand not wanting to create lots of tasks which never are completed, but if they are based on live bugs it will take care of itself [assuming the bugs are worked on by any other person in or out of O&D].
Flags: needinfo?(mjzffr)
Liz can give a more definitive answer from a task owner's perspective.

Here's some of the rationale for the required end-date:
* Bugs that remain untriaged for a "long" time are likely not good for beginners to triage. 
* Unconfirmed bugs that remain unconfirmed have likely been overlooked (?) so they're not good for beginners either.

For other kinds of bugs, I'm not sure. I think the idea is "indicate how long you are willing to actively mentor beginners on this bug," or "how long show this bug be shown in One and Done before you should reassess its value to beginners." Any bug-based task that's around for say, 3 months, should probably not be shown to One and Done users anymore and should be reassessed by the task owner. Maybe we should even enforce a maximal end date, like 3 months. 

[Task owners might be too busy to keep their tasks up-to-date and fresh, especially imported tasks: it's better make certain kind of tasks unavailable after some time until task owners revisit them than to display possibly out-of-date/confusing tasks until task owners revisit them. Less is more?]
Flags: needinfo?(mjzffr) → needinfo?(lhenry)
Maja, would you be able to focus your efforts on this bug? You seem to have already put a lot of thought into it. If you want to come up with some design ideas we can discuss them together.
Flags: needinfo?(mjzffr)
Sounds good. I'll start looking into it.
Assignee: nobody → mjzffr
Flags: needinfo?(mjzffr)
Rebecca, Maja described it pretty well. I think that some of the batch tasks will be finished and thus won't hit their expiration date -- but others may linger past the point where they'd be a good beginner task. The bugs that stay "untriaged" or unconfirmed for months, with 20+ comments back and forth with the bug reporter and QA, aren't what I want to direct one and done users to.   For other possible batch tasks like bug verification, bugs may never get verified, but may still go out of date for our purposes. (For example, it isn't all that useful to ask someone to verify a bug fixed in Firefox 26 at this point.) 

If we figure that in future we will add a feature to clone a batch task, that should help make the process easier.
Flags: needinfo?(lhenry)
I made an initial PR just for the model changes. Could you let me know what you think, Bob? Thanks!
Attachment #8458090 - Flags: feedback?(bob.silverberg)
Comment on attachment 8458090 [details]
Link to Github pull-request: https://github.com/mozilla/oneanddone/pull/162/files

Feedback received. (Thanks, Bob.)
Attachment #8458090 - Flags: feedback?(bob.silverberg)
Up-to-date summary of requirements based on discussion over the past few weeks and reflected in my latest PR: 
* The "Import Task Batch" view is accessible at '/tasks/import' and via a button at the top of '/tasks/create'.
* The form in this view includes all the same fields as the "Add Task" form, as well as a few extra fields at the top: batch summary, query URL, invalidation criteria.
* Based on the "query URL", which is a link to any search-results page from BMO, the tool creates one task for each bug in the BMO search results.
** Every task has the same properties except "Title", which is customized with the number of the associated bug.
** The tool will create at most 20 tasks. The user may fill out the "Import Task Batch" form again with the same query URL in order to import the next 20 bugs at that URL.
** The tool will only create tasks out of bugs that have not already been used to create other tasks with the same query. ("Fresh bugs")
** A query that returns more than 100 results is rejected (a form validation error is displayed).
* Invalidation criteria are for Bug 1033808. Each criterion is of the form "field == value" or "field != value". e.g. (component != untriaged)
** The user may add/remove the number of criteria they need. Client-side validation ensures that the number of criteria of the batch is between 1 and 10.
* When the "Import Task Batch" form is submitted, a confirmation page is displayed. It shows the list of tasks about to be created. The user may return back to edit the form or go ahead and create the tasks.
Attachment #8458090 - Attachment is obsolete: true
Attachment #8476385 - Flags: review?(mkelly)
Attachment #8476385 - Flags: review?(bob.silverberg)
Oh, and the "Import Task Batch" form is initialized with an end-date that is 1 month away to discourage O&D from displaying stale imported tasks.
Attached image Import form view
Rebecca, Liz -- let me know if there's anything you want to me change about the UI. (Screenshots attached.)
Attachment #8476403 - Flags: feedback?(rbillings)
Attachment #8476403 - Flags: feedback?(lhenry)
Comment on attachment 8476385 [details] [review]
Link to Github pull-request: https://github.com/mozilla/oneanddone/pull/170

bsilverberg is plenty for reviewing this and I'm a bit strapped for time. :D
Attachment #8476385 - Flags: review?(mkelly)
Looks good to me Maja! You should get an awesome award for using "criterion" there. Grammar <3
haha, nope, Bob gets to take all the credit for that one, but yes: grammar <3.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
When will this be up on .paas.allizom.org?
I now know this hasn't been fully reviewed yet- removing 'Resolved' status
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment on attachment 8476403 [details]
Import form view

I think we may have issues with the `Field name` being a text input. This opens the process up to human error with misspellings and invalid field names. While it would introduce more work both for the development and for maintenance, it seems like we should maintain a list of valid field names from which a user can choose. These could be maintained via an admin screen, so we wouldn't have to write code to maintain them, and we could prepopulate them with a bunch of fields that we expect will be needed. If a person importing tasks needs a new field for an invalidation criterion then they could add one via the admin.

I think this is much safer than just leaving the field open. What do others think about this?
Comment on attachment 8476404 [details]
Confirmation view

I like the fact that the bug numbers are links to the actual bugs in Bugzilla, but I thought we were going to give the user the chance to choose which bugs to import, via check boxes, as opposed to just a 'Yes' to all or a 'No' to all.

Also, I think it would be an improvement if we could also display the description of the bug on this screen. As it stands not it's impossible to know which bugs are going to be imported (i.e., what those bugs actually are) without manually clicking through to each bug.

What do others think?
Attachment #8476404 - Flags: feedback?(rbillings)
Attachment #8476404 - Flags: feedback?(lhenry)
(In reply to Maja Frydrychowicz (:maja_zf) from comment #10)
> Created attachment 8476385 [details] [review]
> Link to Github pull-request: https://github.com/mozilla/oneanddone/pull/170
> 
> Up-to-date summary of requirements based on discussion over the past few
> weeks and reflected in my latest PR: 
> * The "Import Task Batch" view is accessible at '/tasks/import' and via a
> button at the top of '/tasks/create'.

Should there also be a direct link to /tasks/import from the new Admin menu that we're going to implement for bug 1030767? Right now one either has to type in the url or go through multiple steps in order to get to the Import screen.

> * The form in this view includes all the same fields as the "Add Task" form,
> as well as a few extra fields at the top: batch summary, query URL,
> invalidation criteria.

Should we set the "repeatable" flag on this screen to "False"? It seems to me that most tasks created out of bugs will be one-time tasks.

> * Based on the "query URL", which is a link to any search-results page from
> BMO, the tool creates one task for each bug in the BMO search results.
> ** Every task has the same properties except "Title", which is customized
> with the number of the associated bug.

Having tested this, I think it would be better if the task Title also included the description of the bug, perhaps in parentheses. This would make it easier for the task owner to keep track of which bugs have been actioned and which have not, and also easier for a One and Done user to know something extra about the tasks when listing them, other than the fact that they relate to a bug number. What do you think?

> ** The tool will create at most 20 tasks. The user may fill out the "Import
> Task Batch" form again with the same query URL in order to import the next
> 20 bugs at that URL.
> ** The tool will only create tasks out of bugs that have not already been
> used to create other tasks with the same query. ("Fresh bugs")
> ** A query that returns more than 100 results is rejected (a form validation
> error is displayed).
> * Invalidation criteria are for Bug 1033808. Each criterion is of the form
> "field == value" or "field != value". e.g. (component != untriaged)
> ** The user may add/remove the number of criteria they need. Client-side
> validation ensures that the number of criteria of the batch is between 1 and
> 10.

A note for Maja, when testing this I found the above not to be the case. I was able to create a batch with no invalidation criteria. There was one line on the form, but I left it blank, and no invalidation criteria were created.

> * When the "Import Task Batch" form is submitted, a confirmation page is
> displayed. It shows the list of tasks about to be created. The user may
> return back to edit the form or go ahead and create the tasks.
- yes, there should be a header option under Admin that goes to Import
- repeatable should be set to false, or removed completely as imported bugs are all one-time tasks
- I like the idea of including a description, but it may make for really long task titles. I’m having trouble visualizing it, and whether or not it’s a good idea.  If it’s possible to add the description into the description field that would be preferable.
Comment on attachment 8476404 [details]
Confirmation view

lgtm
Attachment #8476404 - Flags: feedback?(rbillings) → feedback+
Comment on attachment 8476403 [details]
Import form view

lgtm
Attachment #8476403 - Flags: feedback?(rbillings) → feedback+
Rebecca, I made some comments on both of the views that I was hoping to receive feedback on. I'm assuming that your lgtm's were for the views themselves. If you could review my comments and reply to them that would be great.
Flags: needinfo?(rbillings)
Were your questions not answered in comment 22? If I missed something please let me know.
Flags: needinfo?(rbillings)
Comment on attachment 8476385 [details] [review]
Link to Github pull-request: https://github.com/mozilla/oneanddone/pull/170

A great start, maja!  See PR for plentiful comments.
Attachment #8476385 - Flags: review?(bob.silverberg) → review-
Flags: needinfo?(rbillings)
https://bugzilla.mozilla.org/show_bug.cgi?id=1030972#c19
- yes to a populated list maintained in the admin vs open text field

https://bugzilla.mozilla.org/show_bug.cgi?id=1030972#c20
- I'm open to yes/no - the checkboxes would be even better, it's a matter of what we have time to implement
- agree that a description should display with the bug # so users know which ones they are w/o clicking through
Flags: needinfo?(rbillings)
I would also like the bug summary to display with the bug # for the confirmation screen, and otherwise agree with Rebecca.
I'm going to summarize the answers to my questions, which were reinforced by a meeting this afternoon. Here are some new requirements:

1. The confirmation screen should include the summary of the bug in addition to the number and should have a check box next to each bug so a user can indicate which to import. They should all be checked by default.

2. The invalidation criteria should be chosen from a list which is stored in a table. The table will be maintained via a django admin screen.

3. I will open a separate bug for the issue of adding this to a new admin menu, as we have identified 4 screens that should appear on this menu.

4. "Repeatable" should be set to False for all imported tasks when they are created. The form field which allows the user to choose repeatable should be removed when importing bugs.

5. When creating a task from a bug, the summary should also be included in the title of the task. For example, if "Please verify " was entered into the Title field in the Import form, a resulting title would look like "Please verify Bug 123456 (Summary of the bug appears here)".
Bob: on point 5, I think maybe the bug summary should go into the description of the task instead of the title.
Duplicate of this bug: 1030978
Attachment #8476403 - Flags: feedback?(lhenry)
Comment on attachment 8476385 [details] [review]
Link to Github pull-request: https://github.com/mozilla/oneanddone/pull/170

This is ready for another review. 

After consulting with Liz, I have included the bug summary in the confirmation screen visible to admins, but not as part of the task description shown to contributors. (The rationale is that bug summaries are often too verbose/confusing.)

I have addressed all the review comments except a couple that seemed uncertain (worded with "maybe"/"probably") and these two:
* Checkboxes next to tasks in confirmation screen
* Rewriting import-confirmation flow with a different approach. [1] 

I think the checkboxes should actually be a distinct feature/bug. I will be looking into the different approach to import-confirmation today, but I'm submitting for review now anyway because if the overall bug-import feature is time-sensitive, I believe it can/should be merged as is.

[1] https://github.com/mozilla/oneanddone/pull/170#discussion_r16914306
Attachment #8476385 - Flags: review- → review?(bob.silverberg)
Comment on attachment 8476385 [details] [review]
Link to Github pull-request: https://github.com/mozilla/oneanddone/pull/170

Please flag me for review again when the next iteration is available.
Attachment #8476385 - Flags: review?(bob.silverberg) → review-
The changes landed in https://github.com/mozilla/oneanddone/commit/d9374ccadfa88927b1154ec29ddda2d90d62cc64 and have been pushed to dev and stage.

Rebecca, if you can verify I can push to prod later today, which will just leave one bug outstanding for Q3. I did notice a couple of minor cosmetic issues with the feature, but I think it's fine to land as is for now and open extra bugs for those so we can at least get the feature in.

One thing that is currently missing is the menu item for importing, but I plan to add that later today. For now just go to "Create Task" and click the "Import Tasks" button at the top to get to the Import screen. I also added 2 sample Invalidation Criteria that can be used for now. You can add more at https://oneanddone.allizom.org/admin/tasks/taskinvalidationcriterion/
Flags: needinfo?(rbillings)
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → FIXED
Note that I have now updated the menu as well, so you can choose "Import Tasks" from the menu to be taken directly to the Import screen.
Verified able to import tasks from header, able to query import, duplicate queries, queries with no results, cancel import, draft & invalid task creation, queries of 100+, queries of under 100- import first 20, then after that the rest

> Import task button -> needs an “s”
> no link to bug in the task or the task title
> what the is batch summary & why is it required? why fill it out if it doesn't display in /admin and there is no record of the batch to look at?
> display of bug summary on import page- but it isn’t anywhere in the imported task, without the description the tasks are really empty and indistinguishable from each other
> there are no task fields that are automatically populated with bug info - from Comment 1
> No total for # of tasks being imported
> No record of the invalidation criteria after tasks are created
> No messaging about importing a portion of the query - and user is not returned to task import after importing the first chunk of tasks
Flags: needinfo?(rbillings)
Thanks for the review, Rebecca. I am aware of most of those items and agree they need to be addressed. Some are minor and easily fixed and some will require significant work. I really wanted to support Maja in trying to get the feature finished on her own, so I did not raise the bigger issues during my review as I knew it was going to be pretty tight just getting the work she had done thus far completed by the deadline. 

I suggest, therefore, that we accept the feature as is for now and open additional bugs for what is missing. It does seem to work well for importing bugs and creating tasks - the issues are more about dealing with the tasks and batches afterwards.

Perhaps we can have a chat tomorrow about what can be fixed by end of day tomorrow and what can be deferred to a separate bug.
(In reply to Rebecca Billings [:rbillings] from comment #39)
> Verified able to import tasks from header, able to query import, duplicate
> queries, queries with no results, cancel import, draft & invalid task
> creation, queries of 100+, queries of under 100- import first 20, then after
> that the rest
> 
Here's a more detailed reply to the individual issues:

> > Import task button -> needs an “s”
agreed and will be fixed tomorrow

> > no link to bug in the task or the task title
agreed - will linkify the Bug number in the task title, should be done tomorrow

> > what the is batch summary & why is it required? why fill it out if it doesn't display in /admin and there is no record of the batch to look at?
I agree that there should be some way of looking at information for a previously created batch. This is something that has not been addressed by the work thus far, so we should discuss what is needed and where it should go. A quick fix might be to add the batch to the admin, but I'm not sure how helpful that will be.

> > display of bug summary on import page- but it isn’t anywhere in the imported task, without the description the tasks are really empty and indistinguishable from each other
I had raised this before and I feel like I was vetoed by Maja and Liz (see https://bugzilla.mozilla.org/show_bug.cgi?id=1030972#c34) who both seemed to feel that we shouldn't add the summary to the task. I would be happy to revisit this as I agree completely that the tasks are hard to differentiate with only a bug id in them.

> > there are no task fields that are automatically populated with bug info - from Comment 1
I'm not sure what is meant by this. This requires further discussion. This might relate to the point about about bug summary.

> > No total for # of tasks being imported
The confirmation screen displays the text "You are about to import the x tasks listed below.", where x is the number of tasks. Is there somewhere else this should be where it is not?

> > No record of the invalidation criteria after tasks are created
Agreed - this should be viewable somehow. This relates to the item above about viewing info for a batch, because the invalidation criteria belongs to the batch.

> > No messaging about importing a portion of the query - and user is not returned to task import after importing the first chunk of tasks
This requires further discussion. It was not my understanding that the flow of importing multiple times when only a portion of a query can be imported was going to be "programatically" supported. The expectation was that the user would just do another import. If more is required that needs discussion.
Commits pushed to master at https://github.com/mozilla/oneanddone

https://github.com/mozilla/oneanddone/commit/fc60372ce35fdd3de036840444f415fa845e8a79
Bug 1030972 - Import tasks from Bugzilla

Extra clean up
Fix "Import tasks" label
Linkify Bug id in task name

https://github.com/mozilla/oneanddone/commit/6a2b61a8d0bb415d4ee4880bcf97f6ef2be4af42
Merge pull request #188 from bobsilverberg/import_fixes

Bug 1030972 - Import tasks from Bugzilla
All separate issues will be filed as bugs, the import feature is verified on staging.
Verified that task button was updated to Import Tasks, and that imported bug tasks have a link to the bug.
Verified on production able to import bugs
Status: RESOLVED → VERIFIED
Attachment #8476404 - Flags: feedback?(lhenry) → feedback+
Product: Mozilla QA → Mozilla QA Graveyard
You need to log in before you can comment on or make changes to this bug.