26943 - Bulk (mass) changes in one email/bugmail notification

Status: NEW

(Bugzilla :: Email Notifications, enhancement, P1)

Description

[Matthew Tuck [:CodeMachine]]

I'm sure the main reason bulk changes are so annoying is the number of
notifications they spew out.  Why not just one?

eg

x made change:

Milestone -> M13
Keywords -> - css1
Status Whiteboard -> GO HOME IE

Previous differing values:

Bug #111

Milestone: M12

Bug #112

Milestone: M14
Status Whiteboard: GO HOME MOZILLA

Comment 1

[Matthew T (active 1999-2002)]

I'm assuming you mean notify changes of multiple bugs in one e-mail message. That 
would be bad for most users, because it would make it hard for them to keep track 
of/sort/filter notifications for particular bugs.

If that's not what you mean, and you just want changes to multiple fields in the 
same bug to be notified in the same message, well that's exactly what the new 
(`experimental') notification scheme does.

All in all, WORKSFORME. But I have a feeling you'll disagree, so I won't mark it 
as such.

Comment 2

[Matthew Tuck [:CodeMachine]]

Yes, all in one message.  I have no need to filter Bugzilla mail, and I probably
get as many notifications as anybody.  However, I certainly get annoyed by bulk
changes, as I usually mass delete them (one at a time) and it is easy to delete
one in the middle that is not part of the bulk change.

I'm pretty sure there would have been occasions where people didn't do things so
as to no generate spam.

This would be OK as a user pref, although it would result in people still being
afraid of sending out spam.

There's also less need to filter in the case of having no spam and the
implementation of bug #14137 (user-defined notification criteria), since people
would get a lot less email.

Take a look at bug #14137 again.  I added a proposal for keyword filtering.  If
this was implemented, we could make it so a change was sent out for each set of
keywords.  This would allow perfect filtering while still reducing change
messages.  If you didn't set up keywords, you'd always get one message, since
every message has the null set of keywords.

The disadvantage of this is that you can't use the same message for everybody on
the server side.  There's at least one other RFE that could cause different
messages, bug #26194.

Of course, you could probably still generate parts of the message which are
universal - it's how to combine them that derives from what the user chooses. 
If everyone who is monitoring the bug has the same keywords, you'd only need to
generate one.

Of course, I can probably guess Terry's response to the last bit.

Comment 3

[Terry Weissman]

[e-mail] is the new owner of Bugzilla and Bonsai.  (For details,
see my posting in netscape.public.mozilla.webtools,
news://news.mozilla.org/38F5D90D.F40E8C1A%40geocast.com .)

Comment 4

[Matthew Tuck [:CodeMachine]]

Case in point - I just got 150 messages for the Bugzilla component owner change
where 1 would have been nice.

Given that the new notification system already generates a message per user,
plus the lack of dissent, I'll speculate notification classes are a solution to
this.

Comment 5

[Jacob Steenhagen]

This could really become an issue when Mozilla 1.0 comes out and all the
VERIFIED bugs get CLOSED.  Having this feature implimented by then would be
really nice as there could still be a notification that this is happening, but
not require 1,000+ e-mails per user to do it.

Comment 6

[Brian 'netdragon' Bober]

This is probably the best enhancement request in the history of bugzilla. I 
would love to see this implemented!

I have a few suggestions:

A prefs setting:

Field1 - the time to collect all bug changed before sending the large email (0 
means use the old way)

Field2 - A list of bugs you want to keep seperate from other bugs and want it 
to only collect changes in that bug before sending out.

Comment 7

[Jason Kersey]

The closed situation is not true.  This change would happen behind the scenes 
almost for sure.  The only email for that would be to .general or something.

Comment 8

[Brian 'netdragon' Bober]

Also note that if you have any email program that can show links from plaintext 
urls, then the links will change color in the email program and thus that would 
keep track right in the email.


Changes Update -  1/24/00 11:34PM to 1/25/00 12:34PM - Duration: 60 min
                    
                      Numbers                     |         Summaries
--------------------------------------------------------------------------------
http://bugzilla.mozilla.org/show_bug.cgi?id=64179 | "K" should be "KB"
http://bugzilla.mozilla.org/show_bug.cgi?id=29346 | Prevent repeating popup win
http://bugzilla.mozilla.org/show_bug.cgi?id=24689 | XUL Reference Manual
http://bugzilla.mozilla.org/show_bug.cgi?id=16775 | Query confusing for newbies
--------------------------------------------------------------------------------

********************************************************************************

--------------------------------------------------------------------------------
http://bugzilla.mozilla.org/show_bug.cgi?id=64179 | "K" should be "KB"
--------------------------------------------------------------------------------

[e-mail] changed:


           What    |Old Value                   |New Value
----------------------------------------------------------------------------
  BugsThisDependsOn|                            |66480

------- Additional Comments From [e-mail]  2001-01-24 17:11 -------
Patch will be attached to bug 66480 to eliminate duplicating patch attachments.

********************************************************************************

--------------------------------------------------------------------------------
http://bugzilla.mozilla.org/show_bug.cgi?id=29346 | Prevent repeating popup win
--------------------------------------------------------------------------------

------- Additional Comments From [e-mail]  2001-01-24 18:22 -------
OK, I've just tested the following and it works just fine: to block popups from
a specific site or group of sites, add these lines to bin/defaults/pref/all.js
(turns out you have to add these to all.js, not prefs.js, I think):

pref("capability.policy.popupsites.sites", "www.annoyingsite1.com
www.popupsite2.com [and so on, space-separated list of URLs]");
pref("capability.policy.popupsites.windowinternal.open","noAccess");

The first line defines a group (the equivalent of a "security zone" in IE)
consisting of a list of sites, and the second line forbids these sites from
opening new windows. Try it, it works!

------- Additional Comments From [e-mail]  2001-01-24 18:27 -------
Actually, the URLs in the above example have to start with http:// (or ftp or
whatever). Don't leave off the protocol.

********************************************************************************

--------------------------------------------------------------------------------
http://bugzilla.mozilla.org/show_bug.cgi?id=24689 | XUL Reference Manual
--------------------------------------------------------------------------------

[e-mail] changed:

           What    |Old Value                   |New Value
----------------------------------------------------------------------------
                 CC|[e-mail],axel@pi|[e-mail],axel@p
                   |ke.org,biswapesh_chatterjee@|ike.org,biswapesh_chatterje
                   |tcscal.co.in,[e-mail],|[e-mail],boberb@rpi.e
                   |[e-mail],gary_|du,[e-mail],g
                   |[e-mail],mozilla@david|[e-mail],maolson@
                   |krause.com,rudman@netscape.c|earthlink.net,mozilla@david
                   |om                          |krause.com,rudman@netscape.
                   |                            |com

********************************************************************************

--------------------------------------------------------------------------------
http://bugzilla.mozilla.org/show_bug.cgi?id=16775 | Query confusing for newbies
--------------------------------------------------------------------------------

[e-mail] changed:

           What    |Old Value                   |New Value
----------------------------------------------------------------------------
  Status Whiteboard|                            |2.12

------- Additional Comments From [e-mail]  2001-01-24 22:34 -------
now have a viable patch here... nominating for 2.12 to get it on the radar.  
There's a few other things going on with query.cgi, so this may or may not 
actually go in, depending on the others.  This just gets it on the map so we 
can 
compare.

Comment 9

[Brian 'netdragon' Bober]

Dave: oopsies, I don't think you posted that in the right bug ;-)

Comment 10

[Jacob Steenhagen]

That wan't a post from Dave... it was the last past of your post... :)

Comment 11

[Dave Miller [:justdave]]

Jason: under the existing code, CLOSED mail goes to everyone CC'd or otherwise 
attached to a given bug.  If there's any mechanism in place to post to the 
newsgroup, it's a mozilla.org-specific thing, and not part of Bugzilla.

Comment 12

[Brian 'netdragon' Bober]

Sorry, I was really tired when I wrote that.

Comment 13

[Jesse Ruderman]

Re kerz: I don't like the current "behind-the-scenes" system, because at least 
for some changes, it appears as if the next person to change the bug made the 
behind-the-scenes change.  Fixing this bug would remove the need for those 
kinds of changes.

Comment 14

[Matthew Tuck [:CodeMachine]]

> Field1 - the time to collect all bug changed before sending the large
> email (0 means use the old way)

Why?  This isn't about having a behind the scenes process aggregating similar
messages, this is about never splitting them up in the first place.

> Field2 - A list of bugs you want to keep seperate from other bugs and
> want it to only collect changes in that bug before sending out.

As long as that's a user rather than system thing I wouldn't have a problem
other than to question the utility.  I've already suggested a more general
mechanism to separate out different types of bugs.

Comment 15

[Dave Miller [:justdave]]

Moving to real milestones...

Comment 16

[Hixie (not reading bugmail)]

Taking all Bugzilla 3.0 bugs -- congratulations to MattyT for the triage, it
really was spot on. Note: I may end up pushing some of these bugs back to 3.2,
we'll see. However, I believe all these bugs should just fall out of the 
redesign. Let's hope I'm right!

(Note: I'm also resetting the priority field to "---" so that I can retriage any
that I consider important or likely to be dropped.)

Comment 17

[Dave Miller [:justdave]]

The Bugzilla 3 component is going away.  We're going to depend on the Milestones 
for this.  At the time this component was created, we didn't have milestones for 
Bugzilla.

Comment 18

[timeless]

<q src=Jake> timelyx: My most recent suggestion RE that was to provide the
ability to name a mass change and put that name in a header (so it can be
filtered easily client side)</q>
<q src=timeless> Jake: not good enough. I'd actually like one extra checkbox [x]
for mass changes include all bugs even those which i would not normally see in a
single bugmail</q>
<p class=reason> If you can make a mass change in a form you should be able to
send out the mass change as a single message.</p>
<xmp>
These settings were applied:
Field | Value
------+--------
foo   | bar
baz   | snafu
These values were in bug #bleh
foo   | py
These values were in bug leb
baz   | y
...
</xmp>

<q src=Jake> Also, some ppl use mail client filtering to keep track of bug
changes and if you send a mass change as one big message, it will ruin their
filtering.
Unless there's a MAJOR change in the way processmail / process_bug.cgi /
post_bug.cgi / et al. interact. Mass changes are kinda a hack... there's an
array that contains all the bug numbers to change (if it's coming from bug_form,
there's only one value in that array). It then takes the next bug in that array,
makes any changes, send a mail. Loop until they're all done.</q>

<q src=timeless> 
<ul> So it would need to make a second type of message (message.t2). 
<li> Starting by dumping the change info. -that's easy</li>
<li> looping over bugs and 
 <ul>
 <li>building a list of people who want this new style for each bug,
     dump the number summary and old values into the message.t2</li> and then
 <li>send a typical message for people who don't want the mass notify</li>
 </ul></li>
<li> when it's done, it has built a list of all affected people interested in
digest and already sent individual mails for the non digesters</li>
<li>so it then sends 1 huge message multi bcc to the long list of digesters</li>
</ul>
done.
seem sane?
</q>
<q src=Jake> I think there's still gonna be some logistics problems (filtering,
reasons, etc)... but it may be doable</q>
<q src=timeless> basically that [x] send me mass digests bypasses filters. it
takes you out of the indvidual mail and filter pool</q>
<q src=Jake> Paste that info into bug 26943</q>

<q src=timeless> the reason will be mass change. none of the other
justifications</q>

Comment 19

[timeless]

hoping for 2.18

Comment 20

[Matthew Tuck [:CodeMachine]]

*** Bug 104169 has been marked as a duplicate of this bug. ***

Comment 21

[Matthew Tuck [:CodeMachine]]

It's important that the actual changes should appear before the specific bugs,
otherwise the full details of the actual change might get buried.

If we just appended all the bugs together then the full details might not be
obvious.  This is because for certain bugs, parts of the change might not occur
because the new value is the same as the old value.

Comment 22

[Jesse Ruderman]

I think it's more important to see the list of bugs that were changed than it is
to know which changes applied to which bugs.  Let's not include the full details
of each change for each bug.

Comment 23

[Matthew Tuck [:CodeMachine]]

I don't see any way we can justify not doing so ... otherwise there was
information there before that isn't there now.

Comment 24

[Jesse Ruderman]

I'd rather have 99% of the information in a readable format than 100% of the
information in a format that isn't much better than what we have now.

Comment 25

[Ben Bucksch (:BenB)]

Maybe do both?
In the beginning, list all changes done (i.e. the superset) and maybe all bugs
changed. Then list each bug and the changes applied to it.

Comment 26

[Dave Miller [:justdave]]

Changing default owner of Email Notifications component to JayPee, a.k.a.
J. Paul Reed ([e-mail]).  Jake will be offline for a few months.

Comment 27

[Jonadab the Unsightly One (Nathan Eady)]

I like Ben's comment #25.  Then you read as far into the message
as you like, depending on the level of detail you desire.  

Comment 28

[Joel Peshkin]

*** Bug 167533 has been marked as a duplicate of this bug. ***

Comment 29

[Dave Miller [:justdave]]

Enhancements which don't currently have patches on them which are targetted at
2.18 are being retargetted to 2.20 because we're about to freeze for 2.18. 
Consideration will be taken for moving items back to 2.18 on a case-by-case
basis (but is unlikely for enhancements)

Comment 30

[Dave Miller [:justdave]]

Bugzilla 2.20 feature set is now frozen as of 15 Sept 2004.  Anything flagged
enhancement that hasn't already landed is being pushed out.  If this bug is
otherwise ready to land, we'll handle it on a case-by-case basis, please set the
blocking2.20 flag to '?' if you think it qualifies.

Comment 31

[Onno Garms]

*** Bug 261771 has been marked as a duplicate of this bug. ***

Comment 32

[Joel Peshkin]

Actually, the way to do this is probably to stop sending mail at all during bug
processing and have a seperate daemon looking for mail that needs to be sent
every few minutes.  If we only send mail to a user when that user has a change
that is more that 5 minutes old and we then send all mail that needs to go to
that user, we can batch mass-changes and we can also batch multiple changes to
the same bug. (REOPEN, ASSIGN, ACCECPT, ATTACH, Review?)

Comment 33

[Dave Miller [:justdave]]

*** Bug 316402 has been marked as a duplicate of this bug. ***

Comment 34

[Frédéric Buclin]

*** Bug 347986 has been marked as a duplicate of this bug. ***

Comment 35

[My3bIkaHT]

I also encourage sending mass changes in one e-mail. My users are always complaining when they recieve more then 50 mails in several minutes. (And when they recieve about 200 letters, they become mad). Some users avoid doing mass changes because of it. They prefer having a wrong person being assigned to a set of bugs rather then sending this spam. Some users turn off email notification at all.

I could be done as a checkbox in user prefs: "Inform me of mass changes with one email". It could be turned off by default, that doesn't matter to me.
I think the content of the e-mail should be as described in comment 25.
I think that every user should recieve individual email, which contains only bugs which the user is subscribed to, but if it is much harder, multi-recipient email will also do.
I thinks it should never merge changes made by different users into one e-mail (it whould mess everything).
I also don't want different changes made by one user to be merged. I want just one thing: If I do one change (one pressing "submit" button), users should recieve one email.

As for the implementation. I can see 2 major approaches.
1)handle mass changes in different way then changes in one bug. It will need less changes, but is less flexible.
2)consider changing one bug as changing a list of bugs containing one bug. So mass and single changes would be handled in the same way.

As for possible problems. There may be a problem if I'm doing a change to a list of bugs, and it's interrupted (server shutdown for example). In old mechamism I possibly loose only the last email. In the new one I won't be notificated at all. So, bugzilla should email about successful changes, which took place before the interuption. It should do so automatically if it understands the error. Otherwise I should be able to send it by sanity check.

Comment 36

[Ryan Russell]

Would love to see this implemented. As the QA manager at work, I watch all bugs. One of our lead developers just did two mass changes preparatory to making a release branch, and Bugzilla sent me 1200 emails. I'm going to mass-delete, bug it would be ideal if I didn't have to worry about accidentally missing any other emails that made it into the batch during that time.

Comment 38

[[e-mail]]

(In reply to comment #37)
>*** Bug 376397 has been marked as a duplicate of this bug. ***
It's not quite a duplicate, because I suggested notifying for duplication and/or dependency changes in one email too.

I also suggested listing all the changed bugmails as (multipart) attachments.

Comment 40

[Max Kanat-Alexander]

I thought I'd mentioned my suggested implementation here before, but it was actually in another bug, so I'll mention it here:

The top of the email will state what changes were requested by the user, and below that will be a single listing for each bug of what actually changed (though the comment will only appear once, if there is a comment).

Comment 41

[timeless]

mkanat: that's roughly what comment 18 says.

Comment 42

[Mark Goodhand]

Reinforcing earlier comments, I would suggest that users have a tri-valued "global option" in the email preferences,

   Bulk edit notifications: [None, Individual, Digest]

(I can't see a case for a "Field/recipient specific" option; others might)

'None' achieves the same effect as bug 23924, but with the choice made by users rather than admins.  The default would be 'Individual', which would be the same as the current behaviour except that

1) An X-Bugzilla-Bulk-Edit header indicates that the notification was part of a bulk edit.  I don't think these need to be named, as suggested in comment 18.  A timestamp would suffice to distinguish one bulk edit from another, though I expect most people will simply filter based on the presence of the header.  Alternatively, if we're not concerned about distinguishing between batches, perhaps just introduce a new value for X-Bugzilla-Type: "changed (bulk)" ?

2) The subject line should also indicate a bulk edit (this is more obvious and convenient for most users than an X-Bugzilla header).  Now that we have proper threading based on In-Reply-To:/References:, there's no need to worry about having a slightly different subject line.  A "Bulk change: " prefix should cause no more trouble than the "New: " prefix.


This bug has been open for 11 years.  Some sort of fix would be greatly appreciated.  I face the same problem that others have documented.  Bulk edits are useful, but I don't like spamming everyone.  As an admin, I occasionally resort to making bulk edits out-of-hours, temporarily setting mail_delivery_method to none.  That is clearly suboptimal.

If we can't agree on the email format / preferences UI / implementation details for the Digest approach, how about proceeding with points (1) and (2) above? That would address the primary source of frustration (difficult-to-filter bulk edit spam) without requiring major changes.

Perhaps just start with (2): Simplest Thing That Could Possibly Work (TM).

Comment 43

[timeless]

fwiw, while i wrote out how i'd want a single unified thing to work, i that was in 2001.

I still think that's more or less good enough and would like to see it implemented as one of the options.

however, I switched to gmail in 2004. with gmail, i'm now one of the users who wants the individual messages.

@bmo, i'm probably one of the people who has made the most mass changes. unfortunately, that also means i've probably made more errors in mass changes than most people. and it turns out that it's much easier for me to catch and revert mistakes when i review them message at a time and in context of the correct bug.

the preference of none/individual/digest is the correct way to specify this.

As for naming bulk changes, yeah, i guess since people would only get one message it isn't needed, good point.

Fwiw, New: actually causes pain in gmail, since it breaks threading. But as bulk changes are unlikely to thread usefully (although in theory reference: or something could be abused for this), it's fine to avoid that. The prefix should just be 'Bulk:' no need for 2 words where one will suffice. @work we read bugmail in Modest on the n900, and thus every character in the summary is precious.

Lastly, mark, if this is bugging you, could you please try tossing a patch together? -- Bugzilla for the most part is driven by people writing patches (or paying others to write them). And thanks for expressing an interest.

Comment 44

[Mark Goodhand]

Thanks timeless.  'Bulk:' sounds good to me.  I don't really know Perl, but I'll have a quick grep once I've installed Bazaar.  Failing that, I'll see who I can enlist :-)

Comment 45

[Max Kanat-Alexander]

(In reply to comment #42)
> Reinforcing earlier comments, I would suggest that users have a tri-valued
> "global option" in the email preferences,
> 
>    Bulk edit notifications: [None, Individual, Digest]

  This would be a separate bug, not this bug. You are looking for bug 72132.

Comment 46

[Max Kanat-Alexander]

Oh actually, I see what you're talking about. That might not be a bad idea. I was going to do it in another way, though--see comment 40. That's still what I'd prefer.