650575 - A bug's first email has a different subject to subsequent emails

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: Extensions, defect)

Description

[Paul Biggar]

The first bug I get for an email has the subject:

   [Bug NUMBER] New: TITLE

while later bugs have the subject

   [Bug NUMBER] TITLE

My email client (gmail) treats these as separate threads. I seem unable to control this from the bugzilla UI. It seems to me that the first bug should oit "New: ", and the fact that I have no message in that thread is probably notice enough that a new thread exists.

Comment 1

[Max Kanat-Alexander]

Bugzila explicitly sends threading headers along with the emails to resolve this problem.

Comment 2

[Paul Biggar]

Doesn't work on gmail (I know, right!), which I would expect is used by a large segment of bugzilla's user base.

A workaround might be to allow users to specify their own email templates. I can file a bug on that if it has any hope getting somewhere?

Comment 3

[Max Kanat-Alexander]

Hey Paul. If it doesn't work on gmail, why is this the first I've heard of it? (Particularly considering that LpSolit uses gmail.)

Allowing users to specify their own email templates would be a non-starter, yeah.

Comment 4

[Josh Matthews [:jdm]]

It's been an annoyance for me on gmail ever since I started.  I just never thought to file a bug about it.

Comment 5

[Frédéric Buclin]

(In reply to comment #3)
> Hey Paul. If it doesn't work on gmail, why is this the first I've heard of it?
> (Particularly considering that LpSolit uses gmail.)

I use Thunderbird to read my emails, including those coming from Gmail. The web UI for Gmail has several unacceptable limitations, including its inability to display custom headers.

Comment 6

[Paul Biggar]

jdm confirmed I didn't imagine this, so I hope it's ok for me to reopen (though it may end up wontfix :-/)

Comment 7

[Frédéric Buclin]

Sorry, but Gmail is really the culprit to not take Message-ID, In-Reply-To and References headers into account. You should report the problem to them.

Comment 8

[Paul Biggar]

This isn't about blame, it's about making the best interface for users.

Comment 9

[:glob ✱]

here's a similar bug for jira -- http://jira.atlassian.com/browse/JRA-12640

this is a known issue with gmail -- http://mail.google.com/support/bin/static.py?page=known_issues.cs  it's also a very old issue, and i doubt gmail will move on fixing their issue.

i don't think the "new" prefix on the subject add any real value; in email clients that support threading correctly (ie. every modern client except gmail), a new bug shows up as a new thread, so it's obvious that it's a new bug.

as 17% of bmo accounts are gmail addresses, i'm happy to take this as a bmo customisation if upstream doesn't want it.

Comment 10

[Max Kanat-Alexander]

The "New" prefix is very valuable for me and I would object to this even being implemented on bmo. It makes it much easier to scan my incoming bugmail, which I do not thread.

Comment 11

[Frédéric Buclin]

(In reply to comment #10)
> The "New" prefix is very valuable for me and I would object to this even being
> implemented on bmo.

Based on a discussion we had yesterday on IRC about collecting data:

The question would be to know if many people rely on "New:" to be displayed in email subjects, or if it only helps a user or two. Or from the opposite point of view: if many people are bothered by it or not. I personally like to see it displayed because it helps me to quickly detect new bugs, and I really hope we keep it in the email subject, but it is totally inappropriate to argue that because *you* want it, you would object to see it implemented in bmo. It sounds like you have some veto power on this, which you clearly don't.


Paul, I know this bothers you and it probably bothers several other users, but Bugzilla has always worked this way, and you are the first one to complain, not because this feature doesn't make sense, but because Gmail is broken. If we were removing "New:" for new bugs, we would see the exact same bug being filed asking to reimplement it. I honestly think that it's Google's job to fix this problem on their own side, or you could use e.g. Thunderbird, which handles these bugmails with no problem.

Comment 12

[Max Kanat-Alexander]

(In reply to comment #11)
> It sounds
> like you have some veto power on this, which you clearly don't.

  Hey Frederic. "I would object" is an expression of my opinion, not me stating a veto or forbidding anything. Sorry if that was confusing in a language sense.

  I fully agree with your statement to Paul and the decision on this bug.

Comment 13

[Nicholas Nethercote [inactive]]

Count me as a 3rd person who suffers from this, after pbiggar and jdm.  I even wrote a blog post about it before I learned about this bug:  http://blog.mozilla.com/nnethercote/2011/06/09/gmail-and-bugzilla/

Comment 14

[Robert O'Callahan (:roc) (email my personal email if necessary)]

"Me too". I've been suffering from this for years and just never thought about trying to get it fixed.

Could we have a per-user pref that lets users opt out of the "New:"?

Comment 15

[Dirkjan Ochtman (:djc)]

I also suffer from this. If removing "New:" altogether is not a viable option, perhaps the per-user pref roc suggested is.

(I agree that GMail's threading logic is broken at least to some degree, but I don't think there is that much we can do about it...)

Comment 16

[Stefan Plewako [:stef]]

(In reply to comment #9)
> as 17% of bmo accounts are gmail addresses, i'm happy to take this as a bmo
> customisation if upstream doesn't want it.

Some gmail addresses users use email programs and enjoy correct threading so, 17% is really inaccurate measure.

Comment 17

[Ed Morley [:emorley]]

(In reply to comment #16)
> (In reply to comment #9)
> > as 17% of bmo accounts are gmail addresses, i'm happy to take this as a bmo
> > customisation if upstream doesn't want it.
> 
> Some gmail addresses users use email programs and enjoy correct threading
> so, 17% is really inaccurate measure.

On the other hand, some users (like myself) use google apps on their own domain, so whilst don't have a @gmail.com address, still use the gmail / it's web interface, which could mean the 17% is in fact an underestimate.

Comment 18

[:glob ✱]

while i'm hesitant to add another user pref, this would be easy to do in an extension.

Comment 19

[:glob ✱]

Attachment: patch v1

Comment 20

[David Lawrence [:dkl]]

Comment on attachment 538267 [details] [diff] [review]
patch v1

Review of attachment 538267 [details] [diff] [review]:
-----------------------------------------------------------------

runtests.pl error:

t/001compile.t ....... 29/176 "my" variable $user masks earlier declaration in same scope at ./extensions/GmailFriendlySubjects/Extension.pm line 45.

::: extensions/GmailFriendlySubjects/Config.pm
@@ +19,5 @@
> +
> +package Bugzilla::Extension::GmailFriendlySubjects;
> +use strict;
> +
> +use constant NAME => 'GmailFriendlySubjects';

I would rename it to GmailThreading as that is all we are really trying to solve and GmailFriendlySubjects 
is too broad. Unless you think there will be other subject issues down the road that will need to be added
to the extension.

::: extensions/GmailFriendlySubjects/Extension.pm
@@ +37,5 @@
> +    # map to a login
> +    my $login = $to;
> +    my $email_suffix = Bugzilla->params->{emailsuffix};
> +    if ($email_suffix ne '') {
> +        $to =~ s/\Q$email_suffix\E$//;

This should be $login =~ s/\Q$email_suffix\E$//;

Comment 21

[:glob ✱]

Attachment: patch v2

thanks for the quick review; here's an updated patch.

Comment 22

[David Lawrence [:dkl]]

Comment on attachment 538276 [details] [diff] [review]
patch v2

Review of attachment 538276 [details] [diff] [review]:
-----------------------------------------------------------------

Works good. r=dkl

Comment 23

[:glob ✱]

Committing to: bzr+ssh://[e-mail]/bmo/4.0/
added extensions/GmailThreading
added extensions/GmailThreading/Config.pm
added extensions/GmailThreading/Extension.pm
added extensions/GmailThreading/template
added extensions/GmailThreading/template/en
added extensions/GmailThreading/template/en/default
added extensions/GmailThreading/template/en/default/hook
added extensions/GmailThreading/template/en/default/hook/global
added extensions/GmailThreading/template/en/default/hook/global/setting-descs-settings.none.tmpl
Committed revision 7760.

Comment 24

[:glob ✱]

once this change has been pushed production, look for the "Make email subjects compatible with Gmail's threading" setting in "User Preferences".

Comment 25

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Great, thanks!!

Comment 26

[Nicholas Nethercote [inactive]]

(In reply to comment #24)
> once this change has been pushed production

How can I tell when that has happened?

Thanks for doing this.  It's clearly Gmail's bug, but you've helped a bunch of people with your flexibility! :)

Comment 27

[:glob ✱]

this preference is now live on production.

Comment 28

[Max Kanat-Alexander]

After examining the blogosphere, the comments on this bug, Twitter, and talking to many people about weird Gmail problems for many years, I'd love to have a preference for this upstream as well. It would be even cooler if it defaulted to "on" for people whose account was @gmail.com or @googlemail.com. Could we file a separate upstream bug for it?

Comment 29

[Ryan VanderMeulen [:RyanVM]]

When I go into general preferences, I see "gmail_threading" rather than the nicely-labeled version and it doesn't seem to do anything when I set it to on (I just got bugmail with the New: in the subject line).

Comment 30

[Paul [pwd]]

Shouldn't the option be under email preferences rather than general preferences.

Comment 31

[Nicholas Nethercote [inactive]]

(In reply to comment #29)
> When I go into general preferences, I see "gmail_threading" rather than the
> nicely-labeled version and it doesn't seem to do anything when I set it to
> on (I just got bugmail with the New: in the subject line).

I see the same thing;  at first it said "Make email subjects compatible with Gmail’s threading" but now it says "gmail_threading".  And I also just got bugmail with "New:" in the subject line.  This is on bugzilla.mozilla.org.

In other words, the feature isn't working.

Comment 32

[Paul Biggar]

(In reply to comment #31)
> In other words, the feature isn't working.

Same for me. I got "New: " mail due to bugs 664034, 663934, 663910, 663789, 663748, 663708, 663690, 663666 and 663616 since Saturday.

Comment 33

[:glob ✱]

an issue was found which was breaking new user confirmation emails, and the extension was disabled.  the issue has been fixed, and we're waiting on IT to update production and re-enable the extension (bug 663863).

Comment 34

[Gervase Markham [:gerv]]

glob: Particularly if this goes upstream, and even if not, can we remove the Gmail-specific naming in this feature? From the comments, it looks like several people have this problem, and I bet it's not only gmail which has a bad threading algorithm. A better name for the preference would be:

Add "New:" to subject line of email sent when a new bug is filed

and default it to "on".

Gerv

Comment 35

[:glob ✱]

(In reply to comment #34)
> glob: Particularly if this goes upstream, and even if not, can we remove the
> Gmail-specific naming in this feature?

that makes sense.

i'll work on it upstream with the changes you've suggested (bug 663747), and backport it once it has approval to BMO, nuking this extension.

Comment 36

[Nicholas Nethercote [inactive]]

I can confirm it's working!  (I noticed it for bug 666807, which I just filed.)  Thanks.