1035972 - add preference to include all X-Headers in email bodies to allow gmail to filter on them

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: Extensions, defect)

Description

[Gene Wood [:gene]]

To simplify filtering Bugzilla emails in Gmail (which has limited email header filtering capabilities), what do you think about adding the header block to the body of HTML email in a non-displaying block?

I've put together a patch here : https://gist.github.com/gene1wood/62f64b0f363102cb4408

Does this look like something useful?

In this patch, I'm using "font-size:0px" because Gmail removes "dispaly:none" css from html email.

I've sent some emails with this format of html email to gmail and am able to filter on X-Headers.

Comment 1

[Gene Wood [:gene]]

:glob says : "This patch isn't suitable for deployment; it needs to be controlled via a preference (opt-in), needs to be present in both plain-text and html payloads, and should avoid duplication of x-header code (editing these fields should happen in a single place, not across 3 files). "

Comment 2

[Gene Wood [:gene]]

> it needs to be controlled via a preference (opt-in)

Ah, good idea. No need to put the content in everyone's HTML email.

> needs to be present in both plain-text and html payloads

Tell me more about this. Currently there are values which are only present in the body of the HTML emails and not plaintext (e.g. Component, Product, Reason)

> should avoid duplication of x-header code

Agreed

Byron, what do you think of the general concept of putting this type of content into the body of the email for purposes of enabling users of email providers without header filtering the ability to filter emails?

Comment 3

[:glob ✱]

(In reply to Gene Wood [:gene] from comment #2)
> > needs to be present in both plain-text and html payloads
> 
> Tell me more about this. Currently there are values which are only present
> in the body of the HTML emails and not plaintext (e.g. Component, Product,
> Reason)

that isn't correct - that content is present in our plaintext part.

> Byron, what do you think of the general concept of putting this type of
> content into the body of the email for purposes of enabling users of email
> providers without header filtering the ability to filter emails?

even with the x-headers in the message body, we'd still have limited filtering capabilities in comparison with what is currently possible in zimbra.

examples of things that won't be possible:

multiple filters: gmail has a single "has the words" field.  i don't see how you can filter on things like "X-Bugzilla-Flags contains needinfo?" and "X-Bugzilla-Flag-Requestee contains [e-mail]"

negative filtering: you can't express conditions like "X-Bugzilla-Watch-Reason does-not-match-example None"

per-field filtering:  you can't filter on a specific field.  eg. you wouldn't be able express "X-Bugzilla-Flags contains needinfo?" because that header can contain multiple values




it may pan out that dumping the x-headers into the body is "good enough" for most users, but it would be a shame to settle for a partial solution to this issue without first exploring alternatives (such as bug  1036263).

Comment 4

[Gene Wood [:gene]]

One thought that Byron and I just chatted about to resolve the challenge of filtering on an X header that presents a list of values (for example :

X-Bugzilla-Reason: CC Reporter

)

is to take list values and represent them each on their own line for example :

X-Bugzilla-Reason: CC 
X-Bugzilla-Reason: Reporter

This would allow filters like 

from:[e-mail] "X-Bugzilla-Reason: CC" -"X-Bugzilla-Reason: Reporter"

which would include tickets which the user is CC'd on and exclude where the user id the Reporter

Comment 5

[:glob ✱]

Attachment: 1035972_1.patch

- insert placeholder text into the 3 email templates (bugmail.txt, bugmail.html, flagmail)
- adds a user setting, which if set to on:
  - reads the x-bugzilla- headers from the current email
  - splits multi-value items
  - replaces the placeholder
- if set to off, the placeholder text is replaced with an empty string
- in the html part, it's hidden from clients using:
  <pre style="font-size: 0pt; color: #fff">..</pre>
  - tested with gmail web client and thunderbird

Comment 6

[:glob ✱]

Attachment: 1035972_2.patch

fixes an issues where the parts were being updated, but the emails were not reflecting the updates.

Comment 7

[David Lawrence [:dkl]]

Comment on attachment 8495354 [details] [diff] [review]
1035972_2.patch

Review of attachment 8495354 [details] [diff] [review]:
-----------------------------------------------------------------

The code runs as the current user and not the user being sent to the email. So if the current user (UI) has their pref to not include
headers, all emails do not get headers in the body and vice versa.

Unfortunately the mailer_before_send hook does not include the user the email is being sent as a param since it is ran by jobqueue.
You will need to pull out the user from the To: similar to how SecureMail does it currently to get the user's preference.

http://git.mozilla.org/?p=webtools/bmo/bugzilla.git;a=blob;f=extensions/SecureMail/Extension.pm;h=6871129556e86fe607294fb587d916301209d1ee;hb=HEAD#l272 [github]

Should be a quick r+ once that is solved as everything else checks out.

dkl

::: extensions/BMO/template/en/default/hook/global/setting-descs-settings.none.tmpl
@@ +10,5 @@
>      setting_descs.product_chooser        = "Product chooser to use when entering bugs",
>      setting_descs.pretty_product_chooser = "Pretty chooser with common products and icons",
>      setting_descs.full_product_chooser   = "Full chooser with all products",
> +
> +    setting_descs.headers_in_body        = "Include X-Bugzilla- headers in BugMail body",

nit: remove extra line

Comment 8

[:glob ✱]

Attachment: 1035972_3.patch

good catch!

Comment 9

[David Lawrence [:dkl]]

Comment on attachment 8495721 [details] [diff] [review]
1035972_3.patch

Review of attachment 8495721 [details] [diff] [review]:
-----------------------------------------------------------------

r=dkl

Comment 10

[David Lawrence [:dkl]]

To ssh://[e-mail]/webtools/bmo/bugzilla.git
   6612062..9552ab9  master -> master