665419 - Plain-text e-mail notifications: improve readability and accessibility

Status: REOPENED

(Bugzilla :: Email Notifications, enhancement, P4)

Description

[Michael[tm] Smith [:sideshowbarker]]

User-Agent:       Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_7) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/14.0.794.0 Safari/535.1
Build Identifier: 3.6.3

This is an enhancement request to change the formatting of bugzilla e-mail notifications so that they are more readable -- in particular, more readable in mail clients that don't use a fixed-width font, and more readable in screen readers or other non-visual clients.

Specifically, I mean the formatting of added/removed field-change information in Bugzilla e-mail notifications -- that is, the three-column "ascii table" (for lack of better description) part, with the words "What  Removed Added" at the top, and "|" characters to delimit contents of the columns.

That formatting is suboptimal for a number of reasons.

First, that formatting choice seems to assume the user is reading e-mail in a client that uses a fixed-width font. These days, many (or most) users do not, but instead often use mail clients (such as GMail or iPad mail client) which users can't even set to use a fixed-width font even if they want to.

Second, that part of the notifications is nearly unusable in a screen reader -- especially if the contents of an changed field are longer than the column width allotted for it, and thus those contents are wrapped within the column. So it's a significant problem for accessibility.

Reproducible: Always

Comment 1

[Michael[tm] Smith [:sideshowbarker]]

Here's an example of what I would suggest as improved formatting:]

https://gist.github.com/1035260
https://raw.github.com/gist/1035260/5ba6711823a527ba670ad06f1289599426cba6c4/gistfile1.txt

[[
From: [e-mail]                                                                                                                    
To: [e-mail]                                                                                                                                             
Subject: [Bug 1] checking e-mail                                                                                                                            
Date: Mon, 20 Jun 2011 16:00:37 +0900                                                                                                                       
                                                                                                                                                            
http://sideshowbarker.net/bugzilla3/show_bug.cgi?id=1                                                                                                       
                                                                                                                                                            
[e-mail] changed:                                                                                                                       
                                                                                                                                                            
Priority                                                                                                                                                    
  Old: Normal                                                                                                                                               
  New: High                                                                                                                                                 
                                                                                                                                                            
Status                                                                                                                                                      
  Old: ASSIGNED                                                                                                                                             
  New: RESOLVED                                                                                                                                             
                                                                                                                                                            
Resolution                                                                                                                                                  
  Old:                                                                                                                                                      
  New: WONTFIX                                                                                                                                              
                                                                                                                                                            
Severity                                                                                                                                                    
  Old: critical                                                                                                                                             
  New: blocker                                                                                                                                              
                                                                                                                                                            
--- Comment #5 from [e-mail] 2011-06-20 16:00:37 JST ---                                                                                
this is a comment
]]

Comment 2

[Michael[tm] Smith [:sideshowbarker]]

Attachment: patch for BugMail.pm

I'm also adding an example patch to BugMail.pm that implements that suggested formatting for the example message I posted previously.

Comment 3

[:glob ✱]

the next major bugzilla release (4.2) switches to html for notification emails, which should resolve the main issue of rendering on modern email clients.

see bug 65477; followed by bug 604237 and bug 607829 about further improvements.

Comment 4

[Michael[tm] Smith [:sideshowbarker]]

Hi Byron,

Thanks for your comment #3 and for the pointers to the other bugs. In reply to that comment -

> the next major bugzilla release (4.2) switches to html for notification
> emails, which should resolve the main issue of rendering on modern email
> clients.

...I wanted to ask, what effect do those changes have on users who still use non-HTML-capable mail clients? (Or on users who have their mail clients set to default to display of the plain-text of multi-part messages that have both a plain-text part and an HTML part?) Will bugzilla still be sending out plain-text notifications along with the HTML ones? If so, have there been any changes made that affect the  formatting of the plain-text notifications? In particular, do the plain-text notifications still use the three-column ascii-table format?

Comment 5

[:glob ✱]

(In reply to comment #4)
> ...I wanted to ask, what effect do those changes have on users who still use
> non-HTML-capable mail clients?

currently we send both html and plain-text as alternatives.  bug 589128 is about making this configurable (this is important for the multitude of programs which parse bugzilla's current plain-text bugmail).

> what effect do those changes have on users who still use non-HTML-capable
> mail clients (or users who opt for the plain-text part)

as long as their email client supports MIME, they won't notice any changes.

> If so, have there been any changes made that affect the  formatting of the
> plain-text notifications?

i'm pretty certain plain-text notifications are unchanged.


i much prefer the current tabular format over the proposal in comment 1, however i won't notice the plain-text formatting after 4.2 anyhow :)

Comment 6

[:glob ✱]

due to all the programs which parse bugzilla's current plain-text format, unfortunately this isn't something that we can change.

Comment 7

[Michael[tm] Smith [:sideshowbarker]]

(In reply to comment #6)
> due to all the programs which parse bugzilla's current plain-text format,
> unfortunately this isn't something that we can change.

Couldn't it be implemented it as a system option for deployments that choose to use it instead anyway? (I mean those who have full knowledge that it will not work with programs that are hard-coded to parse the current format, but want to use it anyway.) It seems like this is something that deployers might like to have a choice on, and could be left up to deployers to decide what'd be best for their own users.
 
> *** This bug has been marked as a duplicate of bug 65477 ***

Isn't that bug about something else altogether? The summary is "Send HTML bugmail", and that's what the comments there seem to relate to. This bug seems to me at least to be not be a duplicate of that one at all. It's instead very specifically about the plain-text formatted notifications. I'm not going to object if the maintainers don't agree to providing a option for alternative formatting of the plain-text messages, but I would really prefer this bug not be marked as a simple duplicate of an existing bug, unless there is actually an existing bug that's already asking specifically for what I've suggested here (that is, for the plain-text messages to now use table layout).

Comment 8

[:glob ✱]

(In reply to comment #7)

> Couldn't it be implemented it as a system option for deployments that choose
> to use it instead anyway? 

making this a per-installation option probably isn't worth the gain; if installations of bugzilla want to customise the email, they can currently do so by deploying custom email templates (see 'Template Customization' in the documentation).

per-user templates is bug 268577.

> > *** This bug has been marked as a duplicate of bug 65477 ***
> 
> Isn't that bug about something else altogether? The summary is "Send HTML
> bugmail", and that's what the comments there seem to relate to.

you raised two issues:  the email doesn't work unless you're using a monospaced font (bug 64494), and secondly there's accessibility issues (bug 342033).

both of these issues have been addressed by the html bug.

> I would really prefer this bug not be marked as a simple duplicate of an existing
> bug, unless there is actually an existing bug that's already asking specifically
> for what I've suggested here (that is, for the plain-text messages to [not] use
> table layout).

i understand your point, however we generally duplicate bugs based on the issue raised, not the proposed solution.

Comment 9

[Michael[tm] Smith [:sideshowbarker]]

(In reply to comment #8)
> (In reply to comment #7)
> 
> > Couldn't it be implemented it as a system option for deployments that choose
> > to use it instead anyway? 
> 
> making this a per-installation option probably isn't worth the gain; if
> installations of bugzilla want to customise the email, they can currently do
> so by deploying custom email templates (see 'Template Customization' in the
> documentation).

I've looked at the relevant templates already. Unless I'm missing something, the template mechanism doesn't provide any hooks for adjusting the format of that part of the e-mail message. Instead what I find in newchangedmail.txt.tmpl is this:

[%- IF diffs %]

[%+ diffs %]
[% END -%] 

And the rendering of "diffs" is performed by code in BugMail.pm.

If that's the case, the template mechanism does not really help an installation customize the output of that part of the message. They'd need to also change the system BugMail.pm source.

> you raised two issues:  the email doesn't work unless you're using a
> monospaced font (bug 64494), and secondly there's accessibility issues (bug
> 342033).

No, I mentioned those as just two examples of problems that the table-layout formatting of the plain-text e-mail messages cause. 

> both of these issues have been addressed by the html bug.

With respect, no, this issue I have raised is not addressed by the HTML bug. Specifically, the HTML bug is not an acceptable fix for users who prefer to get their mail in plain text.

By the way, this is not a hypothetical problem. I personally know a blind user who relies on mutt to read her bugmail. And I don't think she is going to be particularly thrilled with being told that the solution to the accessibility problems with the e-mail messages is that she needs to use the HTML output instead, and so have to switch to a different mail client.

> i understand your point, however we generally duplicate bugs based on the
> issue raised, not the proposed solution.

So to be clear: The issue I am raising here is very specifically that I am requesting an option for the plain-text output to not use table layout. If you want me raise a new bug for that, with a new description making that more clear, I would be glad to do that. But being told that having HTML-formatted output is a solution to the request I'm making is not cool.

If you decide that you're not going to make any changes to the table-layout stuff in the plain-text formatting (because of the problem you cited with legacy tools expecting it to be in that format in order to parse it), then fine. That's up to you all to decide, of course.

But if so, I'd still like to very politely ask that you not dispose of this bug by marking it as a duplicate of a bug that I've already tried to make clear I don't all all consider to be related to what I'm requesting. Instead, I think it would be a lot more appropriate to mark it as WONTFIX with the comment you already made being the resolution on it (that is, "due to all the programs which parse bugzilla's current plain-text format, unfortunately this isn't something that we can change").

Comment 10

[:glob ✱]

(In reply to comment #9)
> I've looked at the relevant templates already. Unless I'm missing something,
> the template mechanism doesn't provide any hooks for adjusting the format of
> that part of the e-mail message. Instead what I find in
> newchangedmail.txt.tmpl is this:
> 
> [%- IF diffs %]
> 
> [%+ diffs %]
> [% END -%] 
> 
> And the rendering of "diffs" is performed by code in BugMail.pm.

as far as i can tell this has already been fixed in bugzilla 4.2, by bug 65477.

> By the way, this is not a hypothetical problem. I personally know a blind
> user who relies on mutt to read her bugmail. And I don't think she is going
> to be particularly thrilled with being told that the solution to the
> accessibility problems with the e-mail messages is that she needs to use the
> HTML output instead, and so have to switch to a different mail client.

i used mutt for a very long time, reading html emails with it just involves correct mailcap configuration.

> But if so, I'd still like to very politely ask that you not dispose of this
> bug by marking it as a duplicate of a bug that I've already tried to make
> clear I don't all all consider to be related to what I'm requesting.

please don't feel that i'm "disposing" of your bug; i saw the improvements made by switching to html as directly resolving your initially stated problems.  sorry that i didn't pick up on the "must be plain-text" requirement.

Comment 11

[:glob ✱]

unfortunately we can't make changes to the plain-text portion of emails by default due to the number of programs which currently rely on this format.


hopefully as the bugzilla api is extended, more of these programs will switch away from direct email parsing to rpc calls, and we'll have an opportunity to tidy up the plain-text formatting.

Comment 12

[Max Kanat-Alexander]

Hey glob, sorry to reopen this without talking to you about it first, but I believe that you're asleep right now, and I wanted to get to this today.

Michael: Numerous users have reported the same issues to us about Bugzilla's plain-text email, particularly with regard to accessibility. As far as machine parsing, your suggested format would be WAY easier to parse than the current system (I know, I wrote two major systems that currently parse bugmail and do something with them). It would also be much more readable for long text fields than the current system is (and probably more readable for flags, as well).

We are all somewhat partial to the ASCII table, since it's been around for so long and its usability aspects are well-understood, both in its successes and failures. And as Byron has mentioned, the new HTML format will resolve many of the negative usability aspects of the current plain-text format.

However, we could still look at improving the plain-text format in the future, particularly once bug 607829 has been implemented.

Comment 13

[Michael[tm] Smith [:sideshowbarker]]

(In reply to comment #10)
> (In reply to comment #9)
> > I've looked at the relevant templates already. Unless I'm missing something,
> > the template mechanism doesn't provide any hooks for adjusting the format of
> > that part of the e-mail message.
>
> as far as i can tell this has already been fixed in bugzilla 4.2, by bug
> 65477.

Ah, OK. Sorry, I should have taken time to actually look at the patch for that instead of just concluding that it wasn't relevant. Thanks for (re)pointing me to it.

> i used mutt for a very long time, reading html emails with it just involves
> correct mailcap configuration.

True but I think a non-sighted mutt user is probably not going to have her mutt config set to prefer the HTML part of a multipart message. Instead, I think she's going to keep it set to display the plain-text part. Because if she wanted to read HTML-formatted mail she'd probably be using a different client to begin with.

Anyway, the bugzilla HTML e-mail format, as currently implemented, is still suboptimal in terms of accessibility -- due to the fact that just like the plain-text format, it uses tables to organize the change information. A format that instead put the information into a list of some kind would be far friendlier to screen-reader/accessibility-technology users.

> please don't feel that i'm "disposing" of your bug; i saw the improvements
> made by switching to html as directly resolving your initially stated
> problems.  sorry that i didn't pick up on the "must be plain-text"
> requirement.

Yeah, I should have tried to make the more clear in the initial description. Sorry for having gotten testy about it.

Comment 14

[Michael[tm] Smith [:sideshowbarker]]

Hi Max,

Thanks for info in your comment #12:
> Michael: Numerous users have reported the same issues to us about Bugzilla's
> plain-text email, particularly with regard to accessibility. As far as
> machine parsing, your suggested format would be WAY easier to parse than the
> current system (I know, I wrote two major systems that currently parse
> bugmail and do something with them).

Yeah, I would sure think it would be. And to be clear, I'm not wedded to the exact format I posted earlier. To get back to use cases and requirements (which in hindsight is what I should have tried to state in my initial description): A key use case here is the case of non-sighted users trying to read bugmail in screen-reader -- especially bugmail that has long text fields that end up being wrapped inside one of the columns.

So a requirement from that is that they need a format that does not end sounding like gibberish in a screen reader.

I agree that having HTML-formatted bugmail can be part of the solution to that problem (though as I noted in comment #13, an HTML format that relies on table layout to organize the information is still much less screenreader-friendly than one that organizes it into a list).

> It would also be much more readable for
> long text fields than the current system is (and probably more readable for
> flags, as well).

Bingo. The case where a text field ends up getting wrapped is particularly problematic to try to read usably in a screen reader. And frankly, with the format of the current plain-text output, even as a sighted user, I personally often find it difficult to read those messages and figure out what actually changed.

> We are all somewhat partial to the ASCII table, since it's been around for
> so long and its usability aspects are well-understood, both in its successes
> and failures. And as Byron has mentioned, the new HTML format will resolve
> many of the negative usability aspects of the current plain-text format.

Right -- I think a lot of users are going to be very glad to have the HTML format. So that's definitely a big step in the right direction.

> However, we could still look at improving the plain-text format in the
> future, particularly once bug 607829 has been implemented.

I'd be very glad to help with improving the plain-text format if/when you all do have time to turn some further attention toward it.