Open Bug 119857 Opened 23 years ago Updated 7 months ago

There is no way to set the reading/composing directionality via the UI

Categories

(MailNews Core :: Internationalization, defect)

defect

Tracking

(Not tracked)

People

(Reporter: xslf, Assigned: smontagu)

References

()

Details

(Keywords: rtl)

Attachments

(4 obsolete files)

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:0.9.7+) Gecko/20020111
BuildID:    20020211108

When editing or reading Email that contains mixed Hebrew/roman text, there is no
way to set the directionality of the window, making composing plain text
messages with mixed Hebrew english text or reading such messages very difficult
if not impossible

Reproducible: Always
Steps to Reproduce:
1. start a new email messege
2. try to compose mixed text. esp. things that have () or lists (even plain text)


Actual Results:   notice that there is no way to set the window as RTL to
compose the text proprly

OE has an option to change the directionality of a message locally. something
similar would be fine
Summary: There is no why to set the reading/composing directionality via the UI → There is no way to set the reading/composing directionality via the UI
Can't set it for HTMl emai either. Should i open a separate bug for that (since
it affects sending and not just local viewing)?
Changing platform/OS to ALL/ALL.

I think that between this bug and bug 96057 and bug 98160 we have covered all
the bases, and there is no need for another bug for HTML mail. On the other
hand, HTML mail is a more serious case than normal HTML composition, because
there doesn't seem to be a way to workaround by editing the HTML source
directly, unless I am missing it.
OS: Mac System 9.x → All
Hardware: Macintosh → All
*** Bug 120509 has been marked as a duplicate of this bug. ***
Re: bug 98160 - it is a windows only bug (mac does not have that behavier at 
all).
Any progress going to be done on this bug here, which was originaly filed for 
the mac?
It is a real pain to compose plain text email properly on the mac, which kind 
of defeates the whole thing...
Status: UNCONFIRMED → NEW
Ever confirmed: true
ok, until this bug is fixed, there is a workaround published by Mooffie, which
can be fund at: http://www.typo.co.il/~mooffie/mozilla/hebmailpack.html (the
server is not stable so you might need to try a few times before you can get thruh)
what's the differece between this bug and bug 96057?
This bug is about composing plain text, and bug 96057 is about composing in HTML
format.
Shoshannah Forbes:
> "ok, until this bug is fixed, there is a workaround published by Mooffie"

Since this link is now dead, could you please post this workaround here?

Thanks,

Prog.
Well, I don't know about Mooffie's workaround, but Ilya Konstantinov's 
suggestion (in Bug 96057) seem to do the trick here as well:

  Format-> Page Colors and Background-> Advanced Edit-> Attribute=Dir;Value=RTL

Prog.
> Format-> Page Colors and Background-> Advanced Edit-> Attribute=Dir;Value=RTL

1) this does not work with plain text email
2) These menus are not avalible when viewing a message- only when composing
It seems like there's another bug that hinders the use of my workaround:
If the body of the message only includes plain text, Mozilla defaults to 
Content-Type: text/plain, instead of Content-Type: text/html - regardless of 
the settings in the Send Format prefs.

Is this a known issue? should I open a new bug? this problem appears in both 
Mozilla 1.2.1 for Windows and 1.3a for BeOS.

BTW, one can bypass this bug by using some HTML formatting *within* the body of 
the message (setting a single white space as boldface is sufficient).

Prog.
The bug that I described in comment #11 was recently fixed by Scott MacGregor.
More details in Bug 228720 ("Don't down-convert message to plain text if dir
attribute is set").

This makes Moofie's XPI (see comment #5) a much more effective solution, it's a
shame that only hardcore Mozilla enthusiasts know about the existence of this
addon. Is there any opposition to including it by default in Mozilla?

Prog.
Since mooffies's site seems unstable I downloaded mooffie's fix. So if anyone 
want it. send an email to adinaronson at hotmail.com
Attached file Mooffie's xpi fix (obsolete) —
unzip the html page and xpi to a folder. Open the webpage and click the link to
install the xpi patch.
Attachment #139888 - Attachment is patch: false
Attachment #139888 - Attachment mime type: text/plain → application/zip
Can anyone point out why Moofie's XPI, despite containing an 'overlay' for
composition as well as for viewing, only works for viewing messages, not for
composing them? I mean, when I right-click the content-frame in the composer
window and select 'RTL body', nothing happens. What's up with that?
(In reply to comment #15)
> Can anyone point out why Moofie's XPI, despite containing an 'overlay' for
> composition as well as for viewing, only works for viewing messages, not for
> composing them? 

It works very well for both.

> I mean, when I right-click the content-frame in the composer
> window and select 'RTL body', nothing happens. What's up with that?

This is probably just an isolated issue with your setup, but instead of spamming
bugzilla, let's continue this discussion in mozilla.org.il forum.

Prog.
Attached file Alternative hebmailpack (obsolete) —
To solve my problem, I replaced the 'dirAlignComposer' function code (see
inside the zip; note you have multiple unzipping to do before you actually get
to any sources...) with alternative code; it now works for me. I also added a
submenu to the view menu for the composer window, same as for the messenger
main window, with the directionality control.

TODO list (well, my opinion of what a TODO list should be): 
1. this needs a toolbar button, not just a menu setting
2. the visibility of these controls needs to be controlled by a "show
directionality controls" option in Edit|Preference|Mail&Newsgroups|Composition
(this would make the fix more attractive as a checkin into the trunk code)
3. the submenu should show a checkmark for the current directionality
4. how about installation of the XPI into a user's profile rather than the
mozilla program directory?
Blocks: 240501
*** Bug 245551 has been marked as a duplicate of this bug. ***
More items for the TODO list:

5. Need to improve the direction guessing policy, which is currently not at all
perfect.
6. Need to apply the direction guessing policy for composition of 'reply'
messages, that is, if a message is displayed RTL and I press 'reply' or 'reply
all' , the new message's initial direction should be set to RTL.
(In reply to comment #17)

> 2. the visibility of these controls needs to be controlled by a "show
> directionality controls" option in Edit|Preference|Mail&Newsgroups|Composition
> (this would make the fix more attractive as a checkin into the trunk code)

actually, IMO it should go to composer|toolbars prefs panel.

> 4. how about installation of the XPI into a user's profile rather than the
> mozilla program directory?

that's a good thing for every extension. however, things which are only relevant
for an extension shouldn't be discussed here, IMHO.
Over time I've noticed most failures of Moofie's auto-detection are in cases
where latin documents use accented letters within a word. The algorithm should
decide whether the encoding is Hebrew based on words which have 0xE0-and-higher
character values in all characters of the word, rather than based on single
characters with no regard of context (e.g. single 0xE0-and-higher character
within a mostly below-0xE0 word). This would reduce the false positives at
practically no cost of false negatives.

(0xE0 is where Hebrew letters start in windows codepage 1255)
Ok, the 0xE0 is meaningless, apparently moofie checks for Hebrew by the unicode
values, which means after MailNews chooses the encoding. So the false positive
LTR alignment can be fixed by in the extension, but not the wrong selection of
the encoding (I thought the extension also reset the encoding, I was wrong).
Attached file hebmailpack 1.0.2 (obsolete) —
Implemented the "look for words instead of characters" logic I suggested, plus
changed the installation script so as to be able to install to the profile
directory. Note that if the extension has already been installed to the app
directory, it will be installed there, so remove it. Version bumped to 1.0.2.
Attachment #141324 - Attachment is obsolete: true
Attached file hebmailpack 1.2.1 (obsolete) —
Well, it seems Moofie himself has made some modifications since 1.0.0 in his
own typo.co.il webpage (I thought he was inactive). Anyway, this latest
attachment incroporates my changes into his most recent version. This time no
zip, just the xpi. I promise not spam this bug any more until I sort things out
with him (or is it her?).

Eyal

PS - this now has he-IL support, plus context menu for viewed as well as
composed messages
Attachment #154019 - Attachment is obsolete: true
*** Bug 231277 has been marked as a duplicate of this bug. ***
*** Bug 234894 has been marked as a duplicate of this bug. ***
Attachment #154082 - Attachment is obsolete: true
The BiDi UI project now offers an extension which is a further development of
Moofie's fix, above, with plenty of new functionality and other goodies. See:

http://bidiui.mozdev.org/mail/
Product: MailNews → Core
The functionality provided by BiDi Browser UI extension was recently merged into
FF 1.0 and SM 1.8.x. Is there anything that prevents the Mail extension from
sharing the same happy ending?

Prog.
QA Contact: giladehven → prognathous
*** Bug 342112 has been marked as a duplicate of this bug. ***
Component: MailNews: BiDi Hebrew & Arabic → Layout: Text
QA Contact: prognathous → layout.fonts-and-text
Assignee: mozilla → smontagu
Component: Layout: Text → Internationalization
Keywords: rtl
Product: Core → MailNews Core
QA Contact: layout.fonts-and-text → mailnews.i18n
Is there any work done on this in the context of Thunderbird 3?
Not any that I'm aware of.
Comment on attachment 139888 [details]
Mooffie's xpi fix

Obsoleting here to avoid confusion.

And I'll use this opportunity to mention that I'm always willing to collaborate to integrate some of this functionality into the Mozilla codebase...
Attachment #139888 - Attachment is obsolete: true
(In reply to comment #33)
> I'll use this opportunity to mention that I'm always willing to collaborate
> to integrate some of this functionality into the Mozilla codebase...

Which parts do you suggest to omit? I'm not sure if we need the directionality auto-detection now-days, but a RTL/LTR buttons in Composer and a 'Switch page detection' command should be enough.
Well, it's not about omissions, but rather about different implementations. I'll remind everyone of our discussion here:

https://www.mozdev.org/bugs/show_bug.cgi?id=20370

As for directionality auto-detection - it's a must. So much so that I've made it a different, default mode to all-LTR and all-RTL, and users rarely bother to use any of the latter two.
Paragraph auto detection is complicated algorithm, and no other software do this. Currently more users are aware of directionality in mail messages than few years ago, and even web mail sites such as gmail do include those control buttons. 

 While it could be a nice feature, please don't forget that most Thunderbird users won't ever need it. I'd suggest adding the basic functionality to the core, and keeping all the nice features in an optional add-on.
Paragraph auto-detection will come in form of bug 548206.
(In reply to comment #37)
> Paragraph auto-detection will come in form of bug 548206.
What's the status of that code? Is it going to land in the near future? If so, maybe we should make it blocking this bug.
(In reply to comment #38)
> (In reply to comment #37)
> > Paragraph auto-detection will come in form of bug 548206.
> What's the status of that code? Is it going to land in the near future? If so,
> maybe we should make it blocking this bug.

We're still working on the specification in the W3C HTML bidi group.  It should then be submitted to the WHATWG group for submission.  I don't think that anyone has started to work on the implementation yet.
I am not sure that I understand all the comments. This bug was opened 10 years ago. Working code exists in the form of the BiDiUi extension. This extension breaks when updating Thunderbird between version numbers, requiring Arabic, Hebrew, and Persian Thunderbird users to delay upgrading to the latest stable Thunderbird. What is the argument against including the BidiUi code in core Thunderbird? Is it just "nobody's done it yet"? Can I get SVN access to do it?

(In reply to :ehsan akhgari from comment #39)

We're still working on the specification in the W3C HTML bidi group. It
should then be submitted to the WHATWG group for submission. I don't think
that anyone has started to work on the implementation yet.

Has the specification to the HTML bidi group been submitted yet?

@DotanCohen:

Answering your second-to-last comment - from 2011 (!), here are some aspects of the reason:

  • The BiDi Mail UI code is an ex-post-facto hack. What you want is the layout engine to implement something - which is not what my extension does. That means that integrating the BiDi Mail UI functionality will not involve integrating the code.
  • ... actually, it's more complicated than that, because there's the question of what to apply direction to. That's another heuristic; but it also involves how messages are represented and rendered into (X)HTML (e.g. do multiple P elements get created for plain-text messages with \x10 \x10 sequences?) that means messing with MIME library code. The core of that code is from the early 1990s, and nobody will touch it. It will, at some point, be replaced by a Javascript-based MIME library (maybe this is even further along than I realize).
  • There's also the matter of caching the results of applying the direction-detection heuristic. That could also possibly be handled by the layout engine, but maybe it doesn't/wouldn't do that.
  • Not enough Arabic, Farsi, Hebrew and Urdu speakers/emailers in the userbase to push this.
  • Zero or just-about-zero Arabic, Farsi, Hebrew and Urdu speakers/emailers among the core developers to push this.
  • (A bit of a rant/slightly conspiratorial) A weird mentality that has took root overtime among some people recently, supporting over-simplification and slimming-down of the application as something the users supposedly want/need/must have. It's also a bit of a chicken-and-egg situation, since the users who need some feature are driven away, then apparently no users want that feature.

Thank you Eyal!

mentality that has took root overtime among some people recently, supporting
over-simplification and slimming-down of the application as something the
users supposedly want/need/must have

Actually, I'm all for slimming down the application to just the core components needed to fetch, read, compose, and send email. And since setting the reading and composing directional of the text is core to reading and composing email, of course it should be implemented. Implementing this feature does support the goal of slimming down Thunderbird to the core functionality only.

I meant over-simplification and slimming-down of the UI.

Also - what you or I believe is in the core is not what others believe is in the core. However - it is not inconceivable that some sort of a petition to the TB council could have a stronger effect than my occasional remark on the TB discussion mailing list. A petition/letter with speakers of several RTL languages, and it would be even better if it could include a few semi-well-known people.

Just a thought.

To what extent is this bug a duplicate of 464436? In other words, what parts of this bug are not covered in bug 464436?
Also, should bug 96057 not be marked as dependency of one of these?
And are other core bugs from https://mzl.la/31atXTD needed?

See Also: → 464436

Bug 464436 is more specific in that it regards a specific path to addressing directionality control, and at the same time more general in that it involves more missing functionality. So, not a dupe.

About bug 96057 - it's about paragraph direction, while this is about entire-message direction. Also, that one is about composition and this one is - at least in title - about both composition and reading. I suppose that's why they haven't been marked dupes of each other.

From the above you can also gather why 96057 and 464436 are not dupes of each other.

As for the longer list - I'll have a look tomorrow or later this week.

Is there any interest in getting some work done on this?

(In reply to Eyal Rozenberg from comment #46)

Bug 464436 is more specific in that it regards a specific path to addressing directionality control, and at the same time more general in that it involves more missing functionality.
About bug 96057 - it's about paragraph direction, while this is about entire-message direction. Also, that one is about composition and this one is - at least in title - about both composition and reading. I suppose that's why they haven't been marked dupes of each other.

Thank you for clarifying - very helpful. So should this bug focus on the reading functionality, and leave the others to focus on composition?

Is there any interest in getting some work done on this?

I don't know. I am asking around to see determine the interest and what is possible. If anything happens it would obviously depend heavily on what the user and add-on community is willing to contribute - you in particular since you clearly understand the issues very well.

So should this bug focus on the reading functionality, and leave
the others to focus on composition?

The way this bug is worded, it addresses both. But I agree that they are two separate issues. Perhaps one of the dupes that addresses either should be reopened, and this bug focus on the other.

For me personally, received emails are decypherable with the (currently) incorrect directionality, but it is very important to me to send email with the proper directionality. So I would vote to prioritize composition. However, some people would have valid reasons to emphasize a priority on reading.

@Dotan:

  1. I am going to make a TB-78 compatible version of BiDi Mail UI soon. So are you referring to the immediate state of affairs; to the fact that the functionality is not in the core; or to the functionality that cannot be introduced through an extension?

  2. I don't think the "bug configuration" matters all that much. If someone wants to have a clean-slate description of the state of affairs, I'd think some sort of Wiki page somewhere would be a good place for that.

@Wayne:

So should this bug focus on the reading functionality, and leave the others to focus on composition?

See point (2.) of my reply to Dotan. I don't think playing with the bug titles is worth the trouble.

I am asking around to see determine the interest and what is possible.

I was hoping you would say something like "this came up in a council discussion, people said that we need to be more accommodating of other languages and cultures, we decided we should put some developer time on it" ...

If anything happens it would obviously depend heavily on what the user and add-on community is willing to contribute you in particular since you clearly understand the issues very well.

See point (1.) of my reply to Dotan and the question there.

  1. I am going to make a TB-78 compatible version of BiDi Mail UI soon. So are you referring to the
    immediate state of affairs; to the fact that the functionality is not in the core; or to the functionality
    that cannot be introduced through an extension?

All of the above, really. From the end user perspective the result is the same.

I was hoping you would say something like "this came up in a council discussion, people said
that we need to be more accommodating of other languages and cultures, we decided we should
put some developer time on it" ...

Completely disregarding Eyal and my own language Hebrew, Arabic is also RTL and it is the fifth most widely spoken language in the world. Persians, Kurds, and Urdu are also significantly large populations with RTL languages.

Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.