373319 - Setting bidi.controlstextmode to 2 has strange side-effects.

Status: NEW

(Core :: Layout: Text and Fonts, defect)

Description

[Hendrik Maryns]

If you set bidi.controlstextmode to 2 and edit a wiki, all non-Latin1 symbols get converted to html entities.  This is a highly unwanted side-effect.

Steps to reproduce:
1 go to about:config
2 set bidi.controlstextmode to 2 (according to http://kb.mozillazine.org/About:config_entries#Bidi..2A) this should set it to ‘visual’
3 go to e.g. http://en.wiktionary.org/wiki/%D8%A8%D8%A7%D9%84%D8%BA, a wiki page with Arabic or any other non-Western script (actually, unusual symbols such as ‘ suffice)
4 edit the page
5 ask for a preview

Actual result: 
All non-Latin1 characters have been converted to HTML entities.

Expected result: 
This should not happen.

Both other settings (i.e. 1, ‘logical’, the default and 3 ‘container’ seem to work).

Reproduced on both Linux and Windows.

As a side note: If FF can do the inverse action too (i.e. convert html entities into their counterparts) in some way, I’d be much interested how this happens.

Comment 1

[Simon Montagu :smontagu]

'2' for that preference, i.e. visual controls on logical pages, is no longer supported. http://kb.mozillazine.org/About:config_entries needs to be updated and the strange side effect fixed.

Comment 2

[Hendrik Maryns]

Can you explain that a bit further for the layman?  What does ‘no longer supported’ mean, or better, imply?  And why do you call it visual control on logical pages?  What are logical pages?

Comment 3

[Simon Montagu :smontagu]

See http://people.w3.org/rishida/scripts/bidi/ for a good explanation of "logical" and "visual".

"No longer supported" means that setting that option will have no effect. The behaviour should be exactly the same with 1 or 2 for the value of the preference.

Maybe the first question should be "what are you trying to do?" If the option worked, you would have to enter بالغ by typing ghain, lam, alef, beh. Is that really what you want?

Comment 4

[Simon Montagu :smontagu]

So, I was looking through the code for places that still handle IBMBIDI_CONTROLSTEXTMODE_VISUAL and found some strange things.

http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/content/html/content/src/nsFormSubmission.cpp&rev=1.53&mark=1277-1297#1276
resets the submission charset of the form so that if the pref is set to IBMBIDI_CONTROLSTEXTMODE_VISUAL and the charset is any of "windows-1256", "ISO-8859-6" or "UTF-8", the charset becomes "IBM864"; and if the pref is set to IBMBIDI_CONTROLSTEXTMODE_LOGICAL and the charset is "IBM864", the charset becomes "IBM864i".

This should mean that the first two conditions at http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/content/html/content/src/nsFormSubmission.cpp&rev=1.53&mark=1350-1352,1357-1359,1374-1377#1349 can never be true, and if IBMBIDI_CONTROLSTEXTMODE_VISUAL is not supported the third condition will never be true either.

Comment 5

[Simon Montagu :smontagu]

Ahmad, are you aware of any scenarios where the code I pointed to in comment 4 is needed?

Comment 6

[Hendrik Maryns]

(In reply to comment #3)
> See http://people.w3.org/rishida/scripts/bidi/ for a good explanation of
> "logical" and "visual".
> 
> "No longer supported" means that setting that option will have no effect. The
> behaviour should be exactly the same with 1 or 2 for the value of the
> preference.

Ah, ok so this indeed is a bug.
 
> Maybe the first question should be "what are you trying to do?" If the option
> worked, you would have to enter بالغ by typing ghain, lam, alef, beh. Is
> that really what you want?

I want to browse through pages that contain rtl text visually.  I.e., if I use the right arrow, go right.  If I press Shift+right arrow, select to the right.

The reason is that I myself never enter rtl text, but often edit pages which contain some.  In that case, I want to be able to add brackets to it where I want, in a visual way, and select and copy it in a visual way.

Comment 7

[Simon Montagu :smontagu]

I think the bug you really want fixed is bug 195909.

Comment 8

[Hendrik Maryns]

(In reply to comment #1)
> '2' for that preference, i.e. visual controls on logical pages, is no longer
> supported. http://kb.mozillazine.org/About:config_entries needs to be updated
> and the strange side effect fixed.

I updated http://kb.mozillazine.org/About:config_entries to mention this bug.

Comment 9

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Hope someone is still around, three years later :)

Hendrik: What do you mean by that it works when you set it to 3 'container'? What behavior is it that you are expecting that you get? My reading of the code is that 3 'container' behaves largely the same way as 1 'logical', and that the distinction doesn't really seem to make sense.

Comment 10

[Hendrik Maryns]

(In reply to comment #9)
> Hendrik: What do you mean by that it works when you set it to 3 'container'?
> What behavior is it that you are expecting that you get? My reading of the code
> is that 3 'container' behaves largely the same way as 1 'logical', and that the
> distinction doesn't really seem to make sense.

I’m not sure, it’s long ago.  However, this bug is still there. (FF3.6)

If I set it to 3, the non-ASCII symbols are not converted to HTML entities, that’s what ‘works’ probably means.

I’m not sure what the difference between 1 and 3 is supposed to be either.  What I do know is that 2 behaves badly!

And what I want is described above: if I hit arrow right, go right, no matter the script.  If I hit delete, delete the symbol to the right, no matter what the script etc.  For example, try to change {{Arab|بلغ}} into {{term|sc=Arab|بلغ||stuff}} or the other way around: sometimes you just don’t know where the | is going to end which you are typing, or whether, if you hit delete, you will delete the | or the ب to the left of it.