225950 - Changing view's charset should recode (redisplay) the content

Status: RESOLVED INCOMPLETE

(Other Applications Graveyard :: ChatZilla, defect)

Description

[Max Alekseyev]

Related to bug 204411

To reproduce:
1. Open chatzilla
2. Connect to some channel using different charset from your default
3. Wait for some text to appear
4. Try to set up proper charset with ``/charset'' command
5. Notice that /charset doesn't affect the current channel tab content

Expected result: /charset should affect (``recode'') the current channel tab content

Comment 1

[James Ross]

I'm wondering if this is just a side-effect of the prefs manager currently being
used. If (all from the channel tab) you do "/charset <newcharset>" to change the
global charset, then do "/channel-charset -", does it then work?

Comment 2

[Robert Ginda]

I don't think this is practical.  Going through and recoding every message is
time consuming.  If we have to do this for every tab (like if the user changes
the default charset), then /charset could take a very long time to execute. 
Furthermore, all of the messages are inserted into the message window AFTER
conversion to unicode.  If we take kio8-r encoded bytes, convert them to unicode
using the wrong charset (say, utf-8), the process is not reversible.  Go from
encoding to unicode back to the original encoding, if we're using the wrong charset.

I think you're just going to have to live with this behavior.

Comment 3

[Max Alekseyev]

Ops, I seem to miss some points in charset-related features.
If /charset sets up "default codepage" - why it's affect already opened
channels? In my understanding "default codepage" setting should be applicable to
newly created channel tabs only.

In this bugreport I mean command that affects only current tab, i.e.
/channel-charset 
Fixing summary...

Comment 4

[Max Alekseyev]

My point of view on charset commands design is in bug 226408

Comment 5

[James Ross]

Making summary more general to the problem.

My uess is this is a WONTFIX bug, at least for now, due to is being a fair
amount of effort, would use a lot of extra memory (have to store original lines
as well as decoded ones), and isn't going to be useful that often (not to
mention *changing* charset half-way through a conversation still needs to be
possible).

Comment 6

[Robert Ginda]

yeah, this is too much to do even for a single tab.

Comment 7

[Max Alekseyev]

First, I don't understand why there is a need to keep original lines after
reconding. If you care about inproper recoding - leave it to user
responsibility. Say, I know what I'm doing choosing some charset.

Second, there is definitely a need to have a recoding feature. Say, I connected
to a channel and got some unreadable greetings/whatever. How can I learn what
they are? Or if somebody connected me with a different charset, am I forced to
ask him to repeat what he said after a set up a proper charset?

Please don't make it WONTFIX. It's really useful feature. And didn't hear good
reasonning for not having it. BTW, some other IRC clients, e.g. OpenChat,
perform charset change with recoding - exactly the way I'm asking for.

Reopenning...

Comment 8

[James Ross]

The problem is is not for after the recoding, it's before. Data is decoded when
it arrives - directly into the current charset - so if you need it in a
different charset, you're unlikely to be able to do it correctly without still
having the original data (c.f. Mozilla reloads the entire page when changing
charsets).

Obviously, doing the whole thing *is* possible. But it's a lot of effort for us
to code and debug, as well as it'll use a fair chunk of memory to keep all these
original lines. I don't see that we should penalise every user for just the few
who want this - but, what if it only kept last, say, 5 lines? What happens if
someone pastes in 6 before you notice/fix the charset? Also, it *will* take
considerable time to recode an entire channel view. And I mean ages, perhaps
even making it into minutes on my machine.

I'm not going to WONTFIX the bug, but you should be aware that this isn't either
easy to implement, or benificial to most users.

Comment 9

[Max Alekseyev]

Oh, I've got what seems to trouble you. But I have several why's.

Why do not keep a buffer with original data along with recoded one? That just
doubles memory usage and not a big problem. There is also a good idea to have
the buffers bounded in size and to rotate them when filled. Bound should be
specified by user in preferences.

Why it is a problem to recode the whole buffer? Why it's slow?
I was a user of OpenChat client for a long time, and never saw a problem with
recoding, and it always took just less then a second to complete.

And what efforts you talking about? That seems to be pretty easy to implement.

My view of the algorithm: 
1) for each view there are two buffers B1 and B2
2) B1 accumulates original messages as they appear
3) B2 accumulates messages encoded in charset as they appear
4) B2 is shown in the View
5) if user requests charset change, content of B2 is replaced with encoded
content of B1 according to chosen charset

Comment 10

[James Ross]

> Why do not keep a buffer with original data along with recoded one? That just
> doubles memory usage and not a big problem.

Just doubling memory is a problem, though. Or could be.

> There is also a good idea to have the buffers bounded in size and to rotate
> them when filled. Bound should be specified by user in preferences.

We already have limits on this, and it's 500 lines for channels.

> Why it is a problem to recode the whole buffer? Why it's slow?

I don't know, I'd blame Mozilla myself. :)

> I was a user of OpenChat client for a long time, and never saw a problem with
> recoding, and it always took just less then a second to complete.

How much were you recoding? I'm only basing my guesses on it being slow from the
fact that if I change *motif* - that's just changing the stylesheet, not any
content - takes 20 - 30 seconds here. That *is* Mozilla just being slow. If we
have to run all the text through the conversion code too, it *could* be very slow.

Comment 11

[Max Alekseyev]

With default bound of 500 lines per channel the memory is definitely not an
issue. How much memory take two buffers for 500 lines? Almost nothing. ;)

I'm not experienced in Mozilla internal details but from practical point of view
a recoding of 500-line (or even 50000-line) text should NOT be slow. If it's
slow then there should be a bugreport for that. ;)

And last note: usually charset is changed in the beginning of chat session when
there are not too many lines to recode.

Comment 12

[James Ross]

I guess we'll only find out how much Mozilla sucks at recoding if we try... ;)
And I did suspect you'd only usually be recoding a few lines; it's still not
something too many people'd find useful, but the bug can stay and if someone
wants to have a go at coding it, they're welcome to. :)