Closed Bug 173039 Opened 22 years ago Closed 16 years ago

Problem in reading .ics with entries ending with UTF-8 non-ASCII char

Categories

(Calendar :: General, defect)

Product:
Calendar
Component:
General
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
VERIFIED WORKSFORME

People

(Reporter: t-mozilla, Unassigned)

References

(
URL
)

Details

(Keywords: intl) 

http://www.snowelm.com/~t/temp/temp2.ics
Download the .ics at the above URL to a local file.  
Select [Tools] - [Subscribe to Remote Calendar] from the menu, and choose the
file.  You will see ??????s at Oct 28th, 2002 instead of the Japanese summary. 
Worse, when you update the .ics file by adding another event, the SUMMARY and
CATEGORIES entries are truncated by 1 to 3 bytes.
It seems that this problem happens when the last character of an entry is a
UTF-8 non-ascii character (not always, but in high possibility).
is <A HREF=http://bugzilla.mozilla.org/show_bug.cgi?id=170316>170316</A> a
duplicate?
Status: UNCONFIRMED → NEW
Ever confirmed: true
I think this is a limitation of Windows, not Calendar.  I viewed the file as
text and saw question marks there as well.  The file is detected as EUC-JP
character set.  Mozilla usually uses question marks when the system font does
not have a glyph (picture) for that character.
Default QA Contact for Calendar has changed.  If you wish to remain the QA
contact for this bug, feel free to change it back.
QA Contact: colint → brantgurganus2001
*** Bug 190786 has been marked as a duplicate of this bug. ***
I think I had the same experience. I was continuously loosing the last character
"á" (&aacute;) from the event title "dovolená". It sounds like it is the same
bug.  Now I cannot check it as I am not able to install calendar any more (see
bug 197363). I used Win XP a and linux Red Hat 7.3, both with the same .ics
file, and it happened in both OS's.

I confirm this bug and I think that this is a backend issue. 
Description : 
- When you enter a new event ending with a non ascii character at the end of the
title "dovolená", it is correctly enter in the ICS file and appears correctly. 
- Close the calendar and Mozilla.
- Reopen it, the last character has diseapered. It is still correct in the ICS
file, but calendarEventDisplay.event.title does not return the correct string.
Assignee: mikep → mostafah
Same with some cyrillic characters.
For instance, trailing "&#1103" (&amp;#1103;) and "&#1091;" (&amp;#1091;) render
the whole string unreadable (kind of "????? ??? ????") when ICS file is reopened.
Also under Windows XP.
Bug 226569 may be related.
Keywords: intl
Blocks: 162454
*** Bug 287971 has been marked as a duplicate of this bug. ***
Can some brave soul test this on the trunk?  We have an all-new ICS loading
infrastructure, and an updated libical, so it'd be good to see if the symptoms
have changed.
Using trunk, the item showed up just fine, with japanese chars and all. But when
i tried to edit it (change times) calendar crashed somewhere in libical.
It might display the items fine, but editing does not work. The crash seems to
have disappeared, but there are still issues.
When trying to save the edited event, i get an assertion:
###!!! ASSERTION: U+0080/U+0100 - U+FFFF data lost: 'legalRange', file
/home/michiel/mozhack/tree1/mozilla/js/src/xpconnect/src/xpcconvert.cpp, line 790

After this, the file looks bad. (but i don' know japanese, so i'm not sure...)
On opening it again i get an exception:
[Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE)
[nsIScriptableUnicodeConverter.convertFromByteArray]"  nsresult: "0x80004005
(NS_ERROR_FAILURE)"  location: "JS frame ::
file:///home/michiel/mozhack/tree1/obj/dist/bin/components/calICSCalendar.js ::
anonymous :: line 136"  data: no]
and the event doesn't show up.

(using the ics calendar provider)
(In reply to comment #12)
> It might display the items fine, but editing does not work. The crash seems to
> have disappeared, but there are still issues.
> When trying to save the edited event, i get an assertion:
> ###!!! ASSERTION: U+0080/U+0100 - U+FFFF data lost: 'legalRange', file
> /home/michiel/mozhack/tree1/mozilla/js/src/xpconnect/src/xpcconvert.cpp, line 790

This means that |string| is used where |wstring| or |AUTF8String| has to be used
in the interface definition. 


Seems that during writing, the ics string isn't converted to a bytearray. Should
be done in the onOperationComplete callback for writeICS in calICSCalendar.js
*** Bug 270158 has been marked as a duplicate of this bug. ***
Also i would like to note that if the entry is viewalbe(uncorrupted) copying it
to clipboard can result in a crash
QA Contact: gurganbl → general
Update: The event from http://www.snowelm.com/~t/temp/temp2.ics seems to be displayed fine (local ics file), but opening edit event dialog and pressing ok crashes sunbird.
Tried with Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9a1) Gecko/20051220 Mozilla Sunbird/0.3a1+.
Reassigning all automatically assigned bugs from Mostafa to nobody@m.o

Bugspam filter: TorontoMostafaMove
Assignee: mostafah → nobody
Seems to me the problem has been solved with a new back-end introduced by 0.3a. I do not experience it any more; I tried to import and edit the file from comment [17] with 2006-06-19 ant it did not crushed Sunbird. So the time may have come to close this bug as fixed?

If anyone has a different experience with recent builds (post Sunird 0.3a2), please could tyou comment here? And what is the opinion of the reporter?

Regards,

Vladimír

(In reply to comment #19)
> Seems to me the problem has been solved with a new back-end introduced by 0.3a.
> I do not experience it any more; I tried to import and edit the file from
> comment [17] with 2006-06-19 ant it did not crushed Sunbird. So the time may
> have come to close this bug as fixed?
> 
> If anyone has a different experience with recent builds (post Sunird 0.3a2),
> please could tyou comment here? And what is the opinion of the reporter?

I still crash when editing the event, in line with commnet #17.
(In reply to comment #20)
> I still crash when editing the event, in line with commnet #17.

I see ??????? as the event title, but I don't crash after editing the event using
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a1) Gecko/20060819 Calendar/0.3a2+
Event is displayed ok. Editing, Copy+Paste, Drag'n'Drop work ok. No crash -> WORKSFORME

Using Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a1) Gecko/20060828 Calendar/0.3a2+ and Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9a1) Gecko/20060827 Calendar/0.3a2+.
I seem to be having a similar problem. In reloading a remote .ics via https: I get the following error:

CAL_UTF8_DECODING_FAILED
An error occured while decoding an iCalendar (ics) file as UTF-8. Check that the file, including symbols and accented letters, is encoded using the UTF-8 character encoding.

This same file is viewable with no errors in iCal. Removing and re-subscribing the calendar has no effect.

Client OS: Windows XP
Sunbird version: tested with 0.3A2 and latest Trunk build (2006-09-19).
(In reply to comment #23)
> I seem to be having a similar problem. In reloading a remote .ics via https: I
> get the following error:
> 
> CAL_UTF8_DECODING_FAILED

I can confirm this. 
Same with versions - no known calendar version can handle those files, but can easily corrupt existing ;-)
I have sample, of corrupted file (btw, this file was corrupted by last version of mozilla calendar - last installable version availiable on website, yes, i know, it's not supported, but anyway, the same corruption may occur with latest versions of sunbird).
Why? Take look into this :

konferencijÅ
³ kl

It should be "konferencijø kl", but because sunbird wrapped unicode symbol into two lines (reformatting comment before publishing and adding max line lenght, not counting unicode symbols?) now it has incorrect unicode symbols, and can't be read.
Workarounds to repair broken calendar (as i did) - use iconv to find place where it's broken (availiable on many platforms including windows) (like iconv -f utf-8 -t iso-8859-13 calendarname.ics) and find place where iconv can't convert. Correct line ends with editor, and check again with iconv, until it can be converted into selected charset...
Probably possible fix - check for unicode symbol after lineend before decoding (seem iCal does this) or - never wrap unicode symbols.

If some1 is interested, i have broken copy of calendar availiable, and can send it by email (will not publish here).
I think this is the same problem I have. Each time I open Sunbird, I get this error message: http://img155.imageshack.us/img155/9190/calrf7.png
I use WinXP and Sunbird 0.5

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4pre) Gecko/20070614 Sunbird/0.5


WFM for Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.13pre) Gecko/20080220 Calendar/0.8pre

No problems importing, editing or saving that entry for the locally saved .ics.

Could someone pls test for Win?
I tested with winxp hu, with installed asian charsets, an it works great!!
I can't reproduce the bug. German Umlauts work fine.
WFM per comment#27 and own test on WinXP.
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → WORKSFORME
Verifying as per comments 26-29.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.