Open
Bug 1258613
Opened 10 years ago
Updated 4 years ago
Incorrect rendering of attached Japanese filename (HFS+ NFD form sent from Apple, and shown in NFC form incorrectly if the font substition occurs because used font does not support combining charcter )
Categories
(Thunderbird :: Message Compose Window, defect)
Tracking
(Not tracked)
NEW
People
(Reporter: ishikawa, Unassigned)
References
Details
Attachments
(2 files)
Incorrect rendering of file name by TB
Incorrect rendering of attached Japanese filename (HFS+ NFD form sent from Apple, and shown under linux in NFC form)
Prerequisite Background:
Apple uses a file system called HFS+ which uses UTF-8 NFD form
(Normallization Form Decompressed form) for expressing the file path
name in UTF8. When an device running Apple's OS sends an e-mail with file
attachment (from HFS+), it seems that the filename will be expressed in NFD form.
(I don't know which mail client people use to send me such e-mails. But at least a couple of people
sent me e-mails that show the problematic behavior and presumably they use the stock Apple-supplied mail client.
Both uses Apple note PC.)
OTOH, Linux or Windows uses UTF-8 NFC form (Normalization Form
Compressed form) to represent file names in the file system.
The major difference between NFD and NFC is the handling of diacritic
marks, etc. interposed on a character. NFC (used by HFS+) handles the
original character without the mark(s) as a separate character and
follows it with the code points that stand for the accent marks,
etc. to represent the composite character.
NFD (linux, windows) pickes up the code point for composite character
(the original character and the accenet marks, etc. combined) if such
a code point exists (and usually such composite character DO exist in unicode).
Rendering Problem:
Thunderbird under linux mishandles the rendering of the attached file
name if it is sent from Apple device and the file name contains a
letter with diacritic mark(s) (that is, the file name is represented in
NFD form).
I noticed the problem with the use of Japanese diacritic mark '゙'.
TB combines the diacritic mark with the following letter incorrectly
instead of the correct preceding letter (!).
Case in point:
I noticed this from a real world example.
Somebody sent me a file that explains how to get on a bus from Narita Int'l airport.
I am attaching a figure that shows the problem.
The string marked with RED underline is the incorrect rendering of the file name.
The string marked with GREEN underline is the correct rendering of the file name.
Bus is called バス in Japanese.
Note the first letter is considered as a combination of
ハ (ha) with diacritic mark '゙' called in Unicode as "COMBINING
KATAKANA-HIRAGANA VOICED SOUND MARK". The composite characgter is バ (ba).
There *IS* a single code point for the composite character, バ.
In NFC encoding, the code point for バ (e3 83 90) shall be used.
Apple devices generate NFD encoding and
ハ (e3 83 8f) followed by '゙' (e3 82 99 ) is used for file name representation for the character "バ".
Problematic Symptom:
Problem is that, when such a message from Apple device is received, TB
under linux (38.6) produced incorrect rendering of the combined
character. TB tries to combine '゙' with the FOLLOWING charactger
(INCORRECT) instead of the PRECEDING character (CORRECT) and thus
produces an incorrect rendering of a file name.
In my case from the above, the title of the attachment should be shown as:
Correct rendering: 20160302バスの乗り方.zip
But TB showed an incorrect
rendering: 20160302ハズの乗り方.zip
In the incorrect case, instead of the correct バス[＝ ハ＋゙ followed
by ス (su)], an incorrect string, ハズ (= ハ (ha) followed by ス + ゙
(zu)) is produced. So TB, at least, tries to combine the "゙" with an
original character without such mark, but picks up the wrong character
for composition.
Obviously only the file name in NFD form produced on alien system is
shown incorrectly.
Locally generated strings such as the one I type into the main message window
presumably uses NFC form and so not vulnerable to incorrect rendering.
Real world example:
Following is the excerpt of the mime data in a message that contains only an
attachment with the problematic file path, which I noticed.
This is basically the mail body that shows the incorrect
attachment file name in the attached picture.
E.g:
--------------080107070208060409050106 Content-Type: text/plain;
charset="utf-8" Content-Transfer-Encoding: 7bit
--------------080107070208060409050106
Content-Type: application/zip;
name="=?UTF-8?B?MjAxNjAzMDLjg4/jgpnjgrnjga7kuZfjgormlrkuemlw?="
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
filename*0*="utf-8''%32%30%31%36%30%33%30%32%E3%83%8F%E3%82%99%E3%82%B9%E3";
filename*1*="%81%AE%E4%B9%97%E3%82%8A%E6%96%B9%2E%7A%69%70"
UEsDBAoAAAAAACm9YkgAAAAAAAAAAAAAAAANAAAAg2+DWILMj+aC6JX7L1BLAwQUAAAACADV
....
CAAovWJIFAEQxFUmAAAApAAAGQAAAAAAAAAAACAAtoEoGAEAg2+DWILMj+aC6JX7LzkugruC
zJG8LmRvY1BLBQYAAAAACwALAAEDAAC0PgEAAAA=
--------------080107070208060409050106--
cf. Code used in the filename:
filename*0*="utf-8''%32%30%31%36%30%33%30%32%E3%83%8F%E3%82%99%E3%82%B9%E3";
2 0 1 6 0 3 0 2 ハ ゙ ス
filename*1*="%81%AE%E4%B9%97%E3%82%8A%E6%96%B9%2E%7A%69%70"
の 乗 り 方 . z i p
32 2
30 0
31 1
36 6
30 0
33 3
30 0
32 2
e3 83 8f ハ
e3 82 99゙
e3 82 b9 ス
e3 8a ae の
e4 b9 97 乗
e3 82 8a り
e6 96 b9 方
2e .
7a z
69 i
70 p
The combinatin of ハ + ゙
e3 83 8f ハ
e3 82 99 ゙
should be rendered as equivalent to the glpyh of the single code point
e3 83 90 バ
Here, TB is wrong in that it tries to combine "゙" with the following letter "ス", and
produced "ズ" (zu).
In this particular example, there *IS* a code point for ス + "゙" since it is a valid Japanese character.
When I first noticed the problem, the combination of a character and "゙" is not legal (there is no such character) and
so what TB showed looked very strange.
Technologically speaking, it may be a mere bug (probably very easy to fix), but
usage-wise, this is a grave issue.
No data-loss, but very embarassing bug indeed.
Combination of letters may produce very funny and embarrasing misrendering (!)
and this bug can be noticed by NON-TECH-SAAVY ORDINARY USERS VERY EASILY and makes them wonder
who is writing the code for TB (!!!).
TIA
PS: A quick lookup of UT8 code to Japanese character can be found, say,
http://orange-factory.com/sample/utf8/code3-e3.html
|
Reporter | |
Comment 1•10 years ago
|
||
[I noticed that the standalone diacritic mark may not be easily displayed under windows. So
what I wrote in the previous comment may not quite sane to some.]
I hasten to add that I observed the issue under Debian GNU/Linux.
It is possible that there may be an issue in Debian's iconv, etc.
I checked a few programs under Debian that handles mime-encoded filename, etc.
One is uudeview, and the other is nkf.
Both programs seem to correctly parse the file names as series of octets without interpreting (but not try to produce a composite character for バ）。
This means, for example, instead of a single code point for the composite charactert バ, I get
two code points of ハ and the diacritic mark (I have realized if I put a diacritic mark here, the Japanese windows 10 got upset somewhat and I am not sure what is displayed there. I originally wrote the bug memo in GNU Emacs running under linux, and there I had no issue about it.
But I noticed that shell running under a terminal window emulator under X(Debian GNU/linux 64-bit)
shows the same buggy behavior ALTHOUGH the resulting file name stored in the file system is correct !?
So I have to be careful because the DISPLAY rendering under Windows (which lao uses NFC for file pathname) may be different from Linux's.
Under Debian GNU/Linux, I observe the two different behavior of display.
case 1: under xfce4-terminal 0.6.3 terminal emulator.
I ran shell (bash) and fuond that the file name is DISPLAYED with incorrect ハズ string.
case 2: inside Emacs's shell buffer mode, which does not seem to try to convert NFD to NFC string,
the path name contains the decompressed ハ + diacrtic mark (I found if I copy and paste the
string in the Emacs Shell buffer into this firefox text area to submit to bugzilla, the string is automagically shown as "バ" which is clever of Windows10, but not quite helpful in bug reporting here.
So I am attaching what happens in GNU Emacs Shell buffer window and the terminal window (case 1)
for explanation.
Right hand side shows the terminal interaction. It shows the same problem as TB has.
Left hand side shows the GNU Emacs Shell buffer mode window.
It shows the characters corresponding to the code points in the file name
one by one without decompression (i.e., composition).
TIA
|
|
Reporter | |
Comment 2•10 years ago
|
||
An interesting observation.
When the file with NFD pathname is created under /tmp window of linux,
I copied it to Dropbox (the file sharing application) directory by cp command.
The file path in question was converted to NFC format. code point for バ replaced the two code points for ハ and the diacritic mark.
When I tried to copy the file from /tmp to Dropbox directory again,
dropbox daemon cleverly re-created a copy of the file with a Japanese name of
20160302バスの乗り方 (Unicode エンコードの競合).zip
the string within the parenthesis says competition/collision of Unicode encodings (!)
It warns of me saying that there is a file with a file path of which visual presentation will collide with
the visual presentation of the newly copied file. Clever, huh?
(I have been vaguely aware of the encoding issue because my Dropbox directory occasionally produced
ｔhe warnings of filename conflicts before. But only when I see a very strange pathname shown by TB, I got down to the bottom of the story.)
So this thorny issue of file sharing between Apple HFS+ and Windows/Linux are known and
handled in this manner by Dropbox.
Here we have to do something about DISPLAY of such NFD name by TB on linux/Windows where NFC is used.
TIA
|
|
Reporter | |
Comment 3•10 years ago
|
||
I also found that the recent TB under Windows 10 does not show the problem.
Since linux's file system automagically convert NFD form pathname into NFC form path name from what I observed by looking at the creation of NFC form pathname when I copy (cp) an NFD pathname under /tmp to a different directory, I think TB needs to sanitize a pathname sent from a foreign system into a locally supported filename before displaying it (?)
At least TB under linux CAN save the file with NFD form (albeit a strange looking display in GNU Emacs shell buffer mode and incorrect display inside a terminal emulator window), and that file can be accessed by shell script, et al., it is a matter of design decision
whether we should normalize the pathname of an attachment to a locally supported and yet as universal as possible form.
Come to think of it, BTW, does TB under Windows try to do something clever if I send someone an e-mail attachment with a device name as its principal part of a filename?
For example, I created a linux source tree under a share that is managed under ownCloud file share program. I tried to delete the source tree after use, but the tree could not be wiped out completely because there is a file called "aux.c" and "aux" being a device name, unless I write a short program and use so called long format pathname (or something like that in MS speak), I can't touch the file at all.
So there is a wider issue of sanitized file name for each platform including the issue of device name that may conflict with local OS, but at least
we can start with showing the pathname correctly for a professionally acceptable mail client.
TIA
|
|
Reporter | |
Comment 4•10 years ago
|
||
For a test, I tried to attach "aux.c" to an outgoing e-mail, but under Windows 10, I can't even select the file for attachment since I don't have a permission to do so (?).
File selector complained. I will try sending such an e-mail from a linux client and see what happens on Windows client when I try to save it, etc. But that will be another bugzilla.
First, I want the correct display of attached file's name.
|
|
Reporter | |
Comment 5•10 years ago
|
||
I am happy to report 47.0a1 (03-10) daily under linux does not show the problem and
so it is fixed in the current C-C tree.
I wonder though when it was fixed (38 -> ??? -> 47 )
|
|
Reporter | |
Comment 6•9 years ago
|
||
Well, TB 45.0 (linux 32-bit) still does not fix it, and so the users will see the fix in 47.0 eventually.
Hmm... But that means an ordinary user won't see the fix until next year?
|
||
Comment 7•9 years ago
|
||
We are currently being pretty aggressive about backporting fixes from > 45.0 to the 45.0esr repo that we release from. But that assumes you can identify the code that fixes the problem. Otherwise, TB 52.0 is due near the beginning of 2018.
For Linux, keep in mind that 45.0 still uses gtk2 but subsequent versions use gtk3, so when I see "rendering" I wonder if that is the cause. That is NOT something we are going to backport.
|
|
Reporter | |
Comment 8•9 years ago
|
||
(In reply to Kent James (:rkent) from comment #7)
> We are currently being pretty aggressive about backporting fixes from > 45.0
> to the 45.0esr repo that we release from. But that assumes you can identify
> the code that fixes the problem. Otherwise, TB 52.0 is due near the
> beginning of 2018.
I will try to find the real cause of the bug and where it was fixed by using
some binaries available.
But as you noted below:
>
> For Linux, keep in mind that 45.0 still uses gtk2 but subsequent versions
> use gtk3, so when I see "rendering" I wonder if that is the cause. That is
> NOT something we are going to backport.
I have a strong suspicion that GUI toolkit version may play a role here.
In any case, I will try to bi-sect the released beta, etc. to figure out where
the error is fixed.
TIA
|
|
Reporter | |
Comment 9•9 years ago
|
||
(In reply to ISHIKAWA, Chiaki from comment #8)
> (In reply to Kent James (:rkent) from comment #7)
> > We are currently being pretty aggressive about backporting fixes from > 45.0
> > to the 45.0esr repo that we release from. But that assumes you can identify
> > the code that fixes the problem. Otherwise, TB 52.0 is due near the
> > beginning of 2018.
>
> I will try to find the real cause of the bug and where it was fixed by using
> some binaries available.
>
> But as you noted below:
>
> >
> > For Linux, keep in mind that 45.0 still uses gtk2 but subsequent versions
> > use gtk3, so when I see "rendering" I wonder if that is the cause. That is
> > NOT something we are going to backport.
>
> I have a strong suspicion that GUI toolkit version may play a role here.
>
> In any case, I will try to bi-sect the released beta, etc. to figure out
> where
> the error is fixed.
>
> TIA
Mystery!
64bit version of TB does not show the buggy symptom, but 32-bit version did.
Details:
My report in comment 6 was based on linux 32-bit in a PC that runs Debian GNU/Linux 64-bit with standard Debian packages:
"Well, TB 45.0 (linux 32-bit) still does not fix it, and so the users will see the fix in 47.0 eventually.
Hmm... But that means an ordinary user won't see the fix until next year? "
But today I began checking this and as a starter, I checked linux 64-bit which I just installed in another PC, but the
same Debian GNU/Linux 64-bit (albeit a slightly aggressive installation policy to install so called testing repository) to test
unrelated Bug 1257058 - shutdown crash after failed imap login attempt with no password. shutdownhang | WaitForSingleObjectEx | WaitForSingleObject | _PR_MD_WAIT_CV | _PR_WaitCondVar | PR_WaitCondVar | mozilla::CondVar::Wait | nsEventQueue::GetEvent
With 64-bit TB 45.0 under this Debian GNU/Linux 64-bit,
the bug does not happen and display is correct(!?).
So the problem may be with Debian GNU/Linux libraries?
I tried to see what version of libraries:
This is for 64-bit TB under Debian GNU/Linux 64-bit.
ldd ~/thunderbird/thunderbird
linux-vdso.so.1 (0x00007fffd7d9d000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1d2d45c000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1d2d258000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1d2d04f000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f1d2ccd4000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1d2c9d3000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1d2c7bc000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1d2c413000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1d2d697000)
Obviously, dynamically loaded libraries are opened only at run-time and so not shown with ldd. (Is there a command to show which libraries are loaded during runtime of a process)?
In any case, I would like to check which versions of the libraries are loaded on these two PCs to figure out what buggy/correct libraries are linked so that Debian GNU/Linux and presumably Ubuntu users are spared of the problem.
TIA
|
|
Reporter | |
Comment 10•9 years ago
|
||
I used the command mentioned in the following URL to obtain the list of dynamically loaded libraries when the problematic path name was shown on the screen under linux 64-bit.
http://stackoverflow.com/questions/5103443/how-to-check-what-shared-library-is-loaded-at-run-time
cat /proc/20912/maps | awk '{print $6}' | grep '\.so' | sort | uniq
/home/ishikawa/.thunderbird/wczlstuh.default/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/components/libcalbasecomps.so
/home/ishikawa/thunderbird/libfreebl3.so
/home/ishikawa/thunderbird/libldap60.so
/home/ishikawa/thunderbird/libldif60.so
/home/ishikawa/thunderbird/liblgpllibs.so
/home/ishikawa/thunderbird/libmozsqlite3.so
/home/ishikawa/thunderbird/libnspr4.so
/home/ishikawa/thunderbird/libnss3.so
/home/ishikawa/thunderbird/libnssckbi.so
/home/ishikawa/thunderbird/libnssdbm3.so
/home/ishikawa/thunderbird/libnssutil3.so
/home/ishikawa/thunderbird/libplc4.so
/home/ishikawa/thunderbird/libplds4.so
/home/ishikawa/thunderbird/libprldap60.so
/home/ishikawa/thunderbird/libsmime3.so
/home/ishikawa/thunderbird/libsoftokn3.so
/home/ishikawa/thunderbird/libssl3.so
/home/ishikawa/thunderbird/libxul.so
/lib/x86_64-linux-gnu/ld-2.19.so
/lib/x86_64-linux-gnu/libattr.so.1.1.0
/lib/x86_64-linux-gnu/libbsd.so.0.8.2
/lib/x86_64-linux-gnu/libc-2.19.so
/lib/x86_64-linux-gnu/libcap.so.2.24
/lib/x86_64-linux-gnu/libdbus-1.so.3.14.6
/lib/x86_64-linux-gnu/libdl-2.19.so
/lib/x86_64-linux-gnu/libexpat.so.1.6.0
/lib/x86_64-linux-gnu/libgcc_s.so.1
/lib/x86_64-linux-gnu/libgcrypt.so.20.0.5
/lib/x86_64-linux-gnu/libglib-2.0.so.0.4800.0
/lib/x86_64-linux-gnu/libgpg-error.so.0.17.0
/lib/x86_64-linux-gnu/libjson-c.so.2.0.0
/lib/x86_64-linux-gnu/liblzma.so.5.0.0
/lib/x86_64-linux-gnu/libm-2.19.so
/lib/x86_64-linux-gnu/libnsl-2.19.so
/lib/x86_64-linux-gnu/libnss_files-2.19.so
/lib/x86_64-linux-gnu/libpcre.so.3.13.1
/lib/x86_64-linux-gnu/libpng12.so.0.54.0
/lib/x86_64-linux-gnu/libpthread-2.19.so
/lib/x86_64-linux-gnu/libresolv-2.19.so
/lib/x86_64-linux-gnu/librt-2.19.so
/lib/x86_64-linux-gnu/libselinux.so.1
/lib/x86_64-linux-gnu/libsystemd.so.0.3.1
/lib/x86_64-linux-gnu/libtinfo.so.5.9
/lib/x86_64-linux-gnu/libudev.so.1.6.4
/lib/x86_64-linux-gnu/libutil-2.19.so
/lib/x86_64-linux-gnu/libuuid.so.1.3.0
/lib/x86_64-linux-gnu/libwrap.so.0.7.6
/lib/x86_64-linux-gnu/libz.so.1.2.8
/opt/VBoxGuestAdditions-5.0.16/lib/VBoxOGL.so
/opt/VBoxGuestAdditions-5.0.16/lib/VBoxOGLcrutil.so
/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so
/usr/lib/x86_64-linux-gnu/gconv/UTF-16.so
/usr/lib/x86_64-linux-gnu/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-png.so
/usr/lib/x86_64-linux-gnu/gio/modules/libdconfsettings.so
/usr/lib/x86_64-linux-gnu/gio/modules/libgioremote-volume-monitor.so
/usr/lib/x86_64-linux-gnu/gio/modules/libgvfsdbus.so
/usr/lib/x86_64-linux-gnu/gtk-2.0/2.10.0/engines/libxfce.so
/usr/lib/x86_64-linux-gnu/gtk-2.0/2.10.0/immodules/im-xim.so
/usr/lib/x86_64-linux-gnu/gvfs/libgvfscommon.so
/usr/lib/x86_64-linux-gnu/libFLAC.so.8.3.0
/usr/lib/x86_64-linux-gnu/libGL.so.1.2.0
/usr/lib/x86_64-linux-gnu/libICE.so.6.3.0
/usr/lib/x86_64-linux-gnu/libLLVM-3.5.so.1
/usr/lib/x86_64-linux-gnu/libSM.so.6.0.1
/usr/lib/x86_64-linux-gnu/libX11-xcb.so.1.0.0
/usr/lib/x86_64-linux-gnu/libX11.so.6.3.0
/usr/lib/x86_64-linux-gnu/libXau.so.6.0.0
/usr/lib/x86_64-linux-gnu/libXcomposite.so.1.0.0
/usr/lib/x86_64-linux-gnu/libXcursor.so.1.0.2
/usr/lib/x86_64-linux-gnu/libXdamage.so.1.1.0
/usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
/usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
/usr/lib/x86_64-linux-gnu/libXfixes.so.3.1.0
/usr/lib/x86_64-linux-gnu/libXi.so.6.1.0
/usr/lib/x86_64-linux-gnu/libXinerama.so.1.0.0
/usr/lib/x86_64-linux-gnu/libXrandr.so.2.2.0
/usr/lib/x86_64-linux-gnu/libXrender.so.1.3.0
/usr/lib/x86_64-linux-gnu/libXss.so.1.0.0
/usr/lib/x86_64-linux-gnu/libXt.so.6.0.0
/usr/lib/x86_64-linux-gnu/libXtst.so.6.1.0
/usr/lib/x86_64-linux-gnu/libXxf86vm.so.1.0.0
/usr/lib/x86_64-linux-gnu/libasound.so.2.0.0
/usr/lib/x86_64-linux-gnu/libasyncns.so.0.3.1
/usr/lib/x86_64-linux-gnu/libatk-1.0.so.0.22009.1
/usr/lib/x86_64-linux-gnu/libcairo.so.2.11400.4
/usr/lib/x86_64-linux-gnu/libcanberra.so.0.2.5
/usr/lib/x86_64-linux-gnu/libdatrie.so.1.3.3
/usr/lib/x86_64-linux-gnu/libdbus-glib-1.so.2.3.3
/usr/lib/x86_64-linux-gnu/libdrm.so.2.4.0
/usr/lib/x86_64-linux-gnu/libdrm_nouveau.so.2.0.0
/usr/lib/x86_64-linux-gnu/libdrm_radeon.so.1.0.1
/usr/lib/x86_64-linux-gnu/libedit.so.2.0.53
/usr/lib/x86_64-linux-gnu/libelf-0.163.so
/usr/lib/x86_64-linux-gnu/libffi.so.6.0.4
/usr/lib/x86_64-linux-gnu/libfontconfig.so.1.8.0
/usr/lib/x86_64-linux-gnu/libfreetype.so.6.12.1
/usr/lib/x86_64-linux-gnu/libgconf-2.so.4.1.5
/usr/lib/x86_64-linux-gnu/libgdk-x11-2.0.so.0.2400.30
/usr/lib/x86_64-linux-gnu/libgdk_pixbuf-2.0.so.0.3200.3
/usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.4800.0
/usr/lib/x86_64-linux-gnu/libglapi.so.0.0.0
/usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.4800.0
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.4800.0
/usr/lib/x86_64-linux-gnu/libgraphite2.so.3.0.1
/usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0.4800.0
/usr/lib/x86_64-linux-gnu/libgtk-x11-2.0.so.0.2400.30
/usr/lib/x86_64-linux-gnu/libharfbuzz.so.0.10000.1
/usr/lib/x86_64-linux-gnu/libltdl.so.7.3.1
/usr/lib/x86_64-linux-gnu/libogg.so.0.8.2
/usr/lib/x86_64-linux-gnu/libpango-1.0.so.0.3800.1
/usr/lib/x86_64-linux-gnu/libpangocairo-1.0.so.0.3800.1
/usr/lib/x86_64-linux-gnu/libpangoft2-1.0.so.0.3800.1
/usr/lib/x86_64-linux-gnu/libpixman-1.so.0.33.6
/usr/lib/x86_64-linux-gnu/libpulse.so.0.18.0
/usr/lib/x86_64-linux-gnu/libsndfile.so.1.0.25
/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
/usr/lib/x86_64-linux-gnu/libtdb.so.1.3.8
/usr/lib/x86_64-linux-gnu/libthai.so.0.2.4
/usr/lib/x86_64-linux-gnu/libtxc_dxtn_s2tc.so.0.0.0
/usr/lib/x86_64-linux-gnu/libvorbis.so.0.4.8
/usr/lib/x86_64-linux-gnu/libvorbisenc.so.2.0.11
/usr/lib/x86_64-linux-gnu/libvorbisfile.so.3.3.7
/usr/lib/x86_64-linux-gnu/libxcb-dri2.so.0.0.0
/usr/lib/x86_64-linux-gnu/libxcb-dri3.so.0.0.0
/usr/lib/x86_64-linux-gnu/libxcb-glx.so.0.0.0
/usr/lib/x86_64-linux-gnu/libxcb-present.so.0.0.0
/usr/lib/x86_64-linux-gnu/libxcb-randr.so.0.1.0
/usr/lib/x86_64-linux-gnu/libxcb-render.so.0.0.0
/usr/lib/x86_64-linux-gnu/libxcb-shape.so.0.0.0
/usr/lib/x86_64-linux-gnu/libxcb-shm.so.0.0.0
/usr/lib/x86_64-linux-gnu/libxcb-sync.so.1.0.0
/usr/lib/x86_64-linux-gnu/libxcb-xfixes.so.0.0.0
/usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
/usr/lib/x86_64-linux-gnu/libxshmfence.so.1.0.0
/usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-6.0.so
Note that this uses gtk2 and so the issue is probably somewhere else!
Stay tuned when I check the PC where the problem was observed with 32-bit TB on another computer.
TIA
PS: Probably I should be careful about the locale setting, etc.
I will try to see what the possible differences between the two computers...
|
|
Reporter | |
Comment 11•9 years ago
|
||
This is the dynamic library loaded for the execution of TB 45.0 32-bit under Debian GNU/Linux (it support the execution of 32-bit binary if we prepare proper 32-bit libraries for each application).
ishikawa@debian-vbox-ci:/tmp$ cat /proc/27211/maps | awk '{print $6}' | grep '\.so' | sort | uniq
/home/ishikawa/.thunderbird-before/l9d2kdyi.default/extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/components/libcalbasecomps.so
/home/ishikawa/thunderbird/libfreebl3.so
/home/ishikawa/thunderbird/libldap60.so
/home/ishikawa/thunderbird/libldif60.so
/home/ishikawa/thunderbird/liblgpllibs.so
/home/ishikawa/thunderbird/libmozsqlite3.so
/home/ishikawa/thunderbird/libnspr4.so
/home/ishikawa/thunderbird/libnss3.so
/home/ishikawa/thunderbird/libnssckbi.so
/home/ishikawa/thunderbird/libnssdbm3.so
/home/ishikawa/thunderbird/libnssutil3.so
/home/ishikawa/thunderbird/libplc4.so
/home/ishikawa/thunderbird/libplds4.so
/home/ishikawa/thunderbird/libprldap60.so
/home/ishikawa/thunderbird/libsmime3.so
/home/ishikawa/thunderbird/libsoftokn3.so
/home/ishikawa/thunderbird/libssl3.so
/home/ishikawa/thunderbird/libxul.so
/lib/i386-linux-gnu/i686/cmov/libc-2.19.so
/lib/i386-linux-gnu/i686/cmov/libdl-2.19.so
/lib/i386-linux-gnu/i686/cmov/libm-2.19.so
/lib/i386-linux-gnu/i686/cmov/libnsl-2.19.so
/lib/i386-linux-gnu/i686/cmov/libnss_compat-2.19.so
/lib/i386-linux-gnu/i686/cmov/libnss_dns-2.19.so
/lib/i386-linux-gnu/i686/cmov/libnss_files-2.19.so
/lib/i386-linux-gnu/i686/cmov/libnss_nis-2.19.so
/lib/i386-linux-gnu/i686/cmov/libpthread-2.19.so
/lib/i386-linux-gnu/i686/cmov/libresolv-2.19.so
/lib/i386-linux-gnu/i686/cmov/librt-2.19.so
/lib/i386-linux-gnu/ld-2.19.so
/lib/i386-linux-gnu/libdbus-1.so.3.8.13
/lib/i386-linux-gnu/libexpat.so.1.6.0
/lib/i386-linux-gnu/libgcc_s.so.1
/lib/i386-linux-gnu/libglib-2.0.so.0.4200.1
/lib/i386-linux-gnu/libpcre.so.3.13.1
/lib/i386-linux-gnu/libpng12.so.0.50.0
/lib/i386-linux-gnu/libselinux.so.1
/lib/i386-linux-gnu/libuuid.so.1.3.0
/lib/i386-linux-gnu/libz.so.1.2.8
/usr/lib/i386-linux-gnu/gconv/UTF-16.so
/usr/lib/i386-linux-gnu/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-png.so
/usr/lib/i386-linux-gnu/gtk-2.0/2.10.0/immodules/im-xim.so
/usr/lib/i386-linux-gnu/libICE.so.6.3.0
/usr/lib/i386-linux-gnu/libSM.so.6.0.1
/usr/lib/i386-linux-gnu/libX11.so.6.3.0
/usr/lib/i386-linux-gnu/libXau.so.6.0.0
/usr/lib/i386-linux-gnu/libXcomposite.so.1.0.0
/usr/lib/i386-linux-gnu/libXcursor.so.1.0.2
/usr/lib/i386-linux-gnu/libXdamage.so.1.1.0
/usr/lib/i386-linux-gnu/libXdmcp.so.6.0.0
/usr/lib/i386-linux-gnu/libXext.so.6.4.0
/usr/lib/i386-linux-gnu/libXfixes.so.3.1.0
/usr/lib/i386-linux-gnu/libXi.so.6.1.0
/usr/lib/i386-linux-gnu/libXinerama.so.1.0.0
/usr/lib/i386-linux-gnu/libXrandr.so.2.2.0
/usr/lib/i386-linux-gnu/libXrender.so.1.3.0
/usr/lib/i386-linux-gnu/libXt.so.6.0.0
/usr/lib/i386-linux-gnu/libasound.so.2.0.0
/usr/lib/i386-linux-gnu/libatk-1.0.so.0.21409.1
/usr/lib/i386-linux-gnu/libcairo.so.2.11400.0
/usr/lib/i386-linux-gnu/libdatrie.so.1.3.1
/usr/lib/i386-linux-gnu/libdbus-glib-1.so.2.2.2
/usr/lib/i386-linux-gnu/libffi.so.6.0.2
/usr/lib/i386-linux-gnu/libfontconfig.so.1.8.0
/usr/lib/i386-linux-gnu/libfreetype.so.6.11.1
/usr/lib/i386-linux-gnu/libgdk-x11-2.0.so.0.2400.25
/usr/lib/i386-linux-gnu/libgdk_pixbuf-2.0.so.0.3100.1
/usr/lib/i386-linux-gnu/libgio-2.0.so.0.4200.1
/usr/lib/i386-linux-gnu/libgmodule-2.0.so.0.4200.1
/usr/lib/i386-linux-gnu/libgobject-2.0.so.0.4200.1
/usr/lib/i386-linux-gnu/libgraphite2.so.3.0.1
/usr/lib/i386-linux-gnu/libgthread-2.0.so.0.4200.1
/usr/lib/i386-linux-gnu/libgtk-x11-2.0.so.0.2400.25
/usr/lib/i386-linux-gnu/libharfbuzz.so.0.935.0
/usr/lib/i386-linux-gnu/libpango-1.0.so.0.3600.8
/usr/lib/i386-linux-gnu/libpangocairo-1.0.so.0.3600.8
/usr/lib/i386-linux-gnu/libpangoft2-1.0.so.0.3600.8
/usr/lib/i386-linux-gnu/libpixman-1.so.0.32.6
/usr/lib/i386-linux-gnu/libstdc++.so.6.0.20
/usr/lib/i386-linux-gnu/libthai.so.0.2.0
/usr/lib/i386-linux-gnu/libxcb-render.so.0.0.0
/usr/lib/i386-linux-gnu/libxcb-shm.so.0.0.0
/usr/lib/i386-linux-gnu/libxcb.so.1.1.0
ishikawa@debian-vbox-ci:/tmp$
This 32-bit environment shows the incorrect filename handling.
I have to figure out which library is to blame.
This incorrect file name handling has another repercussion.
DonloadIntegration.jsm tries to set the permission of the file TB saves from the attachment(s) of the files.
If I save the attachment using the original apple encoding to a Dropbox directory (a directory which is managed by Dropbox), Dropbox is clever enough to
save the filename into NFC (compressed form with the code point for combined glyph.). So by the time, the permission is being changed, the filename that is on the Dropbox directory and the filename passed to chmod by TB differs and so I have
"no such file exists" error (Unix error 2) in the TB's error console with the incorrectly shown filename as the file that does not exist.
Hmm...
I will investigate more to see what is to blame.
|
|
Reporter | |
Comment 12•9 years ago
|
||
To my surprise, running TB45.0 64-bit on this computer that uses standard (non-testing) packages of Debian GNU/Linux 64-bit also had the incorrect display.
So it seems that there are library binaries that may not show the apple NFD format name correctly.
64-bit version loads the following library at runtime.
cat /proc/833/maps | awk '{print $6}' | grep '\.so' | sort | uniq
/home/ishikawa/thunderbird/libfreebl3.so
/home/ishikawa/thunderbird/libldap60.so
/home/ishikawa/thunderbird/libldif60.so
/home/ishikawa/thunderbird/liblgpllibs.so
/home/ishikawa/thunderbird/libmozsqlite3.so
/home/ishikawa/thunderbird/libnspr4.so
/home/ishikawa/thunderbird/libnss3.so
/home/ishikawa/thunderbird/libnssckbi.so
/home/ishikawa/thunderbird/libnssdbm3.so
/home/ishikawa/thunderbird/libnssutil3.so
/home/ishikawa/thunderbird/libplc4.so
/home/ishikawa/thunderbird/libplds4.so
/home/ishikawa/thunderbird/libprldap60.so
/home/ishikawa/thunderbird/libsmime3.so
/home/ishikawa/thunderbird/libsoftokn3.so
/home/ishikawa/thunderbird/libssl3.so
/home/ishikawa/thunderbird/libxul.so
/lib/x86_64-linux-gnu/ld-2.19.so
/lib/x86_64-linux-gnu/libattr.so.1.1.0
/lib/x86_64-linux-gnu/libc-2.19.so
/lib/x86_64-linux-gnu/libcap.so.2.24
/lib/x86_64-linux-gnu/libdbus-1.so.3.8.13
/lib/x86_64-linux-gnu/libdl-2.19.so
/lib/x86_64-linux-gnu/libexpat.so.1.6.0
/lib/x86_64-linux-gnu/libgcc_s.so.1
/lib/x86_64-linux-gnu/libgcrypt.so.20.0.3
/lib/x86_64-linux-gnu/libglib-2.0.so.0.4200.1
/lib/x86_64-linux-gnu/libgpg-error.so.0.13.0
/lib/x86_64-linux-gnu/libjson-c.so.2.0.0
/lib/x86_64-linux-gnu/liblzma.so.5.0.0
/lib/x86_64-linux-gnu/libm-2.19.so
/lib/x86_64-linux-gnu/libnsl-2.19.so
/lib/x86_64-linux-gnu/libnss_dns-2.19.so
/lib/x86_64-linux-gnu/libnss_files-2.19.so
/lib/x86_64-linux-gnu/libnss_mdns4_minimal.so.2
/lib/x86_64-linux-gnu/libpcre.so.3.13.1
/lib/x86_64-linux-gnu/libpng12.so.0.50.0
/lib/x86_64-linux-gnu/libpthread-2.19.so
/lib/x86_64-linux-gnu/libresolv-2.19.so
/lib/x86_64-linux-gnu/librt-2.19.so
/lib/x86_64-linux-gnu/libselinux.so.1
/lib/x86_64-linux-gnu/libsystemd.so.0.3.1
/lib/x86_64-linux-gnu/libudev.so.1.5.0
/lib/x86_64-linux-gnu/libutil-2.19.so
/lib/x86_64-linux-gnu/libuuid.so.1.3.0
/lib/x86_64-linux-gnu/libwrap.so.0.7.6
/lib/x86_64-linux-gnu/libz.so.1.2.8
/usr/lib/x86_64-linux-gnu/gconv/UTF-16.so
/usr/lib/x86_64-linux-gnu/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-png.so
/usr/lib/x86_64-linux-gnu/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-svg.so
/usr/lib/x86_64-linux-gnu/gio/modules/libgvfsdbus.so
/usr/lib/x86_64-linux-gnu/gtk-2.0/2.10.0/engines/libxfce.so
/usr/lib/x86_64-linux-gnu/gtk-2.0/2.10.0/immodules/im-xim.so
/usr/lib/x86_64-linux-gnu/gvfs/libgvfscommon.so
/usr/lib/x86_64-linux-gnu/libFLAC.so.8.3.0
/usr/lib/x86_64-linux-gnu/libICE.so.6.3.0
/usr/lib/x86_64-linux-gnu/libSM.so.6.0.1
/usr/lib/x86_64-linux-gnu/libX11-xcb.so.1.0.0
/usr/lib/x86_64-linux-gnu/libX11.so.6.3.0
/usr/lib/x86_64-linux-gnu/libXau.so.6.0.0
/usr/lib/x86_64-linux-gnu/libXcomposite.so.1.0.0
/usr/lib/x86_64-linux-gnu/libXcursor.so.1.0.2
/usr/lib/x86_64-linux-gnu/libXdamage.so.1.1.0
/usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
/usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
/usr/lib/x86_64-linux-gnu/libXfixes.so.3.1.0
/usr/lib/x86_64-linux-gnu/libXi.so.6.1.0
/usr/lib/x86_64-linux-gnu/libXinerama.so.1.0.0
/usr/lib/x86_64-linux-gnu/libXrandr.so.2.2.0
/usr/lib/x86_64-linux-gnu/libXrender.so.1.3.0
/usr/lib/x86_64-linux-gnu/libXss.so.1.0.0
/usr/lib/x86_64-linux-gnu/libXt.so.6.0.0
/usr/lib/x86_64-linux-gnu/libXtst.so.6.1.0
/usr/lib/x86_64-linux-gnu/libasound.so.2.0.0
/usr/lib/x86_64-linux-gnu/libasyncns.so.0.3.1
/usr/lib/x86_64-linux-gnu/libatk-1.0.so.0.21409.1
/usr/lib/x86_64-linux-gnu/libbluray.so.1.6.2
/usr/lib/x86_64-linux-gnu/libcairo.so.2.11400.0
/usr/lib/x86_64-linux-gnu/libcanberra.so.0.2.5
/usr/lib/x86_64-linux-gnu/libcroco-0.6.so.3.0.1
/usr/lib/x86_64-linux-gnu/libdatrie.so.1.3.1
/usr/lib/x86_64-linux-gnu/libdbus-glib-1.so.2.2.2
/usr/lib/x86_64-linux-gnu/libffi.so.6.0.2
/usr/lib/x86_64-linux-gnu/libfontconfig.so.1.8.0
/usr/lib/x86_64-linux-gnu/libfreetype.so.6.11.1
/usr/lib/x86_64-linux-gnu/libgconf-2.so.4.1.5
/usr/lib/x86_64-linux-gnu/libgdk-x11-2.0.so.0.2400.25
/usr/lib/x86_64-linux-gnu/libgdk_pixbuf-2.0.so.0.3100.1
/usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.4200.1
/usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.4200.1
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.4200.1
/usr/lib/x86_64-linux-gnu/libgraphite2.so.3.0.1
/usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0.4200.1
/usr/lib/x86_64-linux-gnu/libgtk-x11-2.0.so.0.2400.25
/usr/lib/x86_64-linux-gnu/libharfbuzz.so.0.935.0
/usr/lib/x86_64-linux-gnu/libltdl.so.7.3.0
/usr/lib/x86_64-linux-gnu/libogg.so.0.8.2
/usr/lib/x86_64-linux-gnu/libpango-1.0.so.0.3600.8
/usr/lib/x86_64-linux-gnu/libpangocairo-1.0.so.0.3600.8
/usr/lib/x86_64-linux-gnu/libpangoft2-1.0.so.0.3600.8
/usr/lib/x86_64-linux-gnu/libpixman-1.so.0.32.6
/usr/lib/x86_64-linux-gnu/libpulse.so.0.17.3
/usr/lib/x86_64-linux-gnu/librsvg-2.so.2.40.5
/usr/lib/x86_64-linux-gnu/libsndfile.so.1.0.25
/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.20
/usr/lib/x86_64-linux-gnu/libtdb.so.1.3.6
/usr/lib/x86_64-linux-gnu/libthai.so.0.2.0
/usr/lib/x86_64-linux-gnu/libvorbis.so.0.4.7
/usr/lib/x86_64-linux-gnu/libvorbisenc.so.2.0.10
/usr/lib/x86_64-linux-gnu/libvorbisfile.so.3.3.6
/usr/lib/x86_64-linux-gnu/libxcb-render.so.0.0.0
/usr/lib/x86_64-linux-gnu/libxcb-shm.so.0.0.0
/usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
/usr/lib/x86_64-linux-gnu/libxml2.so.2.9.1
/usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-5.0.so
ishikawa@debian-vbox-ci:/extra/ishikawa/TB-3HG/NEW-COMMSRC$
I will investigate more.
|
|
Reporter | |
Comment 13•9 years ago
|
||
There seems to be a large set of libraries of different versions that
affects the visibility of this bug.
I recently upgraded two PCs that run Debian GNU/Linux 64-bit to use the testing repository mainly (to be the next main distribution named "stretch".)
I will try to report a consistent error behavior: the bug is definitely there and I got hit
with the problem yesterday and today.
It is quite annoying since the particular character sequence appears in a Japanese equivalent of "pamphlet" and when I prepare material for an exhibition, the word appears quite often in attachments people send in :-(
I am going to upload a message with an attachment that shows the symptom. I cannot obviously upload a real world business e-mail and so asked someone to send an empty file with only the problematic name for bug analysis purposes.
TIA
|
|
Reporter | |
Comment 14•8 years ago
|
||
OK, I realized that I forgot to upload the particular file. (As it turned out, MS Office inserts so many private information about the author and the machine the document is created and I had to ask the sender to remove such attributes from the file before sending it in, and I suspect she failed to follow up.)
The bug is still there definitely under linux even with the latest and greatest TB version.
(I am using Debian GNU/Linux.)
So I will ask another person to do create a problematic file for me.
TIA
|
||
Comment 15•8 years ago
|
||
I suspect this occurs when the text is displayed with a font that doesn't support the combining character ゙ and so font fallback comes into effect; the mark is then rendered with a different font from the preceding base character, and its positioning is likely to be badly wrong.
Applying NFC normalization to the text before attempting to render it would avoid this (though it may lead to other issues, so it's not automatically the "right thing" to do).
|
|
Reporter | |
Comment 16•6 years ago
•
|
||
Wow, on Windows 10, TB 32-bit 68.4.1 still has incorrect rendering of attachment filename somehow...
I wonder where and how we should focus on the fix...
Should I file a separate bugzialla since this bugzilla mentions "linux" in the title?
|
|
||
Comment 17•6 years ago
|
||
(In reply to ISHIKAWA, Chiaki (may be slow to respond until Jan 4.) from comment #16)
Wow, on Windows 10, TB 32-bit 68.4.1 still has incorrect rendering of attachment filename somehow...
I wonder where and how we should focus on the fix...Should I file a separate bugzialla since this bugzilla mentions "linux" in the title?
It's almost certainly the same issue as described in comment 15: the font being used to display the filename doesn't support the combining character, font fallback comes into effect, and the mark is mis-positioned. There's no real value in a separate bug, though you could edit the title to remove the specific mention of Linux, or something...
Perhaps the best chance of fixing this would be bug 543200, which would try to ensure that the base character and combining mark are rendered together from the same font.
See Also: → 543200
|
|
Reporter | |
Updated•5 years ago
|
Summary: Incorrect rendering of attached Japanese filename (HFS+ NFD form sent from Apple, and shown under linux in NFC form) → Incorrect rendering of attached Japanese filename (HFS+ NFD form sent from Apple, and shown in NFC form incorrectly if the font substition occurs because used font does not support combining charcter )
|
||
Comment 18•4 years ago
|
||
This does not seem related to CloudFiles but to attachments in general.
Component: FileLink → Message Compose Window
You need to log in
before you can comment on or make changes to this bug.
Description
•