782761 - MDN needs UTF-8 locale

Status: RESOLVED WONTFIX

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[James Bennett [:ubernostrum]]

MDN is seeing a lot of errors whose tracebacks end in a section like this:

> File "/usr/lib64/python2.6/genericpath.py", line 49, in getsize
>    return os.stat(filename).st_size
> 
> UnicodeEncodeError: 'ascii' codec can't encode characters in position 104-113: 
> ordinal not in range(128)

These come from the fact that we're trying to stat a file with non-ASCII characters in its filename. Supporting such filenames is something we almost certainly need to do.

Getting the server on a UTF-8 locale will resolve this.

Comment 1

[Chris Turra [:cturra]]

:ubernostrum - the web heads do appear to be in utf-8 already.

  [[e-mail]3 ~]# echo $LANG
  en_US.UTF-8

Comment 2

[Jake Maul [:jakem]]

we're also able to "touch" filenames with UTF-8 characters in the filename... do you have a way we can replicate this?

Comment 3

[Chris Turra [:cturra]]

to be extra certain, i even created and wrote to a file with python.

[[e-mail]3 ~]$ python
Python 2.6.6 (r266:84292, May  1 2012, 13:52:17) 
[GCC 4.4.6 20110731 (Red Hat 4.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open("mÉ2.txt", "wb")
>>> f.write("writing to utf-8 file.\n");
>>> f.close()
>>> exit()
[[e-mail]3 ~]$ ls
mÉ2.txt  mÉ.txt

Comment 4

[Eric Ziegenhorn :ericz]

Dropping to normal for now.

Comment 5

[Chris Turra [:cturra]]

:ericz - i have confirmed utf-8 files can be written to disk (both from a term and python shell). is there something else i can review for you on the systems? as far as i can tell, everything is setup properly for utf-8 characters on these servers.

Comment 6

[Chris Turra [:cturra]]

:ericz - i am closing this bug r/wontfix since there is nothing on the web servers i can see that should be stopping utf-8 files from being written. please feel free to reopen if there is something further you would like us to review that i may have missed.