474657 - Character encoding problems in tinderbox logs

Status: RESOLVED FIXED

(Webtools Graveyard :: Tinderbox, defect)

Description

[Blake Kaplan (:mrbkap) (inactive)]

Firefox currently displays tinderbox logs such as http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1232565199.1232566177.18853.gz as ISO-8859-1 when the contents (especially the GCC error messages) are actually in UTF-8. It would be nice if tinderbox could send the logs in their original character sets.

I'm leaving this as UNCONFIRMED, since I'm not entirely sure it's tinderbox's fault.

Comment 1

[Reed Loden [:reed]]

http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/webtools/tinderbox/showlog.cgi&rev=1.32&root=/cvsroot&mark=141#139

Change |Content-type: text/html| to |Content-type: text/html; charset=utf-8|?

Comment 2

[Reed Loden [:reed]]

http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/webtools/tinderbox/showlog.cgi&rev=1.32&root=/cvsroot&mark=141,175#139

Comment 4

[Boris Zbarsky [:bzbarsky]]

Confirming based on dup and such.

Comment 5

[cls]

Attachment: v1.0

Force all generated text pages to use charset=utf-8

Commited on tbox1_20080527_cls_branch

http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=&branchtype=match&dir=mozilla%2Fwebtools%2Ftinderbox&file=&filetype=match&who=cls%25seawood.org&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2009-02-10+22%3A09%3A00&maxdate=2009-02-10+22%3A10%3A00&cvsroot=%2Fcvsroot

Comment 6

[:Gavin Sharp [email: [e-mail]]]

Attachment: against trunk

Comment 7

[:Gavin Sharp [email: [e-mail]]]

mozilla/webtools/tinderbox/addnote.cgi 	1.28
mozilla/webtools/tinderbox/admintree.cgi 	1.42
mozilla/webtools/tinderbox/doadmin.cgi 	1.41
mozilla/webtools/tinderbox/showbuilds.cgi 	1.198
mozilla/webtools/tinderbox/showbuilds.pl 	1.37
mozilla/webtools/tinderbox/showlog.cgi 	1.34
mozilla/webtools/tinderbox/tbglobals.pl 	1.71

Comment 8

[Phil Ringnalda (:philor)]

Verified on tinderbox-stage.

Comment 9

[(not currently active) Ted Mielczarek]

Did this ever make it to production tinderbox?

Comment 10

[Phil Ringnalda (:philor)]

Yep, according to both bug 524334 and LiveHTTPHeaders.

Comment 11

[Robert Helmer [:rhelmer]]

Unfortunately setting the content-type to UTF-8 seems to cause fairly severe performance problems with Firefox loading large files (like build logs) due to bug 566066.

This is a lot more apparent from testing bug 390341 (enable gzip for tinderbox logs), see bug 390341 comment 29 and bug 390341 comment 30 for a specific example. The first locks up Firefox for minutes on my MBP, the other takes a few seconds. Same file, different content-type.

Vlad suggests backing out the change introduced here (bug 390341 comment 37) and possibly changing the locale on the build machines (bug 390341 comment 38) as an alternate solution to this bug.

Fixing bug 566066 would make this a moot point, but I'm not sure if the amount of work there is known or assigned to anyone at this time.

Comment 12

[Phil Ringnalda (:philor)]

Straight backing out the patch here would put us back in the previous state, where we loaded logs as either iso-8859-1, or the encoding of the page where you clicked a link to them, until we hit a character which wouldn't fit in iso-8859-1, at which point the entire log would reload. That's not an acceptable option. If you want a new bug to "Intentionally serve tinderbox logs with an encoding which is not capable of representing all the characters in the log to work around a rendering bug" (keeping in mind that showlog.cgi also shows test logs that include Thai linebreaking tests, not just GCC slanted quotes), please file a new bug with that as the summary rather than reopening this.

Comment 13

[Robert Helmer [:rhelmer]]

(In reply to comment #12)
> Straight backing out the patch here would put us back in the previous state,
> where we loaded logs as either iso-8859-1, or the encoding of the page where
> you clicked a link to them, until we hit a character which wouldn't fit in
> iso-8859-1, at which point the entire log would reload. That's not an
> acceptable option. If you want a new bug to "Intentionally serve tinderbox logs
> with an encoding which is not capable of representing all the characters in the
> log to work around a rendering bug" (keeping in mind that showlog.cgi also
> shows test logs that include Thai linebreaking tests, not just GCC slanted
> quotes), please file a new bug with that as the summary rather than reopening
> this.

"Intentionally serve tinderbox logs with an encoding which is not capable of representing all the characters in the log to work around a rendering bug" does not sound like a desirable outcome to me, so I am not going to file that. I'll leave this closed though, we can discuss further in other channels.

A new bug that might make sense would be to be able to specify the encoding in the build email to tinderbox server, instead of forcing UTF-8 for everything. 

In any case it's at least very useful to know that it's now UTF-8 for everything and that test logs depend on this, thanks.