577310 - prefer ascii over unicode for build logs

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Robert Helmer [:rhelmer]]

As noticed in bug 474657 and bug 446505, gcc is putting unicode characters into build logs.

Bug 474657 makes tinderbox set the content-type to UTF-8 for all logs, which is much heavier than ASCII (especially due to bug 566066).

I don't think there's any reason for gcc to emit unicode, it should be told to use ASCII only by setting the locale to C by setting LC_ALL=C in the environment per bug 390341 comment 38.

Please note that this isn't necessarily specific to bug 390341, this is going to be a problem no matter how the build logs are served, if they are intended to be viewed in the browser.

Comment 1

[timeless]

Attachment: like this?

is this right? i don't have any way of testing this :(

Comment 2

[John Ford [:jhford] CET/CEST Berlin Time]

that looks like it would only take effect for mobile builds and not properly because of how we invoke scratchbox.

Comment 3

[timeless]

@@ -309,6 +315,7 @@ PLATFORM_VARS = {
and following should be for non scratchbox stuff.

hrm, i'll have to find a scratchbox log and see if it's affected by this, if not, then i can drop those bits....

Comment 4

[[e-mail] (:bhearsum)]

The PLATFORM_VAR parts seem fine to me. If you drop the MOBILE_PLATFORM_VARS stuff for now we could land this for the other platforms while that gets figured out.

Comment 5

[timeless]

Attachment: patch for PLATFORM_VARS

bhearsum said he'd land this since it requires coordinating with buildbot

Comment 6

[[e-mail] (:bhearsum)]

Comment on attachment 457818 [details] [diff] [review]
patch for PLATFORM_VARS

I'll try to get this landed today.

Comment 7

[John Ford [:jhford] CET/CEST Berlin Time]

specifically, i don't think we pass the build time environment to scratchbox during the build step.  We can land it for all non-scratchbox builds (android and desktop builds).

Comment 8

[[e-mail] (:bhearsum)]

I'll land this in the downtime tomorrow.

Comment 9

[[e-mail] (:bhearsum)]

Attachment: add LC_ALL for non-scratchbox OS' which use gcc

I'll be landing this tomorrow morning.

Comment 10

[[e-mail] (:bhearsum)]

Comment on attachment 459110 [details] [diff] [review]
add LC_ALL for non-scratchbox OS' which use gcc

changeset:   2737:8d80f9421169

Comment 11

[[e-mail] (:bhearsum)]

Anything else we need to do here?

Comment 12

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Filed bug 581164 on the remaining force-ascii issue.

Comment 13

[[e-mail] (:bhearsum)]

i'm pretty sure we've got LC_ALL in all the right places, so it sounds like we're done here.