672029 - [compare-locales] Unable to parse .properties files with lots of comments at the end

Status: RESOLVED FIXED

(Mozilla Localizations :: Infrastructure, defect)

Description

[Florian Quèze [:florian]]

When attempting to read this file https://hg.instantbird.org/l10n/pt-PT/file/ccfb2f1354e3/purple/myspace.properties compare-locales makes Python take 100% of the CPU for a very long time and ends up killed by our buildbot after 20 minutes without any output.

Comment 1

[Axel Hecht [:Pike]]

I thought I had a bug on this, but I can't find it.

The problem is that the regular expression in http://hg.mozilla.org/l10n/compare-locales/file/15f42a0ed926/lib/Mozilla/Parser.py#l268 doesn't cope will with lengthy comments. What in

re.compile('^(\s*)((?:[#!].*?\n\s*)*)(([^#!\s\n][^=:\n]*?)\s*[:=][ \t]*((?:[^\\\\]|\\\\.)*?))([ \t]*$\n?)',re.M|re.S)

makes that thing not scale, oh I don't know.

Comment 3

[Mike Hommey [:glandium]]

Attachment: Patch

This solves the issue for me. AFAICS, there is no reason why re.S ought to be used for that regex (that is, no reason for . to be including '\n')

Comment 4

[Axel Hecht [:Pike]]

Comment on attachment 594988 [details] [diff] [review]
Patch

Thanks for trying to patch this.

Sadly, this patch makes Mozilla/tests/unitProperties.py fail. In a rather significant way, too, seems that the value turns into the complete line.

Comment 5

[Axel Hecht [:Pike]]

I guess I have a patch. Will take some time to regression test this, because, well, the old parser takes ages to create the reference data :-)

Comment 6

[Axel Hecht [:Pike]]

http://hg.mozilla.org/l10n/compare-locales/rev/7ad5db5f3c8c has the parser fix, http://hg.mozilla.org/l10n/compare-locales/rev/c788da422d51 landed the gecko tests before that. FIXED, and released as part of 0.9.5

Comment 7

[Axel Hecht [:Pike]]

PS: The latest bn-IN build on the dashboard took 23 seconds like the rest, instead of 3 minutes+