509220 - Malformed UTF-8 characters cause Bugzilla to crash

Status: RESOLVED FIXED

(Bugzilla :: Bugzilla-General, defect)

Description

[Max Kanat-Alexander]

To see the problem, insert a comment with the Unicode character 0xFFFF. The comment sometimes will insert okay (depending on how you're doing this) but it will cause Bugzilla to die with a "Malformed UTF-8 character (fatal)" error.

This happens whenever a UTF-8 string containing invalid characters is subjected to certain regexes. I'm pretty sure this is a bug in Perl, because it should not have to validate UTF-8 in order to run a regex against it.

This is a security issue because anybody can add comments to any bug that they can see, and so you could theoretically DoS Bugzilla with this.

Comment 2

[Reed Loden [:reed]]

We're running into this on production bmo now. :/

Comment 3

[Daniel Veditz [:dveditz]]

For example, bug 532384 is now unviewable.

Comment 4

[Mike Kaply [:mkaply]]

justdave manually fixed 532384 in the database so it could be viewable again.

Comment 5

[Frédéric Buclin]

Max, any idea how to fix this bug?

Comment 6

[Max Kanat-Alexander]

Well, the problem *is* basically a bug in Perl. There's absolutely no reason we should be hitting it.

I looked into fixing the issue when I first filed it. 


The simplest fix, for most fields, is to modify the js/html filters to remove invalid UTF-8 characters from strings before displaying them. That's somewhat problematic because it happens when you submit the text to certain regexes, so I have no idea if the "html" filter itself wouldn't cause the issue. 

For comments, I'm pretty sure that this happens before the filtering stage (I think it happens when we wrap the comment). So we'd have to strip the invalid characters before they were put into the database. I have some code around somewhere that does this, but it's extremely tricky because you can't run certain regexes against the string.

As I recall, the reason that I didn't post a patch is that it was impractical or impossible for us to fix it in any reliable fashion, and that the bug needs to be fixed in Perl itself.

Comment 7

[Frédéric Buclin]

denying blocking per mkanat's last comment.

Comment 8

[Frédéric Buclin]

FYI, I can reproduce the crash with Perl 5.10.1, but not with 5.12.1.

Comment 10

[Dave Miller [:justdave]]

If we can figure out a version of Perl that fixed this and what patch fixed it upstream, maybe I can talk Red Hat into backporting it for RHEL 5, since it essentially is a DoS for web applications... (or maybe I can just backport it myself and deploy a new build here)

Comment 11

[Dave Miller [:justdave]]

RHEL5 has Perl 5.8.8, RHEL6 has Perl 5.10.1.  Comment 8 says it's still reproducible in 5.10.1, so we're gonna be stuck with this a while if we don't find a way to backport.

Comment 12

[David Lawrence [:dkl]]

glob and I talked about maintaining a BMO yum repo for custom rpms. If Mozilla has an internal RHN Satellite server, a custom channel could do the same. But a yum repo would be easier. Then a newer version of perl and associated perl dependent packages could be stored. But it might be too much time involved keeping it updated and maintained.

dkl

Comment 13

[Frédéric Buclin]

FYI, a user had this problem with Perl 5.10.0. I asked him to upgrade to Perl 5.12, and now this problem is gone. So this is consistent with what I said in comment 8. :)

Comment 14

[Frédéric Buclin]

FYI, RHEL 7 has Perl 5.16, and it should be released this year. Maybe we could bump our min Perl requirement to 5.12, after all. :)

Comment 15

[Frédéric Buclin]

Several Unicode problems have been fixed in Perl 5.14, and Bugzilla is full of hacks due to older releases.

[root@localhost Bugzilla]# trouve 5.13
163:    # until we require Perl 5.13.9 or newer.
1311543   48 -rw-r-----   1 root     apache      49135 fév 23 15:02 ./Template.pm
1911:    # Perl 5.13.8 and older complain about non-characters.
1312366  120 -rw-r-----   1 root     apache     120159 fév 23 15:01 ./Search.pm
447:        # Perl 5.13.8 and older complain about non-characters.
1311541   20 -rw-r-----   1 root     apache      17712 fév 23 15:02 ./Comment.pm
122:        version => ($^V >= v5.13.3) ? '1.614' : '1.54'
315:        version => ($^V >= v5.13.3) ? '3.67' : '3.40',
1311330   32 -rw-r-----   1 root     apache      29931 fév 14 01:59 ./Install/Requirements.pm
3637:                # Perl 5.13.8 and older complain about non-characters.
1311540  168 -rw-r-----   1 root     apache     170870 fév 23 15:02 ./Bug.pm

[root@localhost extensions]# trouve 5.14
485:    # but it's only understood since Perl 5.14. So the workaround is to write
1319056   36 -rw-r-----   1 root     apache      33976 fév 24 10:31 ./Example/Extension.pm


Would you object if we require Perl 5.14 (instead of 5.12) for the next major release? It has been released in May 2011, and all recent Linux distros have it, including RHEL7.

Comment 16

[David Lawrence [:dkl]]

(In reply to Frédéric Buclin from comment #15)
> Would you object if we require Perl 5.14 (instead of 5.12) for the next
> major release? It has been released in May 2011, and all recent Linux
> distros have it, including RHEL7.

I personally do not have a problem with this new requirement for 6.0.

dkl

Comment 17

[:glob ✱]

(In reply to David Lawrence [:dkl] from comment #16)
> I personally do not have a problem with this new requirement for 6.0.

i concur.

Comment 18

[Olav Vitters]

(In reply to Frédéric Buclin from comment #15)
> Would you object if we require Perl 5.14 (instead of 5.12) for the next
> major release? It has been released in May 2011, and all recent Linux
> distros have it, including RHEL7.

GNOME bugzilla has several unviewable bugs due to the utf-8 problem. Upgrading to RHEL7 to solve this is a non-issue.

Comment 19

[Frédéric Buclin]

Bugzilla 6.0 now requires Perl 5.14 (bug 1136137). So this bug is no longer an issue there.

Comment 20

[Dave Miller [:justdave]]

Removing the security flag on this bug as the problem is already public knowledge due to other bugs such as bug 1136137. We will need to backport bug 1136137 to the 5.2 branch, but I'll file a new bug for that.