Closed Bug 486306 Opened 15 years ago Closed 15 years ago

Truncated XML-RPC response (incorrect content-length header)

Categories

(Bugzilla :: WebService, defect)

3.2.3
defect
Not set
major

Tracking

()

RESOLVED FIXED
Bugzilla 3.4

People

(Reporter: nuabaranda, Assigned: mkanat)

References

Details

Attachments

(2 files, 3 obsolete files)

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)
Build Identifier: Testopia 2.2 on Bugzilla 3.2.2

Longer XML-RPC responses of the Testopia XML-RPC interface are truncated by a few bytes leading to invalid XML. Seems to be an issue of the used SOAP:Lite Perl module, see this bug: https://sourceforge.net/tracker/index.php?func=detail&aid=2723971&group_id=66000&atid=513017



Reproducible: Always

Steps to Reproduce:
1. Request longer information items via XML-RPC.
Actual Results:  
Truncated thus invalid XML code in HTTP response.

Expected Results:  
Valid and complete XML response.

Bugzilla checksetup.pl output: 

* This is Bugzilla 3.2.2 on perl 5.8.8

Checking perl modules...
Checking for              CGI.pm (v3.21)   ok: found v3.42
Checking for            TimeDate (v2.21)   ok: found v2.22
Checking for           PathTools (v0.84)   ok: found v3.29
Checking for                 DBI (v1.41)   ok: found v1.53
Checking for    Template-Toolkit (v2.15)   ok: found v2.20
Checking for          Email-Send (v2.00)   ok: found v2.181
Checking for          Email-MIME (v1.861)  ok: found v1.863
Checking for Email-MIME-Modifier (v1.442)  ok: found v1.443
Checking for                JSON (v2.10)   ok: found v2.12
Checking for           Text-Diff (v0.35)   ok: found v0.35
Checking for          GD-Graph3d (v0.63)   ok: found v0.63

Checking available perl DBD modules...
Checking for              DBD-Pg (v1.45)    not found
Checking for           DBD-mysql (v4.00)   ok: found v4.010
Checking for          DBD-Oracle (v1.19)    not found

The following Perl modules are optional:
Checking for                  GD (v1.20)   ok: found v2.34
Checking for               Chart (v1.0)    ok: found v2.4.1
Checking for         Template-GD (any)     ok: found v1.56
Checking for          GDTextUtil (any)     ok: found v0.86
Checking for             GDGraph (any)     ok: found v1.4308
Checking for            XML-Twig (any)     ok: found v3.26
Checking for          MIME-tools (v5.406)  ok: found v5.420
Checking for         libwww-perl (any)     ok: found v2.033
Checking for         PatchReader (v0.9.4)  ok: found v0.9.5
Checking for          PerlMagick (any)     ok: found v6.2.4
Checking for           perl-ldap (any)     ok: found v0.33
Checking for         Authen-SASL (any)     ok: found v2.12
Checking for          RadiusPerl (any)     ok: found v0.13
Checking for           SOAP-Lite (any)     ok: found v0.710.08
Checking for         HTML-Parser (v3.40)   ok: found v3.55
Checking for       HTML-Scrubber (any)     ok: found v0.08
Checking for Email-MIME-Attachment-Stripper (any)     ok: found v1.314
Checking for         Email-Reply (any)     ok: found v1.202
Checking for            mod_perl (v1.999022) ok: found v2.000002
Checking for            Text-CSV (v1.06)   ok: found v1.10

See attached dump file.
Version: unspecified → 2.2
please see http://sourceforge.net/tracker/?func=detail&atid=513017&aid=2723971&group_id=66000 for a possible workaround involving converting UTF8-encoded non-breaking spaces to the text string  
Component: API → WebService
Product: Testopia → Bugzilla
Version: 2.2 → 3.2.2
Why is this bug reported in the Bugzilla product?
I can also confirm it for the Bugzilla XML-RPC interface.
Tested both the Bugzilla and the Testopia XML-RPC interface. For both the following observations are true:

If the response contains at least one 3-byte UTF character the response is correctly encoded, valid and not truncated. If it only contains ASCII and 2-byte UTF characters it is handled as ASCII, thus double-encoded and finally the HTTP response is truncated resulting in invalid XML.
I am fairly sure that the bug on SF is a Testopia problem, given what it says there.
Component: WebService → API
Product: Bugzilla → Testopia
Version: 3.2.2 → 2.2
(In reply to comment #6)
> I am fairly sure that the bug on SF is a Testopia problem, given what it says
> there.

The issue on SF just seems to be a part of the main problem which I described in comment #5. And as I said there I can confirm this issue for both Bugzilla and Testopia XML-RPC interfaces.
Since the Testopia XMLRPC interface is a copy of the Bugzilla interface, it would make sense that this issue appears in both. The real question is whether this is an issue with our implementation, or with SOAP::Lite itself.
Finally found the cause: There is a package Bugzilla::WebService::XMLRPC::Serializer in WebService.pm which says it was used to patch some SOAP::Lite bugs. Its method as_string() leads to the double encoding. Seems the bug was fixed in SOAP::Lite in the meantime. See the patch attached.

Maybe there are more such fixes around. For example I haven't tested anything base64-related up to now...
Component: API → WebService
OS: Linux → All
Product: Testopia → Bugzilla
Version: 2.2 → 3.2.3
Attached patch Patch for UTF-8 encoding bug. (obsolete) — Splinter Review
Commented out method causing the bug.
Attachment #376266 - Flags: review-
Comment on attachment 376266 [details] [diff] [review]
Patch for UTF-8 encoding bug.

Can you show me where in the Change history of SOAP::Lite this was fixed? If it's fixed, we need to remove this function entirely and require the higher version of SOAP::Lite.
Hardware: x86 → All
(In reply to comment #11)
> Can you show me where in the Change history of SOAP::Lite this was fixed? If
> it's fixed, we need to remove this function entirely and require the higher
> version of SOAP::Lite.

As there was no SOAP::Lite bug referenced in the comments to this method (as for a couple of other methods) I could not really retrace it in SOAP::Lite. But I wrote a request to the SOAP::Lite maintainer to have a look here and give us some hint.
(In reply to comment #11)
> Can you show me where in the Change history of SOAP::Lite this was fixed? If
> it's fixed, we need to remove this function entirely and require the higher
> version of SOAP::Lite.

Could be this one here: http://rt.cpan.org/Public/Bug/Display.html?id=35041
(In reply to comment #11)
> Can you show me where in the Change history of SOAP::Lite this was fixed?

Here is Martin Kutters answer, he is the SOAP::Lite maintainer: 

this bug is fixed since 0.71.04 released on CPAN on 21-Apr-2008 (since
rev238 in SOAP::Lite's subversion - see
http://soaplite.svn.sourceforge.net/viewvc/soaplite?view=rev&revision=238 for details).

The description of the Bugzilla bug entry sounds like the double
encoding is due to the this fix: Since 0.71.04 SOAP::Lite encodes
UTF8-Characters (no matter how many bytes long) correctly by calling

encode($encoding, $response)

before sending the content out on the wire.

encode($encoding, $characters) returns a sequence of octets (bytes) in
the encoding specified, so SOAP::Lite now sends out a byte-stream in the
encoding specified.

SOAP::Lite actually determines the content length before performing the
encoding (by asking for the bytelength of the string) which may lead to
invalid results when double encoded characters are contained in the
response. This could be construed as a bug, too.
Flags: blocking3.4+
Target Milestone: --- → Bugzilla 3.2
Assignee: ghendricks → webservice
QA Contact: default-qa → default-qa
Actually, I cannot reproduce. I put several two-byte characters into a bug. In particular, this string as the summary:

ƀƁƂƃƄƅƆƇƈƉƊƋƌƍƎƏƐƑƒƓ

And Bug.get returns the correct XML. SOAP::Lite's XMLRPCsh.pl script correctly decodes the result. I see no truncation whatsoever, and the returned string is correct (XMLRPCsh.pl displays it in Perl Unicode notation):

\x{180}\x{181}\x{182}\x{183}\x{184}\x{185}\x{186}\x{187}\x{188}\x{189}\x{18a}\x{18b}\x{18c}\x{18d}\x{18e}\x{18f}\x{190}\x{191}\x{192}\x{193}

There could not be any double-encoding by us, because the Serializer code that we wrote checks utf8::is_utf8 first before encoding, so if it was already encoded, it would not encode it again.

I tested this all with SOAP::Lite 0.71.07.
Status: UNCONFIRMED → RESOLVED
Closed: 15 years ago
Flags: blocking3.4+ → blocking3.4-
Resolution: --- → WORKSFORME
Are you sure that there is no other 3-byte-character in the response? If this is the case, the encoding is correct. Could you try a single german umlaut as ä or ü or ö?
Did you have a look at my TCP dump which clearly shows the truncation? Could you attach a dump of your test?
(In reply to comment #15)
> There could not be any double-encoding by us, because the Serializer code that
> we wrote checks utf8::is_utf8 first before encoding, so if it was already
> encoded, it would not encode it again.

It's the other way around: you get it from the Bugzilla database and the fix I commented out in my patch encodes it the first time. Then the response is passed to SOAP::Lite and there the second encoding happens. In previous SOAP::Lite releases this (second) encoding was missing but was added in between time. And this leads to the bug. 

In general, as I understand it, Bugzilla should not care about encoding issues. The underlying communication libraries such as SOAP::Lite should do it (or pass it on). So when you add bugfixes for problems in underlying libraries you should carefully regression-test for them being fixed there.
Target Milestone: Bugzilla 3.2 → ---
Summary: Truncated XML-RPC response → Truncated XML-RPC response (incorrect content-length header)
Attached patch Patch, which fixes problem (obsolete) — Splinter Review
Please reopen this bug. I can clearly reproduce it and, what's more, I am able to resolve it with the attached patch. The patch seems basically to do the same than the previous proposed patch, just that the location has changed. Indeed, removing the as_string method in XMLRPC.pm seems to fix my problem.
(In reply to comment #20)
> Created an attachment (id=384817) [details]
> Patch, which fixes problem
> 
> Please reopen this bug. I can clearly reproduce it and, what's more, I am able
> to resolve it with the attached patch. The patch seems basically to do the same
> than the previous proposed patch, just that the location has changed. Indeed,
> removing the as_string method in XMLRPC.pm seems to fix my problem.

Forgot to note: My patch is against Bugzilla 3.3.4.
Max, regarding your comment in bug 499898:

I am ready to pass the contents of my current database (just about 261K) to you privately. (Customer data, I cannot tell how sensitive it is) I can also pass the complete client code, which is nothing but the test suite of a proposed Perl client for the Bugzilla webservices API.
Hey Jochen. Okay, could you send everything to my email address?
Status: RESOLVED → UNCONFIRMED
Resolution: WORKSFORME → ---
Okay, I can reproduce this now, and it's definitely a problem. Anything containing non-ASCII characters doesn't work.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: blocking3.4.2+
Target Milestone: --- → Bugzilla 3.4
Attached patch v1Splinter Review
Okay, here we go. It should be OK to require 0.710.06--it was released in 2008.
Assignee: webservice → mkanat
Attachment #370398 - Attachment is obsolete: true
Attachment #376266 - Attachment is obsolete: true
Attachment #384817 - Attachment is obsolete: true
Status: NEW → ASSIGNED
Attachment #397988 - Flags: review?(LpSolit)
Attached patch v1 - 3.4Splinter Review
And here's the same patch for the branch.
Attachment #397989 - Flags: review?(LpSolit)
Attachment #397988 - Flags: review?(LpSolit) → review+
Comment on attachment 397988 [details] [diff] [review]
v1

>Index: Bugzilla/Install/Requirements.pm

>+        # 0.710.04 is required for correct UTF-8 handling

Nit: It's 0.710.03 or better, as far as I know.

Otherwise, this indeed fixes the problem. r=LpSolit
Attachment #397989 - Flags: review?(LpSolit) → review+
Flags: approval3.4+
Flags: approval+
(In reply to comment #27)
> Nit: It's 0.710.03 or better, as far as I know.

  I got 0.710.04 from comment 14, which has info from the SOAP::Lite maintainer.
tip:

Checking in Bugzilla/Install/Requirements.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Install/Requirements.pm,v  <--  Requirements.pm
new revision: 1.69; previous revision: 1.68
done
Checking in Bugzilla/WebService/Server/XMLRPC.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService/Server/XMLRPC.pm,v  <--  XMLRPC.pm
new revision: 1.6; previous revision: 1.5
done


3.4:

Checking in Bugzilla/Install/Requirements.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Install/Requirements.pm,v  <--  Requirements.pm
new revision: 1.60.2.5; previous revision: 1.60.2.4
done
Checking in Bugzilla/WebService/Server/XMLRPC.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService/Server/XMLRPC.pm,v  <--  XMLRPC.pm
new revision: 1.1.2.3; previous revision: 1.1.2.2
done
Status: ASSIGNED → RESOLVED
Closed: 15 years ago15 years ago
Resolution: --- → FIXED
Just dor a suggestion: It might make sense to always set content-length=-1. That way the client wouldn't expect it to terminate at a given point.
Hi, I think this bug should be reopened. I am having still this bug in version 4.2.1.

Bug #706276 should be marked as duplicated with this one.

As a workaround, I have removed the "Content-Length" header in the XMLRPC.pm file, and server automatically uses Chunked transfer encoding.

The patch for version 4.2.1:
--- bugzilla-4.2.1/Bugzilla/WebService/Server/XMLRPC.pm.orig    2012-04-18 22:32:47.000000000 +0200
+++ bugzilla-4.2.1/Bugzilla/WebService/Server/XMLRPC.pm 2012-06-08 10:57:50.827070643 +0200
@@ -64,6 +64,7 @@
     foreach (@{Bugzilla->cgi->{'Bugzilla_cookie_list'}}) {
         $self->response->headers->push_header('Set-Cookie', $_);
     }
+    $self->response->headers->remove_header("Content-Length");
 }

 sub handle_login {


This patch is also correct for the last XMLRPC.pm file in trunk (version 8259), except for the numbers of lines (the new version has less license lines in the beginning of the file).
After upgrading SOAP::Lite to 1.08 and XMLRPC::Lite 0.717 on Windows I can confirm that this still exists in Bugzilla 4.2.7 and 4.4.1.
It does seem to be fixed in Bugzilla 4.2.6 with SOAP::Lite 1.06 on Ubuntu 12.04.
Hi, I also think this bug should be reopened. 

I still get this bug in Bugzilla version 4.4.2 with SOAP-Lite 1.10, XMLRPC-Lite 0.717 on Windows

The workaround in Comment 36 did the trick for me.
We are not going to reopen this bug. The patch has been checked in a long time ago. File a separate bug if you have exact steps to reproduce the issue.
Wayne, Thomas: We'll definitely need more input. See, for example, the attachment from comment #1. It is unlikely, that we'll be able to reproduce this bug, but input like this should be able to provide a starting point.
You need to log in before you can comment on or make changes to this bug.