Closed Bug 771100 Opened 12 years ago Closed 12 years ago

Unable to attach a file to a bug with perl 5.16.0

Categories

(Bugzilla :: Attachments & Requests, defect)

4.2.1
defect
Not set
major

Tracking

()

RESOLVED FIXED
Bugzilla 4.0

People

(Reporter: phantom4, Assigned: LpSolit)

References

()

Details

Attachments

(2 files, 2 obsolete files)

Hello,

after updating perl to version 5.16.0 I'm unable to attach files to a bug ("You did not specify a file to attach").
With perl 5.12.4 it's working fine.
Results from checksetup.pl:

* This is Bugzilla 4.2.1 on perl 5.16.0
* Running on Linux 3.4.4-gentoo #1 SMP Sun Jun 24 09:06:32 CEST 2012

Checking perl modules...
Checking for               CGI.pm (v3.51)     ok: found v3.59 
Checking for           Digest-SHA (any)       ok: found v5.71 
Checking for             TimeDate (v2.21)     ok: found v2.24 
Checking for             DateTime (v0.28)     ok: found v0.76 
Checking for    DateTime-TimeZone (v0.71)     ok: found v1.46 
Checking for                  DBI (v1.614)    ok: found v1.622 
Checking for     Template-Toolkit (v2.22)     ok: found v2.24 
Checking for           Email-Send (v2.00)     ok: found v2.198 
Checking for           Email-MIME (v1.904)    ok: found v1.910 
Checking for                  URI (v1.37)     ok: found v1.60 
Checking for       List-MoreUtils (v0.22)     ok: found v0.33 
Checking for    Math-Random-ISAAC (v1.0.1)    ok: found v1.004 

Checking available perl DBD modules...
Checking for               DBD-Pg (v1.45)     ok: found v2.19.2 
Checking for            DBD-mysql (v4.001)    not found 
Checking for           DBD-SQLite (v1.29)     not found 
Checking for           DBD-Oracle (v1.19)     not found 

The following Perl modules are optional:
Checking for                   GD (v1.20)     ok: found v2.46 
defined(@array) is deprecated at /usr/lib/perl5/vendor_perl/5.16.0/Chart/Base.pm line 181.
        (Maybe you should just omit the defined()?)
defined(@array) is deprecated at /usr/lib/perl5/vendor_perl/5.16.0/Chart/Base.pm line 233.
        (Maybe you should just omit the defined()?)
Checking for                Chart (v2.1)      ok: found v2.4.5 
Checking for          Template-GD (any)       ok: found v1.56 
Checking for           GDTextUtil (any)       ok: found v0.86 
Checking for              GDGraph (any)       ok: found v1.44 
Checking for           MIME-tools (v5.406)    ok: found v5.503 
Checking for          libwww-perl (any)       ok: found v6.04 
$[ used in numeric lt (<) (did you mean $] ?) at /usr/lib/perl5/vendor_perl/5.16.0/XML/Twig.pm line 7286.
$[ used in numeric lt (<) (did you mean $] ?) at /usr/lib/perl5/vendor_perl/5.16.0/XML/Twig.pm line 7292.
$[ used in numeric lt (<) (did you mean $] ?) at /usr/lib/perl5/vendor_perl/5.16.0/XML/Twig.pm line 7304.
Checking for             XML-Twig (any)       ok: found v3.39 
Checking for          PatchReader (v0.9.6)    ok: found v0.9.6 
Checking for            perl-ldap (any)       ok: found v0.44 
Checking for          Authen-SASL (any)       ok: found v2.15 
Checking for           RadiusPerl (any)       not found 
Checking for            SOAP-Lite (v0.712)    ok: found v0.714 
Checking for             JSON-RPC (any)       ok: found v1.03 
Checking for              JSON-XS (v2.0)      ok: found v2.32 
Checking for           Test-Taint (any)       ok: found v1.04 
Checking for          HTML-Parser (v3.67)     ok: found v3.69 
Checking for        HTML-Scrubber (any)       ok: found v0.09 
Checking for               Encode (v2.21)     ok: found v2.44 
Checking for        Encode-Detect (any)       ok: found v1.01 
Checking for Email-MIME-Attachment-Stripper (any)       ok: found v1.316 
Checking for          Email-Reply (any)       ok: found v1.202 
Checking for          TheSchwartz (any)       ok: found v1.10 
Checking for       Daemon-Generic (any)       ok: found v0.82 
Checking for             mod_perl (v1.999022) not found 
Checking for     Apache-SizeLimit (v0.96)     not found 
Checking for          mod_headers (any)       ok 
Checking for          mod_expires (any)       ok 
Checking for              mod_env (any)       ok 
***********************************************************************
* OPTIONAL MODULES                                                    *
***********************************************************************
* Certain Perl modules are not required by Bugzilla, but by           *
* installing the latest version you gain access to additional         *
* features.                                                           *
*                                                                     *
* The optional modules you do not have installed are listed below,    *
* with the name of the feature they enable. Below that table are the  *
* commands to install each module.                                    *
***********************************************************************
*      MODULE NAME * ENABLES FEATURE(S)                               *
***********************************************************************
*       RadiusPerl * RADIUS Authentication                            *
*         mod_perl * mod_perl                                         *
* Apache-SizeLimit * mod_perl                                         *
***********************************************************************
COMMANDS TO INSTALL OPTIONAL MODULES:

     RadiusPerl: /usr/bin/perl5.16.0 install-module.pl Authen::Radius
       mod_perl: /usr/bin/perl5.16.0 install-module.pl mod_perl2
Apache-SizeLimit: /usr/bin/perl5.16.0 install-module.pl Apache2::SizeLimit


To attempt an automatic install of every required and optional module
with one command, do:

  /usr/bin/perl5.16.0 install-module.pl --all

Reading ./localconfig...

OPTIONAL NOTE: If you want to be able to use the 'difference between two
patches' feature of Bugzilla (which requires the PatchReader Perl module
as well), you should install patchutils from:

    http://cyberelk.net/tim/patchutils/

Checking for               DBD-Pg (v1.45)     ok: found v2.19.2 
Checking for           PostgreSQL (v8.03.0000) ok: found v09.01.0400 
Checking for               DBD-Pg (v2.17.2)   ok: found v2.19.2 

Removing existing compiled templates...
Precompiling templates...done.
Fixing file permissions...
Deriving regex group memberships...
checksetup.pl complete.
Hardware: x86_64 → x86
Version: unspecified → 4.2.1
Keywords: qawanted
I'm receiving the error.  Here is my system:

Gentoo 3.4.4-gentoo
Perl v5.16.1
Apache v2.2.22-r1
mod_perl v2.0.7

modules:
dev-perl/Any-Moose-0.180.0:0
dev-perl/AnyEvent-7.10.0:0
dev-perl/Apache-DBI-1.110.0:0
dev-perl/Apache-Reload-0.120.0:0
dev-perl/Apache-SizeLimit-0.960.0:0
dev-perl/Apache-Test-1.370.0:0
dev-perl/AppConfig-1.660.0:0
dev-perl/Authen-SASL-2.150.0:0
dev-perl/B-Hooks-EndOfScope-0.110.0:0
dev-perl/Capture-Tiny-0.180.0:0
dev-perl/Chart-2.4.5:0
dev-perl/Class-Accessor-0.340.0:0
dev-perl/Class-Accessor-Lite-0.50.0:0
dev-perl/Class-C3-0.230.0:0
dev-perl/Class-Data-Inheritable-0.80.0:0
dev-perl/Class-Inspector-1.270.0:0
dev-perl/Class-Load-0.200.0:0
dev-perl/Class-Load-XS-0.40.0:0
dev-perl/Class-Singleton-1.400.0:0
dev-perl/Class-Trigger-0.140.0:0
dev-perl/Convert-ASN1-0.260.0:0
dev-perl/Crypt-Random-Source-0.70.0:0
dev-perl/Crypt-SSLeay-0.580.0:0
dev-perl/DBD-Pg-2.19.2:0
dev-perl/DBD-mysql-4.20.0:0
dev-perl/DBI-1.622.0:0
dev-perl/Daemon-Generic-0.820.0:0
dev-perl/Data-ObjectDriver-0.90.0:0
dev-perl/Data-OptList-0.107.0:0
dev-perl/DateTime-0.760.0:0
dev-perl/DateTime-Format-DateParse-0.50.0:0
dev-perl/DateTime-Format-Mail-0.300.100:0
dev-perl/DateTime-Format-W3CDTF-0.60.0:0
dev-perl/DateTime-Locale-0.450.0:0
dev-perl/DateTime-TimeZone-1.480.0:0
dev-perl/Devel-GlobalDestruction-0.90.0:0
dev-perl/Devel-StackTrace-1.270.0:0
dev-perl/Devel-StackTrace-AsHTML-0.110.0:0
dev-perl/Digest-HMAC-1.30.0:0
dev-perl/Digest-SHA1-2.130.0:0
dev-perl/Dist-CheckConflicts-0.20.0:0
dev-perl/Email-Abstract-3.3.0:0
dev-perl/Email-Address-1.896.0:0
dev-perl/Email-Date-Format-1.2.0:0
dev-perl/Email-MIME-1.911.0:0
dev-perl/Email-MIME-Attachment-Stripper-1.316.0:0
dev-perl/Email-MIME-ContentType-1.15.0:0
dev-perl/Email-MIME-Encodings-1.313.0:0
dev-perl/Email-MessageID-1.402.0:0
dev-perl/Email-Reply-1.202.0:0
dev-perl/Email-Send-2.198.0:0
dev-perl/Email-Simple-2.102.0:0
dev-perl/Encode-Detect-1.10.0:0
dev-perl/Encode-Locale-1.30.0:0
dev-perl/Error-0.170.180:0
dev-perl/Eval-Closure-0.80.0:0
dev-perl/File-Flock-2008.10.0:0
dev-perl/File-HomeDir-0.990.0:0
dev-perl/File-Listing-6.40.0:0
dev-perl/File-ShareDir-1.30.0:0
dev-perl/File-Slurp-9999.190.0:0
dev-perl/File-Which-1.90.0:0
dev-perl/Filesys-Notify-Simple-0.80.0:0
dev-perl/GD-2.460.0:0
dev-perl/GD-Graph3d-0.630.0:0
dev-perl/GDGraph-1.440.0:0
dev-perl/GDTextUtil-0.860.0:0
dev-perl/HTML-Parser-3.690.0:0
dev-perl/HTML-Scrubber-0.90.0:0
dev-perl/HTML-TableParser-0.380.0:0
dev-perl/HTML-Tagset-3.200.0:0
dev-perl/HTTP-Body-1.150.0:0
dev-perl/HTTP-Cookies-6.0.1:0
dev-perl/HTTP-Daemon-6.10.0:0
dev-perl/HTTP-Date-6.20.0:0
dev-perl/HTTP-Message-6.30.0:0
dev-perl/HTTP-Negotiate-6.0.1:0
dev-perl/Hash-MultiValue-0.120.0:0
dev-perl/IO-Socket-SSL-1.760.0:0
dev-perl/IO-stringy-2.110.0:0
dev-perl/JSON-2.530.0:0
dev-perl/JSON-RPC-1.30.0:0
dev-perl/JSON-XS-2.330.0:0
dev-perl/LWP-MediaTypes-6.20.0:0
dev-perl/LWP-Protocol-https-6.30.0:0
dev-perl/Linux-Pid-0.40.0:0
dev-perl/List-MoreUtils-0.330.0:0
dev-perl/Locale-gettext-1.50.0:0
dev-perl/MIME-Lite-3.28.0:0
dev-perl/MIME-Types-1.350.0:0
dev-perl/MIME-tools-5.503.0:0
dev-perl/MRO-Compat-0.110.0:0
dev-perl/MailTools-2.90.0:0
dev-perl/Math-Random-ISAAC-1.4.0:0
dev-perl/Math-Random-ISAAC-XS-1.4.0:0
dev-perl/Math-Random-Secure-0.60.0:0
dev-perl/Math-Round-0.60.0:0
dev-perl/Module-Find-0.110.0:0
dev-perl/Module-Implementation-0.60.0:0
dev-perl/Module-Runtime-0.13.0:0
dev-perl/Moose-2.60.300:0
dev-perl/Net-Daemon-0.480.0:0
dev-perl/Net-HTTP-6.30.0:0
dev-perl/Net-SMTP-SSL-1.10.0:0
dev-perl/Net-SSLeay-1.480.0-r1:0
dev-perl/Package-DeprecationManager-0.130.0:0
dev-perl/Package-Stash-0.330.0:0
dev-perl/Package-Stash-XS-0.250.0:0
dev-perl/Params-Util-1.60.0:0
dev-perl/Params-Validate-1.60.0:0
dev-perl/PatchReader-0.9.6:0
dev-perl/PlRPC-0.202.0:0
dev-perl/Plack-1.0.100:0
dev-perl/Return-Value-1.666.1:0
dev-perl/Router-Simple-0.90.0:0
dev-perl/SOAP-Lite-0.715.0:0
dev-perl/Sub-Exporter-0.984.0:0
dev-perl/Sub-Exporter-Progressive-0.1.4:0
dev-perl/Sub-Identify-0.40.0:0
dev-perl/Sub-Install-0.926.0:0
dev-perl/Sub-Name-0.50.0:0
dev-perl/Template-DBI-2.650.0:0
dev-perl/Template-GD-2.660.0:0
dev-perl/Template-Latex-2.170.0:0
dev-perl/Template-Toolkit-2.240.0:0
dev-perl/Template-XML-2.170.0:0
dev-perl/TermReadKey-2.300.0:0
dev-perl/Test-SharedFork-0.200.0:0
dev-perl/Test-TCP-1.160.0:0
dev-perl/Test-Taint-1.40.0:0
dev-perl/Text-Iconv-1.700.0:0
dev-perl/TheSchwartz-1.100.0:0
dev-perl/Tie-IxHash-1.220.0:0
dev-perl/TimeDate-1.200.0:0
dev-perl/Try-Tiny-0.110.0:0
dev-perl/URI-1.600.0:0
dev-perl/Variable-Magic-0.500.0:0
dev-perl/WWW-RobotRules-6.10.0:0
dev-perl/XML-DOM-1.440.0:0
dev-perl/XML-Filter-BufferText-1.10.0:0
dev-perl/XML-Handler-YAWriter-0.230.0:0
dev-perl/XML-LibXML-2.0.300:0
dev-perl/XML-NamespaceSupport-1.110.0:0
dev-perl/XML-Parser-2.410.0-r1:0
dev-perl/XML-RSS-1.490.0:0
dev-perl/XML-RegExp-0.40.0:0
dev-perl/XML-SAX-0.990.0:0
dev-perl/XML-SAX-Base-1.80.0:0
dev-perl/XML-SAX-Writer-0.530.0:0
dev-perl/XML-Simple-2.200.0:0
dev-perl/XML-Twig-3.390.0:0
dev-perl/XML-XPath-1.130.0:0
dev-perl/common-sense-3.600.0:0
dev-perl/libwww-perl-6.40.0:0
dev-perl/libxml-perl-0.80.0:0
dev-perl/namespace-clean-0.230.0:0
dev-perl/perl-ldap-0.440.0:0
dev-perl/text-autoformat-1.669.2:0
dev-perl/text-reform-1.200.0:0
Confirmed. I get the same error message with Mageia 3 and Perl 5.16.0.
Severity: normal → major
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: blocking4.4+
Flags: blocking4.2.3+
Keywords: qawanted
OS: Linux → All
Hardware: x86 → All
Summary: Unable to attach a file to a bug with perl 5.16.0 (Gentoo) → Unable to attach a file to a bug with perl 5.16.0
Target Milestone: --- → Bugzilla 4.2
Using CGI 3.59 on Perl 5.14.2 works fine.
Using CGI 3.59 on Perl 5.16.0 fails.

$cgi->param('data') correctly shows the name of the uploaded file, but $cgi->upload('data') is undefined, making Bugzilla to think that the file is missing.

Mark: are you aware of this bug?
It looks like the /usr/tmp/CGItempxxx file is not created at all. If I run this code:

    my ($k) = keys %{$cgi->{'.tmpfiles'}};
    my $n = $cgi->{'.tmpfiles'}->{$k}->{name};
    my $f = Dumper($n);
    $f =~ m{/usr/tmp/CGItemp(\d+)};
    my $i = $1;
    open(FH, "<", "/usr/tmp/CGItemp$i") or die $!;
    my $s = -s FH;
    die $s;

open() is unable to find the file. So nothing is uploaded.
Have the same issue on my FreeBSD 9.0-RELEASE-p3 / Apache 2.2.22 / mod_perl 2.0.7 / Perl v5.16.0 / Bugzilla 4.2.2 right after upgrading to perl 5.16.

It seems that the real issue is about CGI.pm specifics to perl 5.16.

I have no fix or workaround so far.
I'm not so sure it's a bug in CGI.pm itself, or at least it's more subtle than that. If I use CGI directly instead of Bugzilla::CGI, then I can upload files without any problem. So something is maybe wrong on our side.
Attached file PoC (obsolete) —
Here is the script I used for my testing. The single line you need to edit to test with Bugzilla::CGI or CGI alone is line 12:

my $cgi = $bz_cgi;  # Use Bugzilla::CGI
my $cgi = $cpan_cgi # Use CGI only

With Perl 5.16, uploading files work fine with $cpan_cgi, but fails with $bz_cgi.
Attached file PoC
This PoC removes uninitialized warnings.
Attachment #651907 - Attachment is obsolete: true
OK, I found what's wrong. The problem comes from Bugzilla::CGI::_fix_utf8(). For some reason which I don't understand yet, this line is causing trouble in Perl 5.16 only:

    # The is_utf8 is here in case CGI gets smart about utf8 someday.
    utf8::decode($input) if defined $input && !utf8::is_utf8($input);

If I remove this line, then everything works fine.
The problem comes from utf8::decode($input). I printed the content of Dumper($input) right before and right after calling utf8::decode($input), and the result is different in Perl 5.14 and 5.16.

5.14.2:

before: $VAR1 = bless( \*{'Fh::fh00001bug0-6.diff'}, 'Fh' );
after:  $VAR1 = bless( \*{'Fh::fh00001bug0-6.diff'}, 'Fh' );

5.16.0:

before: $VAR1 = bless( \*{'Fh::fh00001bug0-6.diff'}, 'Fh' );
after:  $VAR1 = 'bug0-6.diff'

The results above are the same for $input = $cgi->param('file') and
$input = $cgi->upload('file').

I don't know if this change in Perl 5.16 is intentional or not, but it looks like we can fix the problem ourselves. Patch coming!
Assignee: attach-and-request → LpSolit
Status: NEW → ASSIGNED
(In reply to Frédéric Buclin from comment #10)
> after:  $VAR1 = '<missing bug number>-6.diff'

Arhh, bmo mangled what I wrote, because it thought I was talking about bug zero. Let's say it should be:

 after:  $VAR1 = 'bugO-6.diff'
Per https://rt.perl.org/rt3/Public/Bug/Display.html?id=91852, this behavior change is intentional and has been fixed in Perl 5.16. It was considered a bug.

I'm removing Mark from the CC list as his module is not the culprit and I don't want to spam him any longer. :)
Attached patch patch, v1Splinter Review
If it's a filehandle, then we must not decode it anymore as it will be automatically stringified now.
Attachment #651946 - Flags: review?(dkl)
Great thanks to Frédéric Buclin , attachment 651946 [details] [diff] [review] works for me as well.
I confirm that the patch worked for me, too.
Comment on attachment 651946 [details] [diff] [review]
patch, v1

Review of attachment 651946 [details] [diff] [review]:
-----------------------------------------------------------------

r=dkl
Attachment #651946 - Flags: review?(dkl) → review+
Flags: approval4.2+
Flags: approval+
Committing to: bzr+ssh://lpsolit%40gmail.com@bzr.mozilla.org/bugzilla/trunk/
modified Bugzilla/CGI.pm
Committed revision 8346.

Committing to: bzr+ssh://lpsolit%40gmail.com@bzr.mozilla.org/bugzilla/4.2/
modified Bugzilla/CGI.pm
Committed revision 8119.
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
I was happy with that solution, but not so long. In fact the files attached with the most recent patch are broken. For the instance, attaching any zip archive via web form you can download it back, but it can't be unzipped anymore.
(In reply to Alexei Volkov from comment #18)
> with the most recent patch are broken. For the instance, attaching any zip
> archive via web form you can download it back, but it can't be unzipped
> anymore.

I cannot reproduce your issue. Binary files are correctly uploaded/downloaded, and I can open them as expected. My patch has nothing to do with the content of attachments anyway, it only prevented the filehandle from being converted into a string.
this patch also addresses issues with searching multi-value fields on 4.0.

given this patch is trivial, breaks core bugzilla functionality, and is being encountered more frequently as distros upgrade perl to 5.16, requesting approval for a 4.0 commit (to ride along with the next security release).
Flags: approval4.0?
(In reply to Byron Jones ‹:glob› from comment #20)
> being encountered more frequently as distros upgrade perl to 5.16,

I would expect that if distros provide Perl 5.16, they also provide a pretty recent version of Bugzilla, i.e. 4.2. But it doesn't hurt to take it for 4.0 too.
Flags: approval4.0? → approval4.0+
Target Milestone: Bugzilla 4.2 → Bugzilla 4.0
Committing to: bzr+ssh://lpsolit%40gmail.com@bzr.mozilla.org/bugzilla/4.0/
modified Bugzilla/CGI.pm
Committed revision 7741.
Attached file AddNewBook.form (obsolete) —
Flags: needinfo?(cheretshediso)
Flags: needinfo?(cheretshediso)
Attachment #9229473 - Attachment is obsolete: true
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: