Unable to attach a file to a bug with perl 5.16.0

RESOLVED FIXED in Bugzilla 4.0

Status

()

Bugzilla
Attachments & Requests
--
major
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: Per Pomsel, Assigned: Frédéric Buclin)

Tracking

4.2.1
Bugzilla 4.0
Bug Flags:
approval +
blocking4.4 +
approval4.2 +
blocking4.2.3 +
approval4.0 +

Details

(URL)

Attachments

(2 attachments, 1 obsolete attachment)

PoC
917 bytes, text/plain
Details
442 bytes, patch
dkl
: review+
Details | Diff | Splinter Review
(Reporter)

Description

5 years ago
Hello,

after updating perl to version 5.16.0 I'm unable to attach files to a bug ("You did not specify a file to attach").
With perl 5.12.4 it's working fine.
Results from checksetup.pl:

* This is Bugzilla 4.2.1 on perl 5.16.0
* Running on Linux 3.4.4-gentoo #1 SMP Sun Jun 24 09:06:32 CEST 2012

Checking perl modules...
Checking for               CGI.pm (v3.51)     ok: found v3.59 
Checking for           Digest-SHA (any)       ok: found v5.71 
Checking for             TimeDate (v2.21)     ok: found v2.24 
Checking for             DateTime (v0.28)     ok: found v0.76 
Checking for    DateTime-TimeZone (v0.71)     ok: found v1.46 
Checking for                  DBI (v1.614)    ok: found v1.622 
Checking for     Template-Toolkit (v2.22)     ok: found v2.24 
Checking for           Email-Send (v2.00)     ok: found v2.198 
Checking for           Email-MIME (v1.904)    ok: found v1.910 
Checking for                  URI (v1.37)     ok: found v1.60 
Checking for       List-MoreUtils (v0.22)     ok: found v0.33 
Checking for    Math-Random-ISAAC (v1.0.1)    ok: found v1.004 

Checking available perl DBD modules...
Checking for               DBD-Pg (v1.45)     ok: found v2.19.2 
Checking for            DBD-mysql (v4.001)    not found 
Checking for           DBD-SQLite (v1.29)     not found 
Checking for           DBD-Oracle (v1.19)     not found 

The following Perl modules are optional:
Checking for                   GD (v1.20)     ok: found v2.46 
defined(@array) is deprecated at /usr/lib/perl5/vendor_perl/5.16.0/Chart/Base.pm line 181.
        (Maybe you should just omit the defined()?)
defined(@array) is deprecated at /usr/lib/perl5/vendor_perl/5.16.0/Chart/Base.pm line 233.
        (Maybe you should just omit the defined()?)
Checking for                Chart (v2.1)      ok: found v2.4.5 
Checking for          Template-GD (any)       ok: found v1.56 
Checking for           GDTextUtil (any)       ok: found v0.86 
Checking for              GDGraph (any)       ok: found v1.44 
Checking for           MIME-tools (v5.406)    ok: found v5.503 
Checking for          libwww-perl (any)       ok: found v6.04 
$[ used in numeric lt (<) (did you mean $] ?) at /usr/lib/perl5/vendor_perl/5.16.0/XML/Twig.pm line 7286.
$[ used in numeric lt (<) (did you mean $] ?) at /usr/lib/perl5/vendor_perl/5.16.0/XML/Twig.pm line 7292.
$[ used in numeric lt (<) (did you mean $] ?) at /usr/lib/perl5/vendor_perl/5.16.0/XML/Twig.pm line 7304.
Checking for             XML-Twig (any)       ok: found v3.39 
Checking for          PatchReader (v0.9.6)    ok: found v0.9.6 
Checking for            perl-ldap (any)       ok: found v0.44 
Checking for          Authen-SASL (any)       ok: found v2.15 
Checking for           RadiusPerl (any)       not found 
Checking for            SOAP-Lite (v0.712)    ok: found v0.714 
Checking for             JSON-RPC (any)       ok: found v1.03 
Checking for              JSON-XS (v2.0)      ok: found v2.32 
Checking for           Test-Taint (any)       ok: found v1.04 
Checking for          HTML-Parser (v3.67)     ok: found v3.69 
Checking for        HTML-Scrubber (any)       ok: found v0.09 
Checking for               Encode (v2.21)     ok: found v2.44 
Checking for        Encode-Detect (any)       ok: found v1.01 
Checking for Email-MIME-Attachment-Stripper (any)       ok: found v1.316 
Checking for          Email-Reply (any)       ok: found v1.202 
Checking for          TheSchwartz (any)       ok: found v1.10 
Checking for       Daemon-Generic (any)       ok: found v0.82 
Checking for             mod_perl (v1.999022) not found 
Checking for     Apache-SizeLimit (v0.96)     not found 
Checking for          mod_headers (any)       ok 
Checking for          mod_expires (any)       ok 
Checking for              mod_env (any)       ok 
***********************************************************************
* OPTIONAL MODULES                                                    *
***********************************************************************
* Certain Perl modules are not required by Bugzilla, but by           *
* installing the latest version you gain access to additional         *
* features.                                                           *
*                                                                     *
* The optional modules you do not have installed are listed below,    *
* with the name of the feature they enable. Below that table are the  *
* commands to install each module.                                    *
***********************************************************************
*      MODULE NAME * ENABLES FEATURE(S)                               *
***********************************************************************
*       RadiusPerl * RADIUS Authentication                            *
*         mod_perl * mod_perl                                         *
* Apache-SizeLimit * mod_perl                                         *
***********************************************************************
COMMANDS TO INSTALL OPTIONAL MODULES:

     RadiusPerl: /usr/bin/perl5.16.0 install-module.pl Authen::Radius
       mod_perl: /usr/bin/perl5.16.0 install-module.pl mod_perl2
Apache-SizeLimit: /usr/bin/perl5.16.0 install-module.pl Apache2::SizeLimit


To attempt an automatic install of every required and optional module
with one command, do:

  /usr/bin/perl5.16.0 install-module.pl --all

Reading ./localconfig...

OPTIONAL NOTE: If you want to be able to use the 'difference between two
patches' feature of Bugzilla (which requires the PatchReader Perl module
as well), you should install patchutils from:

    http://cyberelk.net/tim/patchutils/

Checking for               DBD-Pg (v1.45)     ok: found v2.19.2 
Checking for           PostgreSQL (v8.03.0000) ok: found v09.01.0400 
Checking for               DBD-Pg (v2.17.2)   ok: found v2.19.2 

Removing existing compiled templates...
Precompiling templates...done.
Fixing file permissions...
Deriving regex group memberships...
checksetup.pl complete.
(Reporter)

Updated

5 years ago
Hardware: x86_64 → x86
Version: unspecified → 4.2.1
(Assignee)

Updated

5 years ago
Keywords: qawanted

Comment 1

5 years ago
I'm receiving the error.  Here is my system:

Gentoo 3.4.4-gentoo
Perl v5.16.1
Apache v2.2.22-r1
mod_perl v2.0.7

modules:
dev-perl/Any-Moose-0.180.0:0
dev-perl/AnyEvent-7.10.0:0
dev-perl/Apache-DBI-1.110.0:0
dev-perl/Apache-Reload-0.120.0:0
dev-perl/Apache-SizeLimit-0.960.0:0
dev-perl/Apache-Test-1.370.0:0
dev-perl/AppConfig-1.660.0:0
dev-perl/Authen-SASL-2.150.0:0
dev-perl/B-Hooks-EndOfScope-0.110.0:0
dev-perl/Capture-Tiny-0.180.0:0
dev-perl/Chart-2.4.5:0
dev-perl/Class-Accessor-0.340.0:0
dev-perl/Class-Accessor-Lite-0.50.0:0
dev-perl/Class-C3-0.230.0:0
dev-perl/Class-Data-Inheritable-0.80.0:0
dev-perl/Class-Inspector-1.270.0:0
dev-perl/Class-Load-0.200.0:0
dev-perl/Class-Load-XS-0.40.0:0
dev-perl/Class-Singleton-1.400.0:0
dev-perl/Class-Trigger-0.140.0:0
dev-perl/Convert-ASN1-0.260.0:0
dev-perl/Crypt-Random-Source-0.70.0:0
dev-perl/Crypt-SSLeay-0.580.0:0
dev-perl/DBD-Pg-2.19.2:0
dev-perl/DBD-mysql-4.20.0:0
dev-perl/DBI-1.622.0:0
dev-perl/Daemon-Generic-0.820.0:0
dev-perl/Data-ObjectDriver-0.90.0:0
dev-perl/Data-OptList-0.107.0:0
dev-perl/DateTime-0.760.0:0
dev-perl/DateTime-Format-DateParse-0.50.0:0
dev-perl/DateTime-Format-Mail-0.300.100:0
dev-perl/DateTime-Format-W3CDTF-0.60.0:0
dev-perl/DateTime-Locale-0.450.0:0
dev-perl/DateTime-TimeZone-1.480.0:0
dev-perl/Devel-GlobalDestruction-0.90.0:0
dev-perl/Devel-StackTrace-1.270.0:0
dev-perl/Devel-StackTrace-AsHTML-0.110.0:0
dev-perl/Digest-HMAC-1.30.0:0
dev-perl/Digest-SHA1-2.130.0:0
dev-perl/Dist-CheckConflicts-0.20.0:0
dev-perl/Email-Abstract-3.3.0:0
dev-perl/Email-Address-1.896.0:0
dev-perl/Email-Date-Format-1.2.0:0
dev-perl/Email-MIME-1.911.0:0
dev-perl/Email-MIME-Attachment-Stripper-1.316.0:0
dev-perl/Email-MIME-ContentType-1.15.0:0
dev-perl/Email-MIME-Encodings-1.313.0:0
dev-perl/Email-MessageID-1.402.0:0
dev-perl/Email-Reply-1.202.0:0
dev-perl/Email-Send-2.198.0:0
dev-perl/Email-Simple-2.102.0:0
dev-perl/Encode-Detect-1.10.0:0
dev-perl/Encode-Locale-1.30.0:0
dev-perl/Error-0.170.180:0
dev-perl/Eval-Closure-0.80.0:0
dev-perl/File-Flock-2008.10.0:0
dev-perl/File-HomeDir-0.990.0:0
dev-perl/File-Listing-6.40.0:0
dev-perl/File-ShareDir-1.30.0:0
dev-perl/File-Slurp-9999.190.0:0
dev-perl/File-Which-1.90.0:0
dev-perl/Filesys-Notify-Simple-0.80.0:0
dev-perl/GD-2.460.0:0
dev-perl/GD-Graph3d-0.630.0:0
dev-perl/GDGraph-1.440.0:0
dev-perl/GDTextUtil-0.860.0:0
dev-perl/HTML-Parser-3.690.0:0
dev-perl/HTML-Scrubber-0.90.0:0
dev-perl/HTML-TableParser-0.380.0:0
dev-perl/HTML-Tagset-3.200.0:0
dev-perl/HTTP-Body-1.150.0:0
dev-perl/HTTP-Cookies-6.0.1:0
dev-perl/HTTP-Daemon-6.10.0:0
dev-perl/HTTP-Date-6.20.0:0
dev-perl/HTTP-Message-6.30.0:0
dev-perl/HTTP-Negotiate-6.0.1:0
dev-perl/Hash-MultiValue-0.120.0:0
dev-perl/IO-Socket-SSL-1.760.0:0
dev-perl/IO-stringy-2.110.0:0
dev-perl/JSON-2.530.0:0
dev-perl/JSON-RPC-1.30.0:0
dev-perl/JSON-XS-2.330.0:0
dev-perl/LWP-MediaTypes-6.20.0:0
dev-perl/LWP-Protocol-https-6.30.0:0
dev-perl/Linux-Pid-0.40.0:0
dev-perl/List-MoreUtils-0.330.0:0
dev-perl/Locale-gettext-1.50.0:0
dev-perl/MIME-Lite-3.28.0:0
dev-perl/MIME-Types-1.350.0:0
dev-perl/MIME-tools-5.503.0:0
dev-perl/MRO-Compat-0.110.0:0
dev-perl/MailTools-2.90.0:0
dev-perl/Math-Random-ISAAC-1.4.0:0
dev-perl/Math-Random-ISAAC-XS-1.4.0:0
dev-perl/Math-Random-Secure-0.60.0:0
dev-perl/Math-Round-0.60.0:0
dev-perl/Module-Find-0.110.0:0
dev-perl/Module-Implementation-0.60.0:0
dev-perl/Module-Runtime-0.13.0:0
dev-perl/Moose-2.60.300:0
dev-perl/Net-Daemon-0.480.0:0
dev-perl/Net-HTTP-6.30.0:0
dev-perl/Net-SMTP-SSL-1.10.0:0
dev-perl/Net-SSLeay-1.480.0-r1:0
dev-perl/Package-DeprecationManager-0.130.0:0
dev-perl/Package-Stash-0.330.0:0
dev-perl/Package-Stash-XS-0.250.0:0
dev-perl/Params-Util-1.60.0:0
dev-perl/Params-Validate-1.60.0:0
dev-perl/PatchReader-0.9.6:0
dev-perl/PlRPC-0.202.0:0
dev-perl/Plack-1.0.100:0
dev-perl/Return-Value-1.666.1:0
dev-perl/Router-Simple-0.90.0:0
dev-perl/SOAP-Lite-0.715.0:0
dev-perl/Sub-Exporter-0.984.0:0
dev-perl/Sub-Exporter-Progressive-0.1.4:0
dev-perl/Sub-Identify-0.40.0:0
dev-perl/Sub-Install-0.926.0:0
dev-perl/Sub-Name-0.50.0:0
dev-perl/Template-DBI-2.650.0:0
dev-perl/Template-GD-2.660.0:0
dev-perl/Template-Latex-2.170.0:0
dev-perl/Template-Toolkit-2.240.0:0
dev-perl/Template-XML-2.170.0:0
dev-perl/TermReadKey-2.300.0:0
dev-perl/Test-SharedFork-0.200.0:0
dev-perl/Test-TCP-1.160.0:0
dev-perl/Test-Taint-1.40.0:0
dev-perl/Text-Iconv-1.700.0:0
dev-perl/TheSchwartz-1.100.0:0
dev-perl/Tie-IxHash-1.220.0:0
dev-perl/TimeDate-1.200.0:0
dev-perl/Try-Tiny-0.110.0:0
dev-perl/URI-1.600.0:0
dev-perl/Variable-Magic-0.500.0:0
dev-perl/WWW-RobotRules-6.10.0:0
dev-perl/XML-DOM-1.440.0:0
dev-perl/XML-Filter-BufferText-1.10.0:0
dev-perl/XML-Handler-YAWriter-0.230.0:0
dev-perl/XML-LibXML-2.0.300:0
dev-perl/XML-NamespaceSupport-1.110.0:0
dev-perl/XML-Parser-2.410.0-r1:0
dev-perl/XML-RSS-1.490.0:0
dev-perl/XML-RegExp-0.40.0:0
dev-perl/XML-SAX-0.990.0:0
dev-perl/XML-SAX-Base-1.80.0:0
dev-perl/XML-SAX-Writer-0.530.0:0
dev-perl/XML-Simple-2.200.0:0
dev-perl/XML-Twig-3.390.0:0
dev-perl/XML-XPath-1.130.0:0
dev-perl/common-sense-3.600.0:0
dev-perl/libwww-perl-6.40.0:0
dev-perl/libxml-perl-0.80.0:0
dev-perl/namespace-clean-0.230.0:0
dev-perl/perl-ldap-0.440.0:0
dev-perl/text-autoformat-1.669.2:0
dev-perl/text-reform-1.200.0:0
(Assignee)

Comment 2

5 years ago
Confirmed. I get the same error message with Mageia 3 and Perl 5.16.0.
Severity: normal → major
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: blocking4.4+
Flags: blocking4.2.3+
Keywords: qawanted
OS: Linux → All
Hardware: x86 → All
Summary: Unable to attach a file to a bug with perl 5.16.0 (Gentoo) → Unable to attach a file to a bug with perl 5.16.0
Target Milestone: --- → Bugzilla 4.2
(Assignee)

Comment 3

5 years ago
Using CGI 3.59 on Perl 5.14.2 works fine.
Using CGI 3.59 on Perl 5.16.0 fails.

$cgi->param('data') correctly shows the name of the uploaded file, but $cgi->upload('data') is undefined, making Bugzilla to think that the file is missing.

Mark: are you aware of this bug?
(Assignee)

Comment 4

5 years ago
It looks like the /usr/tmp/CGItempxxx file is not created at all. If I run this code:

    my ($k) = keys %{$cgi->{'.tmpfiles'}};
    my $n = $cgi->{'.tmpfiles'}->{$k}->{name};
    my $f = Dumper($n);
    $f =~ m{/usr/tmp/CGItemp(\d+)};
    my $i = $1;
    open(FH, "<", "/usr/tmp/CGItemp$i") or die $!;
    my $s = -s FH;
    die $s;

open() is unable to find the file. So nothing is uploaded.

Comment 5

5 years ago
Have the same issue on my FreeBSD 9.0-RELEASE-p3 / Apache 2.2.22 / mod_perl 2.0.7 / Perl v5.16.0 / Bugzilla 4.2.2 right after upgrading to perl 5.16.

It seems that the real issue is about CGI.pm specifics to perl 5.16.

I have no fix or workaround so far.
(Assignee)

Comment 6

5 years ago
I'm not so sure it's a bug in CGI.pm itself, or at least it's more subtle than that. If I use CGI directly instead of Bugzilla::CGI, then I can upload files without any problem. So something is maybe wrong on our side.
(Assignee)

Comment 7

5 years ago
Created attachment 651907 [details]
PoC

Here is the script I used for my testing. The single line you need to edit to test with Bugzilla::CGI or CGI alone is line 12:

my $cgi = $bz_cgi;  # Use Bugzilla::CGI
my $cgi = $cpan_cgi # Use CGI only

With Perl 5.16, uploading files work fine with $cpan_cgi, but fails with $bz_cgi.
(Assignee)

Comment 8

5 years ago
Created attachment 651916 [details]
PoC

This PoC removes uninitialized warnings.
Attachment #651907 - Attachment is obsolete: true
(Assignee)

Comment 9

5 years ago
OK, I found what's wrong. The problem comes from Bugzilla::CGI::_fix_utf8(). For some reason which I don't understand yet, this line is causing trouble in Perl 5.16 only:

    # The is_utf8 is here in case CGI gets smart about utf8 someday.
    utf8::decode($input) if defined $input && !utf8::is_utf8($input);

If I remove this line, then everything works fine.
(Assignee)

Comment 10

5 years ago
The problem comes from utf8::decode($input). I printed the content of Dumper($input) right before and right after calling utf8::decode($input), and the result is different in Perl 5.14 and 5.16.

5.14.2:

before: $VAR1 = bless( \*{'Fh::fh00001bug0-6.diff'}, 'Fh' );
after:  $VAR1 = bless( \*{'Fh::fh00001bug0-6.diff'}, 'Fh' );

5.16.0:

before: $VAR1 = bless( \*{'Fh::fh00001bug0-6.diff'}, 'Fh' );
after:  $VAR1 = 'bug0-6.diff'

The results above are the same for $input = $cgi->param('file') and
$input = $cgi->upload('file').

I don't know if this change in Perl 5.16 is intentional or not, but it looks like we can fix the problem ourselves. Patch coming!
Assignee: attach-and-request → LpSolit
Status: NEW → ASSIGNED
(Assignee)

Comment 11

5 years ago
(In reply to Frédéric Buclin from comment #10)
> after:  $VAR1 = '<missing bug number>-6.diff'

Arhh, bmo mangled what I wrote, because it thought I was talking about bug zero. Let's say it should be:

 after:  $VAR1 = 'bugO-6.diff'
(Assignee)

Comment 12

5 years ago
Per https://rt.perl.org/rt3/Public/Bug/Display.html?id=91852, this behavior change is intentional and has been fixed in Perl 5.16. It was considered a bug.

I'm removing Mark from the CC list as his module is not the culprit and I don't want to spam him any longer. :)
(Assignee)

Comment 13

5 years ago
Created attachment 651946 [details] [diff] [review]
patch, v1

If it's a filehandle, then we must not decode it anymore as it will be automatically stringified now.
Attachment #651946 - Flags: review?(dkl)

Comment 14

5 years ago
Great thanks to Frédéric Buclin , attachment 651946 [details] [diff] [review] works for me as well.

Comment 15

5 years ago
I confirm that the patch worked for me, too.
Comment on attachment 651946 [details] [diff] [review]
patch, v1

Review of attachment 651946 [details] [diff] [review]:
-----------------------------------------------------------------

r=dkl
Attachment #651946 - Flags: review?(dkl) → review+
(Assignee)

Updated

5 years ago
Flags: approval4.2+
Flags: approval+
(Assignee)

Comment 17

5 years ago
Committing to: bzr+ssh://lpsolit%40gmail.com@bzr.mozilla.org/bugzilla/trunk/
modified Bugzilla/CGI.pm
Committed revision 8346.

Committing to: bzr+ssh://lpsolit%40gmail.com@bzr.mozilla.org/bugzilla/4.2/
modified Bugzilla/CGI.pm
Committed revision 8119.
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED

Comment 18

5 years ago
I was happy with that solution, but not so long. In fact the files attached with the most recent patch are broken. For the instance, attaching any zip archive via web form you can download it back, but it can't be unzipped anymore.
(Assignee)

Comment 19

5 years ago
(In reply to Alexei Volkov from comment #18)
> with the most recent patch are broken. For the instance, attaching any zip
> archive via web form you can download it back, but it can't be unzipped
> anymore.

I cannot reproduce your issue. Binary files are correctly uploaded/downloaded, and I can open them as expected. My patch has nothing to do with the content of attachments anyway, it only prevented the filehandle from being converted into a string.
this patch also addresses issues with searching multi-value fields on 4.0.

given this patch is trivial, breaks core bugzilla functionality, and is being encountered more frequently as distros upgrade perl to 5.16, requesting approval for a 4.0 commit (to ride along with the next security release).
Flags: approval4.0?
(Assignee)

Comment 21

5 years ago
(In reply to Byron Jones ‹:glob› from comment #20)
> being encountered more frequently as distros upgrade perl to 5.16,

I would expect that if distros provide Perl 5.16, they also provide a pretty recent version of Bugzilla, i.e. 4.2. But it doesn't hurt to take it for 4.0 too.
Flags: approval4.0? → approval4.0+
Target Milestone: Bugzilla 4.2 → Bugzilla 4.0
(Assignee)

Comment 22

5 years ago
Committing to: bzr+ssh://lpsolit%40gmail.com@bzr.mozilla.org/bugzilla/4.0/
modified Bugzilla/CGI.pm
Committed revision 7741.
You need to log in before you can comment on or make changes to this bug.