467920 - Email sending doesn't work if you save a template with DOS newlines

Status: RESOLVED FIXED

(Bugzilla :: Email Notifications, defect)

Description

[Dan]

User-Agent:       Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506; InfoPath.2)
Build Identifier: 

UTF-8 DOS formatted email templates nolonger work as of the 3.2 release.  

Reproducible: Always

Steps to Reproduce:
1) Use the CVS install method with the default CVS client settings for Windows or download the tar.gz package and use WinZip with the default Winzip settings to extract Bugzilla then all non-binary files will be have the CTRL-LF converted to UTF8 DOS format.  The default settings for both CVS and Winzip are to convert text files to DOS format for readability reasons, you must disable this functionality by default.

2) I've not tested a clean install at this time, only an upgrade from 2.22.4 and 3.0.6.  So upgrade to 3.2 from one of these releases.  I belive you will see the issue on a clean install as well.

3) Setup email to use SMTP.

4) Create or update an existing bug to cause Bugzilla to generate an outbound email message.



Actual Results:  
The email message that will be sent out will have corrupted email headers.

Expected Results:  
The email message shouldn't have corrupt email headers.

Here is what the email headers look like before the upgrade to 3.2...

Date: Sun, 23 Nov 2008 22:59:31 -0600
Message-ID: <[e-mail]>
Subject: [Bug 1345] test
X-Bugzilla-Product: TestProduct
In-Reply-To: <[e-mail]/>
X-Mailer: Mail::Mailer[v2.04] Net::SMTP[v2.30]
X-Bugzilla-Severity: 4-normal
X-Bugzilla-Keywords:
X-Bugzilla-Reason: CC
X-Bugzilla-Component: TestComponent
To: <[e-mail]>
From: <[e-mail]>
X-pstn-neptune: 0/0/0.00/0
X-pstn-levels: (S:10.86326/99.90000 CV:99.9999 R:95.9108 P:95.9108 M:97.0282 
C:98.6951 )
X-pstn-settings: 3 (1.0000:1.0000) s cv gt3 gt2 gt1 r p m c
X-pstn-addresses: from <[e-mail]> forward (user 
good) [398/15]
MIME-Version: 1.0
Content-Type: text/plain
Return-Path: [e-mail]

After the upgrade to the 3.2 release, several of the email headers are 
jammed in the From: mail header.plus a bunch of other garbage.

Date: Sun, 23 Nov 2008 23:11:13 -0600
From:
                <"[e-mail] 
To:[e-mail]:dmclaughlin"@mydomain.org.Subject:[Bug.3420]>,
                <"New:test.X-Bugzilla-Reason:Reporter.X-Bugzilla-Type:newchanged.X-Bugzilla-Watch-Reason:None.Auto-Submitted:auto-generated.Con"@mydomain.org>,
                <[e-mail]>, <[e-mail]>, <[e-mail]>,
                <"23:23.Nov.2008.23:11:13.-0600"@mydomain.org>
X-Bugzilla-Product: CRIS
X-Bugzilla-Component: Common
X-Bugzilla-Keywords:
X-Bugzilla-Severity: 5-minor
X-Bugzilla-Who: [e-mail]
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: [e-mail]
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields:
Message-ID: <[e-mail]/>
To: undisclosed-recipients:;
X-pstn-neptune: 0/0/0.00/0
X-pstn-levels: (S: 0.00889/97.87704 CV:99.9999 R:95.9108 P:95.9108 M:97.0282 
C:98.6951 )
X-pstn-settings: 3 (1.0000:1.0000) s cv gt3 gt2 gt1 r p m c
X-pstn-addresses: from <[e-mail] 
To:[e-mail]:dmclaughlin> [398/15]
X-pstn-disposition: quarantine
MIME-Version: 1.0
Content-Type: text/plain
Return-Path: [e-mail]


The only bug I could find that looked remotely similar was this: 
https://bugzilla.mozilla.org/attachment.cgi?id=335914&action=diff&headers=1&context=file#mozilla/webtools/bugzilla/Bugzilla/Mailer.pm_sec1

Here are the versions of everything we are running.



httpd.exe -V
Server version: Apache/2.2.10 (Win32)
Server built:   Oct 10 2008 12:39:04
Server's Module Magic Number: 20051115:18
Server loaded:  APR 1.3.3, APR-Util 1.3.4
Compiled using: APR 1.3.3, APR-Util 1.3.4
Architecture:   32-bit
Server MPM:     WinNT
  threaded:     yes (fixed thread count)
    forked:     no
Server compiled with....
 -D APACHE_MPM_DIR="server/mpm/winnt"
 -D APR_HAS_SENDFILE
 -D APR_HAS_MMAP
 -D APR_HAS_OTHER_CHILD
 -D AP_HAVE_RELIABLE_PIPED_LOGS
 -D DYNAMIC_MODULE_LIMIT=128
 -D HTTPD_ROOT="/apache"
 -D SUEXEC_BIN="/apache/bin/suexec"
 -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
 -D DEFAULT_ERRORLOG="logs/error.log"
 -D AP_TYPES_CONFIG_FILE="conf/mime.types"
 -D SERVER_CONFIG_FILE="conf/httpd.conf"
mysql -V
mysql  Ver 14.14 Distrib 5.1.30, for Win32 (ia32)

perl -V
Set up gcc environment - 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)
Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
  Platform:
    osname=MSWin32, osvers=5.00, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags 
='-DNDEBUG -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DUSE_SITECUSTOMIZE 
 -DPRIVLIB_LAST_IN_INC -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO 
 -DPERL_MSVCRT_READFI
 -fno-strict-aliasing -mms-bitfields',
    optimize='-O2',
    cppflags='-DWIN32'
    ccversion='', gccversion='3.4.4 (cygming special, gdc 0.12, using dmd 
0.125)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', 
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='g++', ldflags ='-L"D:\Perl\lib\CORE"'
    libpth=\lib
    libs=-lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 
 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 
 -lodbccp32 -lmsvcrt
    perllibs=-lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 
 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 
 -lodbccp32 -lmsvcrt
    libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl510.lib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-mdll -L"D:\Perl\lib\CORE"'


Characteristics of this binary (from libperl):
  Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
                        PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS
                        PERL_MALLOC_WRAP PL_OP_SLAB_ALLOC USE_ITHREADS
                        USE_LARGE_FILES USE_PERLIO USE_SITECUSTOMIZE
  Locally applied patches:
        ActivePerl Build 1004 [287188]
        33741 avoids segfaults invoking S_raise_signal() (on Linux)
        33763 Win32 process ids can have more than 16 bits
        32809 Load 'loadable object' with non-default file extension
        32728 64-bit fix for Time::Local
  Built under MSWin32
  Compiled at Sep  3 2008 13:16:37
  @INC:
    D:/Perl/site/lib
    D:/Perl/lib

checksetup.pl
* This is Bugzilla 3.2 on perl 5.10.0
* Running on WinXP/.Net Build 2600 (Service Pack 2)

Checking perl modules...
Checking for              CGI.pm (v3.33)   ok: found v3.41
Checking for            TimeDate (v2.21)   ok: found v2.22
Checking for           PathTools (v0.84)   ok: found v3.2501
Checking for                 DBI (v1.41)   ok: found v1.607
Checking for    Template-Toolkit (v2.15)   ok: found v2.20
Checking for          Email-Send (v2.16)   ok: found v2.192
Checking for          Email-MIME (v1.861)  ok: found v1.861
Checking for Email-MIME-Modifier (v1.442)  ok: found v1.442

Checking available perl DBD modules...
Checking for              DBD-Pg (v1.45)    not found
Checking for           DBD-mysql (v4.00)   ok: found v4.005
Checking for          DBD-Oracle (v1.19)   ok: found v1.21

The following Perl modules are optional:
Checking for                  GD (v1.20)   ok: found v2.41
Checking for               Chart (v1.0)    ok: found v2.4.1
Checking for         Template-GD (any)     ok: found v1.56
Checking for          GDTextUtil (any)     ok: found v0.86
Checking for             GDGraph (any)     ok: found v1.44
Checking for            XML-Twig (any)     ok: found v3.32
Checking for          MIME-tools (v5.406)  ok: found v5.427
Checking for         libwww-perl (any)     ok: found v5.814
Checking for         PatchReader (v0.9.4)  ok: found v0.9.5
Checking for          PerlMagick (any)      not found
Checking for           perl-ldap (any)     ok: found v0.39
Checking for         Authen-SASL (any)      not found
Checking for          RadiusPerl (any)      not found
Checking for           SOAP-Lite (any)     ok: found v0.69
Checking for         HTML-Parser (v3.40)   ok: found v3.56
Checking for       HTML-Scrubber (any)     ok: found v0.08
Checking for Email-MIME-Attachment-Stripper (any)     ok: found v1.315
Checking for         Email-Reply (any)     ok: found v1.202
Checking for            mod_perl (v1.999022)  not found

Comment 1

[Dan]

In all previous releases the templates could be either DOS or UNIX formatted 
and they would work.  But in the 3.2 release only a UNIX formatted 
newchangedmail.txt.tmpl will work.  There looks to be a lot of code in 3.2 
that has changed related to email encoding, so I'm assuming that has 
something to do with why DOS formatted templates won't work anymore.

For now the workarounds are:

1) Use the CVS method for installing, but only if you configure the CVS client to use UNIX formatting for txt files.
2) Use Cygwin tar to decompress the tar.gz.
3) Use Winzip to decompress the tar.gz, but only if you have the Smart CR/LF 
functionality turned off.
4) Use a thirdparty utility sed, vi/vim or Windows text editor such as 
UltraEdit to convert the templates so they unix formatted.

Comment 2

[Max Kanat-Alexander]

DOS newlines are not just \r-- I have no idea how you got such a file. Any line that ends with just \r is clearly not a valid email header, which makes it tempting to mark this INVALID.

Comment 3

[Dan]

Attachment: Debug email log from 3.0.6 showing DOS and UNIX formatted templates work.

This morning I downloaded a new 3.0.6 tar.gz and ran a test using a DOS then a UNIX formatted template.  There are no problems in the 3.0.6 release.

Comment 4

[Dan]

Attachment: Debug email log from 3.2 showing DOS formatted templates don't work anymore.

I downloaded a new 3.2 tar.gz this morning and in the 3.2 release only the UNIX formatted template works.

Comment 5

[Dan]

This morning I tested 3.2 on our origial Windows 2003 server, my Windows XP workstation and on Windows 2003 VM.  I was able to reproduce the issue in all three environments by simply converting the template back and forth from DOS to UNIX formatting.  Doing the same tests with the 3.0.6 release worked in every case.

Comment 6

[Dan]

If you look at the logs I attached you will see that on 3.0.6 the From header with both a DOS and UNIX formatted template always has the comma after the \r. 

Here is the DOS formatted on 3.0.6:

[Fri Dec 05 08:20:29 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x2ae378c)>>> From: [e-mail]\r, referer: http://cricket.mydomain.org:8180/show_bug.cgi?id=1345
[Fri Dec 05 08:20:29 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x2ae378c)>>> To: [e-mail]\r, referer: http://cricket.mydomain.org:8180/show_bug.cgi?id=1345
[Fri Dec 05 08:20:29 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x2ae378c)>>> Subject: [Bug 1345] test\r, referer: http://cricket.mydomain.org:8180/show_bug.cgi?id=1345
[Fri Dec 05 08:20:29 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x2ae378c)>>> X-Bugzilla-Reason: Reporter\r, referer: http://cricket.mydomain.org:8180/show_bug.cgi?id=1345
[Fri Dec 05 08:20:29 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x2ae378c)>>> X-Bugzilla-Type: newchanged\r, referer: http://cricket.mydomain.org:8180/show_bug.cgi?id=1345
[Fri Dec 05 08:20:29 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x2ae378c)>>> X-Bugzilla-Watch-Reason: None\r, referer: http://cricket.mydomain.org:8180/show_bug.cgi?id=1345

Here is the UNIX formatted on 3.0.6:

[Fri Dec 05 08:22:00 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x2ae55f4)>>> From: [e-mail], referer: http://cricket.mydomain.org:8180/process_bug.cgi
[Fri Dec 05 08:22:00 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x2ae55f4)>>> To: [e-mail], referer: http://cricket.mydomain.org:8180/process_bug.cgi
[Fri Dec 05 08:22:00 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x2ae55f4)>>> Subject: [Bug 1345] test, referer: http://cricket.mydomain.org:8180/process_bug.cgi
[Fri Dec 05 08:22:00 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x2ae55f4)>>> X-Bugzilla-Reason: Reporter, referer: http://cricket.mydomain.org:8180/process_bug.cgi
[Fri Dec 05 08:22:00 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x2ae55f4)>>> X-Bugzilla-Type: newchanged, referer: http://cricket.mydomain.org:8180/process_bug.cgi
[Fri Dec 05 08:22:00 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x2ae55f4)>>> X-Bugzilla-Watch-Reason: None, referer: http://cricket.mydomain.org:8180/process_bug.cgi

When you look at the logs from the 3.2 release you will see that if the template is DOS formatted the comma after the \r is gone and all the headers are jammed together.

Here is the DOS formatted on 3.2:

[Fri Dec 05 07:33:41 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x2f3c4bc)>>> From: [e-mail]\rTo: [e-mail]\rSubject: [Bug 1345] test\rX-Bugzilla-Reason: Reporter\rX-Bugzilla-Type: newchanged\rX-Bugzilla-Watch-Reason: None\rAuto-Submitted: auto-generated\rContent-Type: text/plain; charset="UTF-8"\rMIME-Version: 1.0\rDate: Fri,  5 Dec 2008 07:33:37 -0600\r\r, referer: http://cricket.mydomain.org:8180/process_bug.cgi

Here is the UNIX formatted on 3.2:

[Fri Dec 05 07:35:59 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x20852f0)>>> From: [e-mail], referer: http://cricket.mydomain.org:8180/process_bug.cgi
[Fri Dec 05 07:35:59 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x20852f0)>>> To: [e-mail], referer: http://cricket.mydomain.org:8180/process_bug.cgi
[Fri Dec 05 07:35:59 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x20852f0)>>> Subject: [Bug 1345] test, referer: http://cricket.mydomain.org:8180/process_bug.cgi
[Fri Dec 05 07:35:59 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x20852f0)>>> X-Bugzilla-Reason: Reporter, referer: http://cricket.mydomain.org:8180/process_bug.cgi
[Fri Dec 05 07:35:59 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x20852f0)>>> X-Bugzilla-Type: newchanged, referer: http://cricket.mydomain.org:8180/process_bug.cgi
[Fri Dec 05 07:35:59 2008] [error] [client 192.168.2.2] Net::SMTP=GLOB(0x20852f0)>>> X-Bugzilla-Watch-Reason: None, referer: http://cricket.mydomain.org:8180/process_bug.cgi

If I can be of any further assistance please don't hesitate to ask.  We have had a death in the family and I will be away for the next few days, so if I don't respond that is why.

Comment 7

[Marc Schumann [:Wurblzap]]

Hmmm... I see you're using Cygwin. Cygwin does some magic on its own when it
comes to line endings. I wouldn't be too surprised if this is what's behind it.
I'm using ActiveState Perl on Windows with a Bugzilla configured to use SMTP,
and I don't experience such a problem.

Please continue handling this at @support at your initial posting, as I believe
this is not an issue of Bugzilla itself (which would make this bug INVALID, as
Max suggested).

Comment 8

[Marc Schumann [:Wurblzap]]

Oh, and the comma is probably being added by the program doing the logging, I'd say.

Comment 9

[Dan]

Yes. I'm sure the \r is logging related because the actual mail headers as seen in my earlier postings to this ticket don't show the \r, only that the headers are all jammed together.  Sorry for the confusion, I wasn't trying to state that the \r has anything to do with it, but was only trying to describe where in the log file to see the issue in the logs.

Comment 10

[Frédéric Buclin]

Reopen if you can reproduce the problem using ActivePerl.

Comment 11

[Dan]

I am using Active Perl, seem my posting above.  I think there was an assumption that I was using Cygwin because I stated that you could use Cygwin tar to decompress the tar.gz to avoid having the templates converted to DOS format.  I've reproduced this issue on three completely seperate environments and only one of them has Cygwin installed.  Infact I just now uninstalled Cygwin from my XP workstation, deleted the Cywin directory and rebooted.  I'm still seeing the exact same issue.

Comment 12

[Dan]

Attachment: DOS and UNIX UTF8 Formatted Templates to test with.

Here are some test templates for you to use to reproducte the issue.  Switch them out with the default template in 3.2 (specifically the DOS formatted tmpl) and you should see the same issue.

Comment 13

[Dan]

Without access to vim, dos2unix, or sed on Windows....I've found that UltraEdit or jEdit are useful for converting files between different formats and encodings.

Comment 14

[Marc Schumann [:Wurblzap]]

The assumption that you're using Cygwin stems from here:

(In reply to comment #0)
> perl -V
> Set up gcc environment - 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)

So if you're running checksetup.pl, I assume it uses Cygwin, mangling templates
on precompilation. This *will* hurt even if your Apache and registry are set up
so that ActiveState Perl is used for CGI.

If in doubt, uninstall Perl from Cygwin.
http://www.bugzilla.org/docs/win32install.html as well as
http://www.bugzilla.org/docs/tip/en/html/os-specific.html ask for ActiveState
Perl.

Comment 15

[Dan]

The comment you are referring only means that Active State is using gcc from Cygwin.

Regardless, I only had Cygwin on one of the environments and I just uninstalled it and deleted the directory and rebooted and the problem doesn't go away.  To confirm, on the one system that had Cygwin that Cygwin has nothing to do with it I've deleted the entire bugzilla directory, reextracted the tar.gz, and restored the database to 2.22.4.  I will post my findings shortly.

D:\WINDOWS>perl -V
Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
  Platform:
    osname=MSWin32, osvers=5.00, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DUSE_SITEC
USTOMIZE -DPRIVLIB_LAST_IN_INC -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX',
    optimize='-MD -Zi -DNDEBUG -O1',
    cppflags='-DWIN32'
    ccversion='12.00.8804', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf  -libpath:"D:\Perl\lib\CORE"  -machine:x86'
    libpth=\lib
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib
oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.
lib oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl510.lib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf  -libpath:"D:\Perl\lib\CORE"  -machine:x86
'


Characteristics of this binary (from libperl):
  Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
                        PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS
                        PERL_MALLOC_WRAP PL_OP_SLAB_ALLOC USE_ITHREADS
                        USE_LARGE_FILES USE_PERLIO USE_SITECUSTOMIZE
  Locally applied patches:
        ActivePerl Build 1004 [287188]
        33741 avoids segfaults invoking S_raise_signal() (on Linux)
        33763 Win32 process ids can have more than 16 bits
        32809 Load 'loadable object' with non-default file extension
        32728 64-bit fix for Time::Local
  Built under MSWin32
  Compiled at Sep  3 2008 13:16:37
  @INC:
    D:/Perl/site/lib
    D:/Perl/lib

Comment 16

[Dan]

I restored my 2.22.4 database, extracted the 3.2 tar.gz using Winzip with default settings (this will convert text files to DOS format, but using the CVS install method will also convert text files to DOS format by default), confirmed the perl version, ran checksetup.pl.  The problem is still there.  All I do to fix it is open the the template in UltraEdit and from the File option choose Conversions->DOS to UNIX, save it, and problem is gone.  If I change the template back by choosing Conversions->UNIX to DOS then the problem is back.  BTW... I checked, and the newchangedmail.txt.tmpl is in the tar file and it is not modified by the checksetup.pl because the checksums before and after running checksetup.pl are identical.


D:\servers\bugzilla\bugzilla-3.2>perl -V
Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
  Platform:
    osname=MSWin32, osvers=5.00, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DUSE_SITEC
USTOMIZE -DPRIVLIB_LAST_IN_INC -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX',
    optimize='-MD -Zi -DNDEBUG -O1',
    cppflags='-DWIN32'
    ccversion='12.00.8804', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf  -libpath:"D:\Perl\lib\CORE"  -machine:x86'
    libpth=\lib
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib
oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.
lib oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl510.lib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf  -libpath:"D:\Perl\lib\CORE"  -machine:x86
'


Characteristics of this binary (from libperl):
  Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
                        PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS
                        PERL_MALLOC_WRAP PL_OP_SLAB_ALLOC USE_ITHREADS
                        USE_LARGE_FILES USE_PERLIO USE_SITECUSTOMIZE
  Locally applied patches:
        ActivePerl Build 1004 [287188]
        33741 avoids segfaults invoking S_raise_signal() (on Linux)
        33763 Win32 process ids can have more than 16 bits
        32809 Load 'loadable object' with non-default file extension
        32728 64-bit fix for Time::Local
  Built under MSWin32
  Compiled at Sep  3 2008 13:16:37
  @INC:
    D:/Perl/site/lib
    D:/Perl/lib
    .

D:\servers\bugzilla\bugzilla-3.2>perl checksetup.pl
* This is Bugzilla 3.2 on perl 5.10.0
* Running on WinXP/.Net Build 2600 (Service Pack 2)

Checking perl modules...
Checking for              CGI.pm (v3.33)   ok: found v3.41
Checking for            TimeDate (v2.21)   ok: found v2.22
Checking for           PathTools (v0.84)   ok: found v3.2501
Checking for                 DBI (v1.41)   ok: found v1.607
Checking for    Template-Toolkit (v2.15)   ok: found v2.20
Checking for          Email-Send (v2.16)   ok: found v2.192
Checking for          Email-MIME (v1.861)  ok: found v1.861
Checking for Email-MIME-Modifier (v1.442)  ok: found v1.442

Checking available perl DBD modules...
Checking for              DBD-Pg (v1.45)    not found
Checking for           DBD-mysql (v4.00)   ok: found v4.005
Checking for          DBD-Oracle (v1.19)   ok: found v1.21

The following Perl modules are optional:
Checking for                  GD (v1.20)   ok: found v2.41
Checking for               Chart (v1.0)    ok: found v2.4.1
Checking for         Template-GD (any)     ok: found v1.56
Checking for          GDTextUtil (any)     ok: found v0.86
Checking for             GDGraph (any)     ok: found v1.44
Checking for            XML-Twig (any)     ok: found v3.32
Checking for          MIME-tools (v5.406)  ok: found v5.427
Checking for         libwww-perl (any)     ok: found v5.814
Checking for         PatchReader (v0.9.4)  ok: found v0.9.5
Checking for          PerlMagick (any)      not found
Checking for           perl-ldap (any)     ok: found v0.39
Checking for         Authen-SASL (any)      not found
Checking for          RadiusPerl (any)      not found
Checking for           SOAP-Lite (any)     ok: found v0.69
Checking for         HTML-Parser (v3.40)   ok: found v3.56
Checking for       HTML-Scrubber (any)     ok: found v0.08
Checking for Email-MIME-Attachment-Stripper (any)     ok: found v1.315
Checking for         Email-Reply (any)     ok: found v1.202
Checking for            mod_perl (v1.999022)  not found

* NOTE: You must run any commands listed below as Administrator.

***********************************************************************
* Note For Windows Users                                              *
***********************************************************************
* In order to install the modules listed below, you first have to run *
* the following command as an Administrator:                          *
*                                                                     *
*   ppm repo add theory58S http://cpan.uwinnipeg.ca/PPMPackages/10xx/
*                                                                     *
***********************************************************************
**********************************************************************
* OPTIONAL MODULES                                                   *
**********************************************************************
* Certain Perl modules are not required by Bugzilla, but by          *
* installing the latest version you gain access to additional        *
* features.                                                          *
*                                                                    *
* The optional modules you do not have installed are listed below,   *
* with the name of the feature they enable. If you want to install   *
* one of these modules, just run the appropriate command in the      *
* "COMMANDS TO INSTALL" section.                                     *
**********************************************************************

***********************************************************************
* MODULE NAME * ENABLES FEATURE(S)                                    *
***********************************************************************
*  PerlMagick * Optionally Convert BMP Attachments to PNGs            *
* Authen-SASL * SMTP Authentication                                   *
*  RadiusPerl * RADIUS Authentication                                 *
*    mod_perl * mod_perl                                              *
***********************************************************************
COMMANDS TO INSTALL:

     PerlMagick: ppm install PerlMagick
    Authen-SASL: ppm install Authen-SASL
     RadiusPerl: ppm install RadiusPerl
       mod_perl: ppm install mod_perl
Reading ./localconfig...

OPTIONAL NOTE: If you want to be able to use the 'difference between two
patches' feature of Bugzilla (which requires the PatchReader Perl module
as well), you should install patchutils from:

    http://cyberelk.net/tim/patchutils/


This version of Bugzilla contains some variables that you may want to
change and adapt to your local settings. Please edit the file
./localconfig and rerun checksetup.pl.

The following variables are new to ./localconfig since you last ran
checksetup.pl:  create_htaccess, webservergroup, db_driver, db_host, db_name, db_user, db_pass, db_port, db_sock, db_che
ck, index_html, cvsbin, interdiffbin, diffpath


D:\servers\bugzilla\bugzilla-3.2>notepad localconfig

D:\servers\bugzilla\bugzilla-3.2>perl checksetup.pl
* This is Bugzilla 3.2 on perl 5.10.0
* Running on WinXP/.Net Build 2600 (Service Pack 2)

Checking perl modules...
Checking for              CGI.pm (v3.33)   ok: found v3.41
Checking for            TimeDate (v2.21)   ok: found v2.22
Checking for           PathTools (v0.84)   ok: found v3.2501
Checking for                 DBI (v1.41)   ok: found v1.607
Checking for    Template-Toolkit (v2.15)   ok: found v2.20
Checking for          Email-Send (v2.16)   ok: found v2.192
Checking for          Email-MIME (v1.861)  ok: found v1.861
Checking for Email-MIME-Modifier (v1.442)  ok: found v1.442

Checking available perl DBD modules...
Checking for              DBD-Pg (v1.45)    not found
Checking for           DBD-mysql (v4.00)   ok: found v4.005
Checking for          DBD-Oracle (v1.19)   ok: found v1.21

The following Perl modules are optional:
Checking for                  GD (v1.20)   ok: found v2.41
Checking for               Chart (v1.0)    ok: found v2.4.1
Checking for         Template-GD (any)     ok: found v1.56
Checking for          GDTextUtil (any)     ok: found v0.86
Checking for             GDGraph (any)     ok: found v1.44
Checking for            XML-Twig (any)     ok: found v3.32
Checking for          MIME-tools (v5.406)  ok: found v5.427
Checking for         libwww-perl (any)     ok: found v5.814
Checking for         PatchReader (v0.9.4)  ok: found v0.9.5
Checking for          PerlMagick (any)      not found
Checking for           perl-ldap (any)     ok: found v0.39
Checking for         Authen-SASL (any)      not found
Checking for          RadiusPerl (any)      not found
Checking for           SOAP-Lite (any)     ok: found v0.69
Checking for         HTML-Parser (v3.40)   ok: found v3.56
Checking for       HTML-Scrubber (any)     ok: found v0.08
Checking for Email-MIME-Attachment-Stripper (any)     ok: found v1.315
Checking for         Email-Reply (any)     ok: found v1.202
Checking for            mod_perl (v1.999022)  not found

* NOTE: You must run any commands listed below as Administrator.

***********************************************************************
* Note For Windows Users                                              *
***********************************************************************
* In order to install the modules listed below, you first have to run *
* the following command as an Administrator:                          *
*                                                                     *
*   ppm repo add theory58S http://cpan.uwinnipeg.ca/PPMPackages/10xx/
*                                                                     *
***********************************************************************
**********************************************************************
* OPTIONAL MODULES                                                   *
**********************************************************************
* Certain Perl modules are not required by Bugzilla, but by          *
* installing the latest version you gain access to additional        *
* features.                                                          *
*                                                                    *
* The optional modules you do not have installed are listed below,   *
* with the name of the feature they enable. If you want to install   *
* one of these modules, just run the appropriate command in the      *
* "COMMANDS TO INSTALL" section.                                     *
**********************************************************************

***********************************************************************
* MODULE NAME * ENABLES FEATURE(S)                                    *
***********************************************************************
*  PerlMagick * Optionally Convert BMP Attachments to PNGs            *
* Authen-SASL * SMTP Authentication                                   *
*  RadiusPerl * RADIUS Authentication                                 *
*    mod_perl * mod_perl                                              *
***********************************************************************
COMMANDS TO INSTALL:

     PerlMagick: ppm install PerlMagick
    Authen-SASL: ppm install Authen-SASL
     RadiusPerl: ppm install RadiusPerl
       mod_perl: ppm install mod_perl
Reading ./localconfig...

OPTIONAL NOTE: If you want to be able to use the 'difference between two
patches' feature of Bugzilla (which requires the PatchReader Perl module
as well), you should install patchutils from:

    http://cyberelk.net/tim/patchutils/

Checking for           DBD-mysql (v4.00)   ok: found v4.005
Checking for           MySQL (v4.1.2)  ok: found v5.1.30-community

Creating database bugs...
The 'bugs' database could not be created. The error returned was:

    Access denied for user 'bugs'@'%' to database 'bugs'

This might have several reasons:

* MySQL is not running.
* MySQL is running, but there is a problem either in the
  server configuration or the database access rights. Read the Bugzilla
  Guide in the doc directory. The section about database configuration
  should help.
* Your password for the 'bugs' user, specified in $db_pass, is
  incorrect, in './localconfig'.
* There is a subtle problem with Perl, DBI, or MySQL. Make
  sure all settings in './localconfig' are correct. If all else fails, set
  '$db_check' to 0.


D:\servers\bugzilla\bugzilla-3.2>perl checksetup.pl
* This is Bugzilla 3.2 on perl 5.10.0
* Running on WinXP/.Net Build 2600 (Service Pack 2)

Checking perl modules...
Checking for              CGI.pm (v3.33)   ok: found v3.41
Checking for            TimeDate (v2.21)   ok: found v2.22
Checking for           PathTools (v0.84)   ok: found v3.2501
Checking for                 DBI (v1.41)   ok: found v1.607
Checking for    Template-Toolkit (v2.15)   ok: found v2.20
Checking for          Email-Send (v2.16)   ok: found v2.192
Checking for          Email-MIME (v1.861)  ok: found v1.861
Checking for Email-MIME-Modifier (v1.442)  ok: found v1.442

Checking available perl DBD modules...
Checking for              DBD-Pg (v1.45)    not found
Checking for           DBD-mysql (v4.00)   ok: found v4.005
Checking for          DBD-Oracle (v1.19)   ok: found v1.21

The following Perl modules are optional:
Checking for                  GD (v1.20)   ok: found v2.41
Checking for               Chart (v1.0)    ok: found v2.4.1
Checking for         Template-GD (any)     ok: found v1.56
Checking for          GDTextUtil (any)     ok: found v0.86
Checking for             GDGraph (any)     ok: found v1.44
Checking for            XML-Twig (any)     ok: found v3.32
Checking for          MIME-tools (v5.406)  ok: found v5.427
Checking for         libwww-perl (any)     ok: found v5.814
Checking for         PatchReader (v0.9.4)  ok: found v0.9.5
Checking for          PerlMagick (any)      not found
Checking for           perl-ldap (any)     ok: found v0.39
Checking for         Authen-SASL (any)      not found
Checking for          RadiusPerl (any)      not found
Checking for           SOAP-Lite (any)     ok: found v0.69
Checking for         HTML-Parser (v3.40)   ok: found v3.56
Checking for       HTML-Scrubber (any)     ok: found v0.08
Checking for Email-MIME-Attachment-Stripper (any)     ok: found v1.315
Checking for         Email-Reply (any)     ok: found v1.202
Checking for            mod_perl (v1.999022)  not found

* NOTE: You must run any commands listed below as Administrator.

***********************************************************************
* Note For Windows Users                                              *
***********************************************************************
* In order to install the modules listed below, you first have to run *
* the following command as an Administrator:                          *
*                                                                     *
*   ppm repo add theory58S http://cpan.uwinnipeg.ca/PPMPackages/10xx/
*                                                                     *
***********************************************************************
**********************************************************************
* OPTIONAL MODULES                                                   *
**********************************************************************
* Certain Perl modules are not required by Bugzilla, but by          *
* installing the latest version you gain access to additional        *
* features.                                                          *
*                                                                    *
* The optional modules you do not have installed are listed below,   *
* with the name of the feature they enable. If you want to install   *
* one of these modules, just run the appropriate command in the      *
* "COMMANDS TO INSTALL" section.                                     *
**********************************************************************

***********************************************************************
* MODULE NAME * ENABLES FEATURE(S)                                    *
***********************************************************************
*  PerlMagick * Optionally Convert BMP Attachments to PNGs            *
* Authen-SASL * SMTP Authentication                                   *
*  RadiusPerl * RADIUS Authentication                                 *
*    mod_perl * mod_perl                                              *
***********************************************************************
COMMANDS TO INSTALL:

     PerlMagick: ppm install PerlMagick
    Authen-SASL: ppm install Authen-SASL
     RadiusPerl: ppm install RadiusPerl
       mod_perl: ppm install mod_perl
Reading ./localconfig...

OPTIONAL NOTE: If you want to be able to use the 'difference between two
patches' feature of Bugzilla (which requires the PatchReader Perl module
as well), you should install patchutils from:

    http://cyberelk.net/tim/patchutils/

Checking for           DBD-mysql (v4.00)   ok: found v4.005
Checking for           MySQL (v4.1.2)  ok: found v5.1.30-community

Removing index 'bugs_short_desc_idx' from the bugs table...
Removing index 'longdescs_thetext_idx' from the longdescs table...
Bugzilla now uses the InnoDB storage engine in MySQL for most tables.
Converting tables to InnoDB:
Converting table attach_data... done.
Converting table attachments... done.
Converting table bug_group_map... done.
Converting table bug_severity... done.
Converting table bug_status... done.
Converting table bugs... done.
Converting table bugs_activity... done.
Converting table bz_schema... done.
Converting table category_group_map... done.
Converting table cc... done.
Converting table classifications... done.
Converting table components... done.
Converting table dependencies... done.
Converting table duplicates... done.
Converting table email_setting... done.
Converting table fielddefs... done.
Converting table flagexclusions... done.
Converting table flaginclusions... done.
Converting table flags... done.
Converting table flagtypes... done.
Converting table group_control_map... done.
Converting table group_group_map... done.
Converting table groups... done.
Converting table keyworddefs... done.
Converting table keywords... done.
Converting table logincookies... done.
Converting table longdescs... done.
Converting table milestones... done.
Converting table namedqueries... done.
Converting table op_sys... done.
Converting table priority... done.
Converting table products... done.
Converting table profile_setting... done.
Converting table profiles... done.
Converting table profiles_activity... done.
Converting table quips... done.
Converting table rep_platform... done.
Converting table resolution... done.
Converting table series... done.
Converting table series_categories... done.
Converting table series_data... done.
Converting table setting... done.
Converting table setting_value... done.
Converting table tokens... done.
Converting table user_group_map... done.
Converting table versions... done.
Converting table votes... done.
Converting table watch... done.
Converting table whine_events... done.
Converting table whine_queries... done.
Converting table whine_schedules... done.
Adding new table bugs_fulltext ...
Adding new table component_cc ...
Adding new table namedqueries_link_in_footer ...
Adding new table namedquery_group_map ...
Adding new table status_workflow ...
Creating ./data directory...
Creating ./data/attachments directory...
Creating ./data/duplicates directory...
Creating ./data/mining directory...
Creating ./data/webdot directory...
Creating ./skins/contrib/Dusk/yui directory...
Creating graphs directory...
Creating skins/custom directory...
Creating ./skins/contrib/Dusk/IE-fixes.css...
Creating ./skins/contrib/Dusk/admin.css...
Creating ./skins/contrib/Dusk/create_attachment.css...
Creating ./skins/contrib/Dusk/dependency-tree.css...
Creating ./skins/contrib/Dusk/duplicates.css...
Creating ./skins/contrib/Dusk/editusers.css...
Creating ./skins/contrib/Dusk/help.css...
Creating ./skins/contrib/Dusk/index.css...
Creating ./skins/contrib/Dusk/panel.css...
Creating ./skins/contrib/Dusk/params.css...
Creating ./skins/contrib/Dusk/release-notes.css...
Creating ./skins/contrib/Dusk/show_bug.css...
Creating ./skins/contrib/Dusk/show_multiple.css...
Creating ./skins/contrib/Dusk/summarize-time.css...
Creating ./skins/contrib/Dusk/voting.css...
Creating ./skins/contrib/Dusk/yui/calendar.css...
Creating ./Bugzilla/.htaccess...
Creating ./data/.htaccess...
Creating ./data/attachments/.htaccess...
Creating ./data/webdot/.htaccess...
Creating ./lib/.htaccess...
Creating ./template/.htaccess...
Creating .htaccess...

Bugzilla requires an SMTP server to function on Windows.
Please enter your SMTP server's hostname: smtp
Precompiling templates...done.
Adding new column 'type' to the 'fielddefs' table...
Adding new column 'custom' to the 'fielddefs' table...
Adding new column 'enter_bug' to the 'fielddefs' table...
Renaming column 'fielddefs.fieldid' to 'fielddefs.id'...
Updating column creator in table series ...
Old: mediumint NOT NULL
New: mediumint
Adding new column 'comment_id' to the 'longdescs' table...
Updating column id in table flags ...
Old: mediumint NOT NULL PRIMARY KEY
New: mediumint auto_increment NOT NULL PRIMARY KEY
Deleting the 'is_active' column from the 'flags' table...
Updating column short_desc in table bugs ...
Old: mediumtext NOT NULL
New: varchar(255) NOT NULL
Adding new column 'id' to the 'namedqueries' table...
Deleting the 'linkinfooter' column from the 'namedqueries' table...
Adding new column 'sortkey' to the 'classifications' table...
Adding new column 'disable_mail' to the 'profiles' table...
Updating column milestoneurl in table products ...
Old: tinytext NOT NULL
New: tinytext DEFAULT '' NOT NULL
Updating column disallownew in table products ...
Old: tinyint NOT NULL
New: tinyint DEFAULT 0 NOT NULL
Updating column votesperuser in table products ...
Old: smallint NOT NULL
New: smallint DEFAULT 0 NOT NULL
Updating column votestoconfirm in table products ...
Old: smallint NOT NULL
New: smallint DEFAULT 0 NOT NULL
Deleting the 'refreshed_when' column from the 'profiles' table...
Deleting the 'last_changed' column from the 'groups' table...
Updating column id in table flagtypes ...
Old: smallint NOT NULL PRIMARY KEY
New: smallint auto_increment NOT NULL PRIMARY KEY
Updating column id in table keyworddefs ...
Old: smallint NOT NULL PRIMARY KEY
New: smallint auto_increment NOT NULL PRIMARY KEY
Updating column userid in table tokens ...
Old: mediumint NOT NULL
New: mediumint
Updating column disabledtext in table profiles ...
Old: mediumtext NOT NULL
New: mediumtext DEFAULT '' NOT NULL
Updating column realname in table profiles ...
Old: varchar(255)
New: varchar(255) DEFAULT '' NOT NULL
Removing index 'longdescs_who_idx' from the longdescs table...
Adding new index 'longdescs_who_idx' to the longdescs table ...
Adding new column 'subclass' to the 'setting' table...
Updating column thetext in table longdescs ...
Old: mediumtext
New: mediumtext NOT NULL
Adding new column 'editcomponents' to the 'group_control_map' table...
Adding new column 'editbugs' to the 'group_control_map' table...
Adding new column 'canconfirm' to the 'group_control_map' table...
Adding new column 'type' to the 'longdescs' table...
Adding new column 'extra_data' to the 'longdescs' table...
Adding new column 'id' to the 'versions' table...
Adding new column 'id' to the 'milestones' table...
Adding new column 'is_open' to the 'bug_status' table...
Marking closed bug statuses as such...
Now filling the 'status_workflow' table with valid bug status transitions...
Adding new column 'icon_url' to the 'groups' table...
Updating column bug_file_loc in table bugs ...
Old: TEXT
New: mediumtext
Updating column description in table attachments ...
Old: mediumtext NOT NULL
New: tinytext NOT NULL
Updating column mimetype in table attachments ...
Old: mediumtext NOT NULL
New: tinytext NOT NULL
Updating column description in table flagtypes ...
Old: TEXT
New: mediumtext NOT NULL
Updating column description in table fielddefs ...
Old: mediumtext NOT NULL
New: tinytext NOT NULL
Updating column description in table groups ...
Old: TEXT NOT NULL
New: mediumtext NOT NULL
Updating column quip in table quips ...
Old: TEXT NOT NULL
New: mediumtext NOT NULL
Updating column query in table namedqueries ...
Old: mediumtext NOT NULL
New: mediumtext NOT NULL
Adding new column 'modification_time' to the 'attachments' table...
Setting the modification time for attachments...
Updating column modification_time in table attachments ...
Old: datetime
New: datetime NOT NULL
Adding new index 'attachments_modification_time_idx' to the attachments table ...
Updating column query_type in table namedqueries ...
Old: tinyint NOT NULL
New: tinyint DEFAULT 0 NOT NULL
Populating bugs_fulltext... (this can take a long time.)
Adding foreign key: attachments.submitter_id -> profiles.userid...
Adding foreign key: bugs_activity.who -> profiles.userid...
Adding foreign key: bugs_fulltext.bug_id -> bugs.bug_id...
Adding foreign key: cc.who -> profiles.userid...
Adding foreign key: component_cc.user_id -> profiles.userid...
Adding foreign key: components.initialowner -> profiles.userid...
Adding foreign key: components.initialqacontact -> profiles.userid...
Adding foreign key: email_setting.user_id -> profiles.userid...
Adding foreign key: logincookies.userid -> profiles.userid...
Adding foreign key: namedqueries.userid -> profiles.userid...
Adding foreign key: namedqueries_link_in_footer.namedquery_id -> namedqueries.id...
Adding foreign key: namedqueries_link_in_footer.user_id -> profiles.userid...
Adding foreign key: profile_setting.user_id -> profiles.userid...
Adding foreign key: profiles_activity.userid -> profiles.userid...
Adding foreign key: profiles_activity.who -> profiles.userid...
Adding foreign key: profiles_activity.fieldid -> fielddefs.id...
Adding foreign key: tokens.userid -> profiles.userid...
Adding foreign key: votes.who -> profiles.userid...
Adding foreign key: watch.watcher -> profiles.userid...
Adding foreign key: watch.watched -> profiles.userid...
Adding foreign key: whine_events.owner_userid -> profiles.userid...
Adding foreign key: whine_queries.eventid -> whine_events.id...
Adding foreign key: whine_schedules.eventid -> whine_events.id...
Adding a new user setting called 'skin'
Adding a new user setting called 'quote_replies'
Adding a new user setting called 'lang'
Adding a new user setting called 'zoom_textareas'
Adding a new user setting called 'state_addselfcc'

Now that you have installed Bugzilla, you should visit the
'Parameters' page (linked in the footer of the Administrator
account) to ensure it is set up as you wish - this includes
setting the 'urlbase' option to the correct URL.

Comment 17

[Dan]

In case it's not perfectly clear from my previous postings.  If I follow the exact same steps using the 3.0.6 release it doesn't matter what format the template is in it always works fine.  Something in 3.2 has changed that doesn't handle DOS formatted templates anymore.

Comment 18

[Marc Schumann [:Wurblzap]]

Looking through http://bonsai.mozilla.org/cvslog.cgi?file=mozilla/webtools/bugzilla/Bugzilla/Template.pm, it might be that we're talking about bug 413121. What happens if back out its patch? (Besides from re-introducing the bug it fixes.)

Comment 19

[Vitaly Fedrushkov]

Attachment: Message sample

Confirmed CVS tip, ActivePerl 5.10 and Test mailer.

Note how CR (0x0D) chars are escaped anywhere except MIME headers.  Note double CRs where

[% IF Param('useclassification') %]

was parsed.

Comment 20

[Max Kanat-Alexander]

What's the difference here between Windows and Linux? If we save the template with DOS newlines on Linux, does the same thing happen?

Comment 21

[Dan]

My Linux VM is messed up right now, but I just completed an install of on OS X 10.5 and the problem is seen there as well.  DOS formatted is broken, but UNIX format works fine.  I would assume the same is the case for Linux.

Comment 22

[Dan]

My Linux VM is messed up right now, but I just completed a clean install of Bugzilla 3.2 on OS X 10.5  (no upgrade) and the problem is seen there as well.  I would assume the same is the case for Linux.

Comment 23

[Max Kanat-Alexander]

Well, in that case this is probably something we should look at and get fixed pretty soon. It's not the *most* important bug we have (since most people won't have DOS newlines in that file), but it's clearly something we're doing (or more likely, a MIME module is doing) new in 3.2 that should be fixed.

Comment 24

[Yann]

Confirmed with w2k3, activestate perl 5.10 (latest version), bugzilla 3.2.

To fix, I took the 3.0 template, and modified manually to look like 3.2 one.

Comment 25

[Thomas Stegh]

Confirmed for Windows Server 2003, mysql-5.1.30-win32, ActivePerl 5.8.8.824 for Windows (x86), Bugzilla 3.2 (from CVS).

To fix, I took the newchangedmail.txt.u8-unix.tmpl from the attachment https://bugzilla.mozilla.org/attachment.cgi?id=351552 (see comment #c12)

Comment 26

[Vitaly Fedrushkov]

Bug 464992 comment 41: related bug can be remedied by upgrading Perl to 5.10.0.

Comment 28

[Yann]

(In reply to comment #26)
> Bug 464992 comment 41: related bug can be remedied by upgrading Perl to 5.10.0.

I already run perl 5.10 (activeperl build 1004), and this bug occurs on that version.

Comment 29

[Frédéric Buclin]

mkanat, this is the last bug blocking 3.2.1. Is this a hard blocker or can we postpone it?

Comment 30

[Max Kanat-Alexander]

(In reply to comment #29)
> mkanat, this is the last bug blocking 3.2.1. Is this a hard blocker or can we
> postpone it?

  Due to the security fixes that we want to release in 3.2.1, and given that there is no patch here, yes, we can postpone it. (But I want a blocking3.2.2 flag for it.)

Comment 31

[Frédéric Buclin]

I just played with Bugzilla 3.3.3 on Windows 2000, and bugmail looks good, but emails you get for new accounts is clearly wrong. Almost everything is on a single line.

Comment 32

[Max Kanat-Alexander]

I'm working on this. Apparently what's happening is that one of the lines has two \r's in it, and Email::MIME->new thinks that thus the first header ends there. Not quite sure why.

Comment 33

[Max Kanat-Alexander]

Attachment: What Email::MIME gets and how it parses it

So, here's a file containing the same email twice.

The first part is the exact input that Email::MIME is getting on Windows. The second part is the output of $email->as_string after parsing the above input.

rjbs--Is this something that we have to fix in Bugzilla (somehow making sure that headers don't have two \r's in them in a row) or that needs to be fixed in the Email:: modules?

Comment 34

[Max Kanat-Alexander]

Comment on attachment 360937 [details]
What Email::MIME gets and how it parses it

I'm changing the MIME type to octet-stream so that the newlines come through intact. This file has to be downloaded, it can't just be viewed in the browser (since the browser will do newline conversion).

Comment 35

[Max Kanat-Alexander]

Okay, so, short story is that it seems to be a bug in Template-Toolkit. I just filed this:

  https://rt.cpan.org/Ticket/Display.html?id=43345

I don't know why we weren't hitting it before--maybe it's also a bug in Email::MIME that only happens with the newer versions that Bugzilla 3.2 requires.

Comment 36

[Max Kanat-Alexander]

Attachment: v1

Okay, this fixes it. There are probably more efficient ways to fix it, but this won't really be slow and it's the simplest solution. Also, we're sending bad line endings (in violation of RFC 2822) sometimes, so I fixed that at the same time since it was now convenient to do so.

(Applicable section of RFC is: http://tools.ietf.org/html/rfc2822#section-2.1 )

Comment 37

[Max Kanat-Alexander]

Attachment: v2

Oh, there was a slight bug in the previous patch (didn't check "ref $msg" in one place properly).

Comment 38

[Max Kanat-Alexander]

Attachment: Actual v2

Accidentally uploaded v1 again and called it v2. Here's the right v2.

Comment 39

[Max Kanat-Alexander]

Attachment: v3

glob pointed out that \n can mean something different on different platforms. Also, thanks to some inspiration from the perlport man page, I made it into one regex.

Comment 40

[:glob ✱]

Comment on attachment 362664 [details] [diff] [review]
v3

r=glob

Comment 41

[Frédéric Buclin]

Max, please commit this patch so that we can do some testing on Windows too (so that we just have to cvs update rather than applying the patch manually).

Comment 42

[Max Kanat-Alexander]

tip:

Checking in Bugzilla/Mailer.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Mailer.pm,v  <--  Mailer.pm
new revision: 1.27; previous revision: 1.26
done

3.2:

Checking in Bugzilla/Mailer.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Mailer.pm,v  <--  Mailer.pm
new revision: 1.20.2.4; previous revision: 1.20.2.3
done

Comment 43

[David Lawrence [:dkl]]

Max, ever since we applied this patch, we have been getting reports of emails being sent out with extra ^M's visible in them that were not there before. It shows up most in people who are using text mail clients such as Mutt. Here is an example of an email that a user reported:

----------

In-Reply-To: <[e-mail]>
References: <[e-mail]>
Auto-Submitted: auto-generated
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.58 on 172.16.52.254
Status: RO
Content-Length: 671
Lines: 21

Please do not reply directly to this email. All additional=0D
comments should be made in the comments box of this bug.=0D
=0D
=0D
https://bugzilla.redhat.com/show_bug.cgi?id=3D250412=0D
=0D
=0D
J=C3=A9r=C3=B4me Fenal <[e-mail]> changed:=0D
=0D
           What    |Removed                     |Added=0D
---------------------------------------------------------------------------=
-=0D
                 CC|                            |[e-mail]=0D
=0D
=0D
=0D
=0D
-- =0D
Configure bugmail: https://bugzilla.redhat.com/userprefs.cgi?tab=3Demail=0D
------- You are receiving this mail because: -------=0D
You are watching someone on the CC list of the bug.=

-----------

It seems to only effect emails with are sent through Bugzilla::Mailer::MessageToMTA (which is most) and is related to the recent change made there.

    my $email;
    if (ref $msg) {
        $email = $msg;
    }   
    else {
        # RFC 2822 requires us to have CRLF for our line endings and
        # Email::MIME doesn't do this for us. We use \015 (CR) and \012 (LF)
        # directly because Perl translates "\n" depending on what platform
        # you're running on. See http://perldoc.perl.org/perlport.html#Newlines
        # We check for multiple CRs because of this Template-Toolkit bug:
        # https://rt.cpan.org/Ticket/Display.html?id=43345
        $msg =~ s/(?:\015+)?\012/\015\012/msg;
        $email = new Email::MIME($msg);
    }   

If I make the following change while doing a reproducer, I do not get the extra ^M's:

-        $msg =~ s/(?:\015+)?\012/\015\012/msg;
+        $msg =~ s/(?:\015+)\012/\015\012/msg;

I understand that the RFC requires \015\012 at the end of each line in the message body but for some reason we are getting \015\015\012 in some cases.

Any ideas why this might be happening?

Dave

Comment 44

[Max Kanat-Alexander]

(In reply to comment #43)
> Max, ever since we applied this patch, we have been getting reports of emails
> being sent out with extra ^M's visible in them that were not there before.

  bug 486206