Closed Bug 350112 Opened 18 years ago Closed 17 years ago

Error about "data/bugzilla-update.xml" being unreadable

Categories

(Bugzilla :: Installation & Upgrading, defect)

2.23
defect
Not set
major

Tracking

()

RESOLVED FIXED
Bugzilla 3.0

People

(Reporter: kbenton, Assigned: LpSolit)

Details

(Whiteboard: [relnote about the new 'proxy_url' parameter])

Attachments

(1 file, 2 obsolete files)

Having created a brand new installation from the CVS tip (from scratch), ran checksetup.pl, fixed the localconfig, then re-ran checksetup.pl, at the main page, I got this error message logging in as admin:

The local XML file './data/bugzilla-update.xml' cannot be read. Please make sure this file exists and has the correct rights set on it.

data/bugzilla-update.xml does not exist (which explains why it can't be read).  I don't know if checksetup.pl or index.cgi should create that file in its absence, but I think one of them should.
No, that's not the problem. Bugzilla::Update doesn't complain if the file's missing. It just downloads it.

What are the permissions on your data directory? Also, what OS are you running on?
Summary: bugzilla-update.xml missing → Error about "data/bugzilla-update.xml" being unreadable
> No, that's not the problem. Bugzilla::Update doesn't complain if the file's
> missing. It just downloads it.
> 
> What are the permissions on your data directory? Also, what OS are you running
> on?

data is currently 770 httpd httpd and the OS is RHEL3

BTW - running checksetup.pl as root changes ownership of all the files to root - not something I want it to do.
So what's wrong here? A problem in checksetup.pl? Or specific to Kevin installation, i.e. invalid?
I haven't looked into this more deeply yet, however, it may be related to the fact that my Bugzilla system is behind a firewall and can't reach the Internet to grab the file unless a proxy is specified.
I have the same problem with Bugzilla 3.0rc1, running on Apache 2.0.55, using mod_cgi on Slackware 10.2.
The permissions on the data directory are 755 for nobody:nobody (Apache user and group), but even with 777 it doesn't work, and the permissions on the Bugzilla directory are also 755 for nobody:nobody.
Touching and wgetting a file as nobody in the data/ dir works ok.
The file doesn't exist here either and creating an empty file with the same name makes Bugzilla say that the file format is invalid and that I should delete it (after deleting it, the same error reapears).
Manually downloading the file from http://updates.bugzilla.org/bugzilla-update.xml and setting the right permissions on it removes the error.
If this problem is real, this is a blocker.  If it's not, it'll go away anyway when we figure that out.

(In reply to comment #1)
> Bugzilla::Update doesn't complain if the file's missing. It just downloads it.

What's it do if it can't download it after it found it missing?  That seems to be what's happening here.
Flags: blocking3.0+
A user on IRC just reported the same problem. The reason of the failure seems to be that the webserver group doesn't have the proxy ENV variable defined, explaining why Bugzilla was unable to download bugzilla-update.xml, and explaining why he can do it as a "normal" using using wget.
(In reply to comment #8)
> explaining why he can do it as a "normal" using using wget.

I meant "as a normal user using wget".
I downloaded 3.0rc1 yesterday (2007.04.11), installed it and ran into this bug.  The permissions were correct for root and apache to read/write into the data directory.  Setting the "upgrade_notifications" parameter to "disabled" at least gets rid of the error message.  

This was on RedHat Fedora 6 with Apache 2.2.

I'll use wget to retreive the http://updates.bugzilla.org/bugzilla-update.xml file manually some time later today.
OK, this sounds like a logical explanation.  Are the users that are hitting this all behind proxy servers?  Maybe we need a configuration option to specify a proxy to use for checking for updates.  No matter what we do, it's at least clear that we need better error handling for the cases when the file can't be downloaded.
Kevin and other users affected by this bug, could you try this patch, and report in this bug if this fixes the problem or not? Kevin, the request for review is only to tell me if this fixes the problem for you. This is not what will be checked in, see below.

Be sure to first remove data/bugzilla-update.xml to see if it's correctly created again. All you have to do is to edit the |my $proxy_url = "";| line and replace it with the correct URL (and port) to your proxy.

If this patch fixes the problem you have, I will implement a real 'proxy_url' parameter which will be accessible from editparams.cgi, and I will also update the error message telling you to set this parameter correctly if Bugzilla is unable to download bugzilla-update.xml.
Attachment #261596 - Flags: review?(kevin.benton)
Comment on attachment 261596 [details] [diff] [review]
hack (that's not an official fix)

>+        $ua->proxy(['http', 'ftp'], $proxy_url);

Oops! I mean s/ftp/https/ of course.
My server running Bugzilla is not behind a proxy server. It is directly connected to the internet and also acts as a router/gateway for my home network.

Apache is configured for both the external (internet) and the internal network, I'm having the problem when accessing it from both the internal and external net (and when somebody else, with an other internet connection, connects to the external IP). There's a firewall running on the server, but disabling the firewall doesn't help.

And I can download (with wget) the file in Bugzilla's data dir as the user which Apache runs as.
(In reply to comment #14)
Peter: Could you give us information on what the permissions are on your data directory, what they are on bugzilla-update.xml, and what *group* and user your Apache runs as?
Delete data/bugzilla-update.xml if it exists and then run this command in a shell from your bugzilla/ directory:

perl -MBugzilla -MBugzilla::Update -we 'Bugzilla::Update::get_notifications();'

Do you get any error? Has bugzilla-update.xml been correctly downloaded? Is the LWP::UserAgent module available?

perl -MLWP::UserAgent -we 'print $LWP::UserAgent::VERSION;'
Reply to #15:
The data/ directory:
drwxrwx---   8 nobody nobody  4096 2007-04-15 20:25 data/

bugzilla-update.xml doesn't exist unless I manualy download it with wget, which is part of the problem I think. When I manualy downloaded it, the rights where:
-rw-r--r--   1 nobody nobody  677 2007-04-15 20:00 bugzilla-update.xml

Apache runs as nobody:nobody.


Reply to #16:
Running perl -MBugzilla -MBugzilla::Update -we 'Bugzilla::Update::get_notifications();' outputs nothing to the shell and doesn't download the bugzilla-update.xml file.
LWP::UserAgent isn't available, it won't install here (haven't looked much into the error for that though).
perl -MLWP::UserAgent -we 'print $LWP::UserAgent::VERSION;' returns:
Can't locate LWP/UserAgent.pm in @INC (@INC contains: /usr/lib/perl5/5.8.7/i486-linux /usr/lib/perl5/5.8.7 /usr/lib/perl5/site_perl/5.8.7/i486-linux /usr/lib/perl5/site_perl/5.8.7 /usr/lib/perl5/site_perl .).
BEGIN failed--compilation aborted.

Trying to install LWP::UserAgent gives the following error (but I think that has nothing to do with Bugzilla):
root:/var/www/htdocs/bugzilla# /usr/bin/perl5.8.7 -MCPAN -e 'install LWP::UserAgent'
Can't locate object method "install" via package "LWP::UserAgent" at -e line 1.
Can't locate object method "install" via package "LWP::UserAgent" at -e line 1.
(In reply to comment #17)
> root:/var/www/htdocs/bugzilla# /usr/bin/perl5.8.7 -MCPAN -e 'install
> LWP::UserAgent'
> Can't locate object method "install" via package "LWP::UserAgent" at -e line 1.
> Can't locate object method "install" via package "LWP::UserAgent" at -e line 1.
> 
It only displays the 'Can't locate object method "install" via package "LWP::UserAgent" at -e line 1.' line ones, pasted it twice apparently.
(In reply to comment #18)
> It only displays the 'Can't locate object method "install" via package
> "LWP::UserAgent" at -e line 1.' line ones, pasted it twice apparently.

OK, so this is not a bug in Bugzilla, at least in your case; you simply don't have this package installed.

Currently, Bugzilla::Update silently gives up if there is a missing package. I will update the error message to mention that there are missing packages and that you should either install them or turn off the notification system.

I still want feedback from other users having this problem to know if their problem is related to the lack of a package, or if it's related to their proxy.
Assignee: installation → LpSolit
OS: Linux → All
Hardware: Sun → All
This patch improves error messages. No each step of the process has its own error message. I tested them all.
Attachment #261607 - Flags: review?(mkanat)
Peter, note that LWP::UserAgent is in libwww-perl.
Status: NEW → ASSIGNED
(In reply to comment #21)
> Peter, note that LWP::UserAgent is in libwww-perl.
> 

I already have it fixed, but thanks anyway. Installing it from the CPAN shell worked. I thought I already had libwww-perl installed separately, but I don't know for sure.

And installing the LWP::UserAgent module did indeed fix the problem.
For me, have the LWP package installed... Typing "perl -MLWP::UserAgent -we 'print $LWP::UserAgent::VERSION;'" gives me "2.033".

I need to talk to our IT department to find out what our firewall settings are and what proxies I can use.
Comment on attachment 261607 [details] [diff] [review]
patch, v1 (better error messages)

>-    return if $@;
>+    return {'error' => 'missing_package', 'package' => 'XML::Twig'} if $@;

  If this package isn't installed, we should just never do the check, even if the parameter is set. That's how Bugzilla works for all of the other optional-package things.

>+    [% ELSIF release.error == "cannot_download" %]
>+      <p>The local XML file '[% release.xml_file FILTER html %]' cannot be created.
>+      Please make sure the web server can write in this directory.</p>

  This is great. :-)
Attachment #261607 - Flags: review?(mkanat) → review-
(In reply to comment #24)
>   If this package isn't installed, we should just never do the check, even if
> the parameter is set. That's how Bugzilla works for all of the other
> optional-package things.

Max, a missing package is the exact reason why Peter was reporting this problem in this bug. If we silently ignore this error, and the admin expects Bugzilla to notify him about new releases (and especially security releases), he will never know he has a missing package and will never get any notification. This seems much more critical than not being able to use Interdiff or save BMP images as PNG.

Moreover, the message from checksetup.pl about XML::Twig mentions that it's required to import bugs using importxml.pl, so an admin would not know that this package is required to get update notifications too.

If you really want to omit this warning, then we would have to disable the update_notification by default (which is currently set to latest_stable_release) and only let an admin enable it if XML::Twig is available. Is that really what we want?
Okay--then we should fix checksetup.pl to note that XML::Twig is also required.

We should not error out just because a feature isn't enabled. The upgrade notifications are set to enabled by default, and we don't want to modify that parameter's default. Instead, if the packages aren't installed, the parameter shouldn't appear, and the feature should just not be there.
(In reply to comment #26)
> parameter's default. Instead, if the packages aren't installed, the parameter
> shouldn't appear, and the feature should just not be there.

The parameter should always appear in editparams.cgi, but turned off if it cannot detect all required packages. convert_uncompressed_images cannot be turned on if Image::Magick isn't installed. We have to do the same thing here. Hiding a parameter is not the right way to go. We have to let the admin try to turn it on and see the error message explaining that some packages are missing. This way he will understand what's going on.
Yes, but if somebody moves from one server to another, and keeps their data/params but doesn't install the same modules, then the parameter could be set incorrectly.

Also, if somebody installs the required modules, then the parameter is disabled, that makes this particular feature a lot less useful, because they have to *discover* the disabled parameter and enable it, instead of it just "magically working."

Having it fail silently and then "just work" when the module is installed is the least surprising and probably most usable option, I think.
Comment on attachment 261596 [details] [diff] [review]
hack (that's not an official fix)

That worked.  Nit: I would hope that the environment would override data/params, but I'll leave that up to you to determine how to deal with it.
Attachment #261596 - Flags: review?(kevin.benton) → review+
Attached patch patch, v2Splinter Review
Combined both patches into a single one, with a new parameter proxy_url to replace the hack of the first patch.
Attachment #261596 - Attachment is obsolete: true
Attachment #261607 - Attachment is obsolete: true
Attachment #261882 - Flags: review?(mkanat)
Attachment #261882 - Flags: review?(kevin.benton)
Comment on attachment 261882 [details] [diff] [review]
patch, v2

Works as expected.  URL format is required (http://hostname:port) though the trailing / is optional.  Ran tests against Bugzilla 3.0rc1 CVS checkout.  Param was created and UI updated without incident.
Attachment #261882 - Flags: review?(kevin.benton) → review+
Comment on attachment 261882 [details] [diff] [review]
patch, v2

Make sure you pecify *which* environment variable you're talking about, or just not mention anything about environment variables at all.
Attachment #261882 - Flags: review?(mkanat) → review+
My locally updated patch mentions HTTP_PROXY explicitly.
Flags: approval3.0+
Flags: approval+
tip:

Checking in Bugzilla/Update.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Update.pm,v  <--  Update.pm
new revision: 1.7; previous revision: 1.6
done
Checking in Bugzilla/Config/Core.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Config/Core.pm,v  <--  Core.pm
new revision: 1.8; previous revision: 1.7
done
Checking in template/en/default/index.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/index.html.tmpl,v  <--  index.html.tmpl
new revision: 1.36; previous revision: 1.35
done
Checking in template/en/default/admin/params/core.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/admin/params/core.html.tmpl,v  <--  core.html.tmpl
new revision: 1.6; previous revision: 1.5
done


3.0 RC1:

Checking in Bugzilla/Update.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Update.pm,v  <--  Update.pm
new revision: 1.5.2.2; previous revision: 1.5.2.1
done
Checking in Bugzilla/Config/Core.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Config/Core.pm,v  <--  Core.pm
new revision: 1.7.2.1; previous revision: 1.7
done
Checking in template/en/default/index.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/index.html.tmpl,v  <--  index.html.tmpl
new revision: 1.33.2.3; previous revision: 1.33.2.2
done
Checking in template/en/default/admin/params/core.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/admin/params/core.html.tmpl,v  <--  core.html.tmpl
new revision: 1.5.2.1; previous revision: 1.5
done
Status: ASSIGNED → RESOLVED
Closed: 17 years ago
Keywords: relnote
Resolution: --- → FIXED
Whiteboard: [relnote about the new 'proxy_url' parameter]
Added to relnotes in bug 379777.
Keywords: relnote
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: