Error about "data/bugzilla-update.xml" being unreadable

RESOLVED FIXED in Bugzilla 3.0

Status

()

--
major
RESOLVED FIXED
12 years ago
12 years ago

People

(Reporter: kbenton, Assigned: LpSolit)

Tracking

2.23
Bugzilla 3.0
Bug Flags:
approval +
approval3.0 +
blocking3.0 +

Details

(Whiteboard: [relnote about the new 'proxy_url' parameter])

Attachments

(1 attachment, 2 obsolete attachments)

(Reporter)

Description

12 years ago
Having created a brand new installation from the CVS tip (from scratch), ran checksetup.pl, fixed the localconfig, then re-ran checksetup.pl, at the main page, I got this error message logging in as admin:

The local XML file './data/bugzilla-update.xml' cannot be read. Please make sure this file exists and has the correct rights set on it.

data/bugzilla-update.xml does not exist (which explains why it can't be read).  I don't know if checksetup.pl or index.cgi should create that file in its absence, but I think one of them should.

Comment 1

12 years ago
No, that's not the problem. Bugzilla::Update doesn't complain if the file's missing. It just downloads it.

What are the permissions on your data directory? Also, what OS are you running on?
Summary: bugzilla-update.xml missing → Error about "data/bugzilla-update.xml" being unreadable
(Reporter)

Comment 2

12 years ago
> No, that's not the problem. Bugzilla::Update doesn't complain if the file's
> missing. It just downloads it.
> 
> What are the permissions on your data directory? Also, what OS are you running
> on?

data is currently 770 httpd httpd and the OS is RHEL3

BTW - running checksetup.pl as root changes ownership of all the files to root - not something I want it to do.
(Assignee)

Comment 3

12 years ago
So what's wrong here? A problem in checksetup.pl? Or specific to Kevin installation, i.e. invalid?
(Reporter)

Comment 4

12 years ago
I haven't looked into this more deeply yet, however, it may be related to the fact that my Bugzilla system is behind a firewall and can't reach the Internet to grab the file unless a proxy is specified.

Comment 5

12 years ago
I have the same problem with Bugzilla 3.0rc1, running on Apache 2.0.55, using mod_cgi on Slackware 10.2.
The permissions on the data directory are 755 for nobody:nobody (Apache user and group), but even with 777 it doesn't work, and the permissions on the Bugzilla directory are also 755 for nobody:nobody.
Touching and wgetting a file as nobody in the data/ dir works ok.
The file doesn't exist here either and creating an empty file with the same name makes Bugzilla say that the file format is invalid and that I should delete it (after deleting it, the same error reapears).

Comment 6

12 years ago
Manually downloading the file from http://updates.bugzilla.org/bugzilla-update.xml and setting the right permissions on it removes the error.
If this problem is real, this is a blocker.  If it's not, it'll go away anyway when we figure that out.

(In reply to comment #1)
> Bugzilla::Update doesn't complain if the file's missing. It just downloads it.

What's it do if it can't download it after it found it missing?  That seems to be what's happening here.
Flags: blocking3.0+
(Assignee)

Comment 8

12 years ago
A user on IRC just reported the same problem. The reason of the failure seems to be that the webserver group doesn't have the proxy ENV variable defined, explaining why Bugzilla was unable to download bugzilla-update.xml, and explaining why he can do it as a "normal" using using wget.
(Assignee)

Comment 9

12 years ago
(In reply to comment #8)
> explaining why he can do it as a "normal" using using wget.

I meant "as a normal user using wget".

Comment 10

12 years ago
I downloaded 3.0rc1 yesterday (2007.04.11), installed it and ran into this bug.  The permissions were correct for root and apache to read/write into the data directory.  Setting the "upgrade_notifications" parameter to "disabled" at least gets rid of the error message.  

This was on RedHat Fedora 6 with Apache 2.2.

I'll use wget to retreive the http://updates.bugzilla.org/bugzilla-update.xml file manually some time later today.
OK, this sounds like a logical explanation.  Are the users that are hitting this all behind proxy servers?  Maybe we need a configuration option to specify a proxy to use for checking for updates.  No matter what we do, it's at least clear that we need better error handling for the cases when the file can't be downloaded.
(Assignee)

Comment 12

12 years ago
Created attachment 261596 [details] [diff] [review]
hack (that's not an official fix)

Kevin and other users affected by this bug, could you try this patch, and report in this bug if this fixes the problem or not? Kevin, the request for review is only to tell me if this fixes the problem for you. This is not what will be checked in, see below.

Be sure to first remove data/bugzilla-update.xml to see if it's correctly created again. All you have to do is to edit the |my $proxy_url = "";| line and replace it with the correct URL (and port) to your proxy.

If this patch fixes the problem you have, I will implement a real 'proxy_url' parameter which will be accessible from editparams.cgi, and I will also update the error message telling you to set this parameter correctly if Bugzilla is unable to download bugzilla-update.xml.
Attachment #261596 - Flags: review?(kevin.benton)
(Assignee)

Comment 13

12 years ago
Comment on attachment 261596 [details] [diff] [review]
hack (that's not an official fix)

>+        $ua->proxy(['http', 'ftp'], $proxy_url);

Oops! I mean s/ftp/https/ of course.

Comment 14

12 years ago
My server running Bugzilla is not behind a proxy server. It is directly connected to the internet and also acts as a router/gateway for my home network.

Apache is configured for both the external (internet) and the internal network, I'm having the problem when accessing it from both the internal and external net (and when somebody else, with an other internet connection, connects to the external IP). There's a firewall running on the server, but disabling the firewall doesn't help.

And I can download (with wget) the file in Bugzilla's data dir as the user which Apache runs as.

Comment 15

12 years ago
(In reply to comment #14)
Peter: Could you give us information on what the permissions are on your data directory, what they are on bugzilla-update.xml, and what *group* and user your Apache runs as?
(Assignee)

Comment 16

12 years ago
Delete data/bugzilla-update.xml if it exists and then run this command in a shell from your bugzilla/ directory:

perl -MBugzilla -MBugzilla::Update -we 'Bugzilla::Update::get_notifications();'

Do you get any error? Has bugzilla-update.xml been correctly downloaded? Is the LWP::UserAgent module available?

perl -MLWP::UserAgent -we 'print $LWP::UserAgent::VERSION;'

Comment 17

12 years ago
Reply to #15:
The data/ directory:
drwxrwx---   8 nobody nobody  4096 2007-04-15 20:25 data/

bugzilla-update.xml doesn't exist unless I manualy download it with wget, which is part of the problem I think. When I manualy downloaded it, the rights where:
-rw-r--r--   1 nobody nobody  677 2007-04-15 20:00 bugzilla-update.xml

Apache runs as nobody:nobody.


Reply to #16:
Running perl -MBugzilla -MBugzilla::Update -we 'Bugzilla::Update::get_notifications();' outputs nothing to the shell and doesn't download the bugzilla-update.xml file.
LWP::UserAgent isn't available, it won't install here (haven't looked much into the error for that though).
perl -MLWP::UserAgent -we 'print $LWP::UserAgent::VERSION;' returns:
Can't locate LWP/UserAgent.pm in @INC (@INC contains: /usr/lib/perl5/5.8.7/i486-linux /usr/lib/perl5/5.8.7 /usr/lib/perl5/site_perl/5.8.7/i486-linux /usr/lib/perl5/site_perl/5.8.7 /usr/lib/perl5/site_perl .).
BEGIN failed--compilation aborted.

Trying to install LWP::UserAgent gives the following error (but I think that has nothing to do with Bugzilla):
root:/var/www/htdocs/bugzilla# /usr/bin/perl5.8.7 -MCPAN -e 'install LWP::UserAgent'
Can't locate object method "install" via package "LWP::UserAgent" at -e line 1.
Can't locate object method "install" via package "LWP::UserAgent" at -e line 1.

Comment 18

12 years ago
(In reply to comment #17)
> root:/var/www/htdocs/bugzilla# /usr/bin/perl5.8.7 -MCPAN -e 'install
> LWP::UserAgent'
> Can't locate object method "install" via package "LWP::UserAgent" at -e line 1.
> Can't locate object method "install" via package "LWP::UserAgent" at -e line 1.
> 
It only displays the 'Can't locate object method "install" via package "LWP::UserAgent" at -e line 1.' line ones, pasted it twice apparently.
(Assignee)

Comment 19

12 years ago
(In reply to comment #18)
> It only displays the 'Can't locate object method "install" via package
> "LWP::UserAgent" at -e line 1.' line ones, pasted it twice apparently.

OK, so this is not a bug in Bugzilla, at least in your case; you simply don't have this package installed.

Currently, Bugzilla::Update silently gives up if there is a missing package. I will update the error message to mention that there are missing packages and that you should either install them or turn off the notification system.

I still want feedback from other users having this problem to know if their problem is related to the lack of a package, or if it's related to their proxy.
Assignee: installation → LpSolit
OS: Linux → All
Hardware: Sun → All
(Assignee)

Comment 20

12 years ago
Created attachment 261607 [details] [diff] [review]
patch, v1 (better error messages)

This patch improves error messages. No each step of the process has its own error message. I tested them all.
Attachment #261607 - Flags: review?(mkanat)
(Assignee)

Comment 21

12 years ago
Peter, note that LWP::UserAgent is in libwww-perl.
Status: NEW → ASSIGNED

Comment 22

12 years ago
(In reply to comment #21)
> Peter, note that LWP::UserAgent is in libwww-perl.
> 

I already have it fixed, but thanks anyway. Installing it from the CPAN shell worked. I thought I already had libwww-perl installed separately, but I don't know for sure.

And installing the LWP::UserAgent module did indeed fix the problem.

Comment 23

12 years ago
For me, have the LWP package installed... Typing "perl -MLWP::UserAgent -we 'print $LWP::UserAgent::VERSION;'" gives me "2.033".

I need to talk to our IT department to find out what our firewall settings are and what proxies I can use.

Comment 24

12 years ago
Comment on attachment 261607 [details] [diff] [review]
patch, v1 (better error messages)

>-    return if $@;
>+    return {'error' => 'missing_package', 'package' => 'XML::Twig'} if $@;

  If this package isn't installed, we should just never do the check, even if the parameter is set. That's how Bugzilla works for all of the other optional-package things.

>+    [% ELSIF release.error == "cannot_download" %]
>+      <p>The local XML file '[% release.xml_file FILTER html %]' cannot be created.
>+      Please make sure the web server can write in this directory.</p>

  This is great. :-)
Attachment #261607 - Flags: review?(mkanat) → review-
(Assignee)

Comment 25

12 years ago
(In reply to comment #24)
>   If this package isn't installed, we should just never do the check, even if
> the parameter is set. That's how Bugzilla works for all of the other
> optional-package things.

Max, a missing package is the exact reason why Peter was reporting this problem in this bug. If we silently ignore this error, and the admin expects Bugzilla to notify him about new releases (and especially security releases), he will never know he has a missing package and will never get any notification. This seems much more critical than not being able to use Interdiff or save BMP images as PNG.

Moreover, the message from checksetup.pl about XML::Twig mentions that it's required to import bugs using importxml.pl, so an admin would not know that this package is required to get update notifications too.

If you really want to omit this warning, then we would have to disable the update_notification by default (which is currently set to latest_stable_release) and only let an admin enable it if XML::Twig is available. Is that really what we want?

Comment 26

12 years ago
Okay--then we should fix checksetup.pl to note that XML::Twig is also required.

We should not error out just because a feature isn't enabled. The upgrade notifications are set to enabled by default, and we don't want to modify that parameter's default. Instead, if the packages aren't installed, the parameter shouldn't appear, and the feature should just not be there.
(Assignee)

Comment 27

12 years ago
(In reply to comment #26)
> parameter's default. Instead, if the packages aren't installed, the parameter
> shouldn't appear, and the feature should just not be there.

The parameter should always appear in editparams.cgi, but turned off if it cannot detect all required packages. convert_uncompressed_images cannot be turned on if Image::Magick isn't installed. We have to do the same thing here. Hiding a parameter is not the right way to go. We have to let the admin try to turn it on and see the error message explaining that some packages are missing. This way he will understand what's going on.

Comment 28

12 years ago
Yes, but if somebody moves from one server to another, and keeps their data/params but doesn't install the same modules, then the parameter could be set incorrectly.

Also, if somebody installs the required modules, then the parameter is disabled, that makes this particular feature a lot less useful, because they have to *discover* the disabled parameter and enable it, instead of it just "magically working."

Having it fail silently and then "just work" when the module is installed is the least surprising and probably most usable option, I think.
(Reporter)

Comment 29

12 years ago
Comment on attachment 261596 [details] [diff] [review]
hack (that's not an official fix)

That worked.  Nit: I would hope that the environment would override data/params, but I'll leave that up to you to determine how to deal with it.
Attachment #261596 - Flags: review?(kevin.benton) → review+
(Assignee)

Comment 30

12 years ago
Created attachment 261882 [details] [diff] [review]
patch, v2

Combined both patches into a single one, with a new parameter proxy_url to replace the hack of the first patch.
Attachment #261596 - Attachment is obsolete: true
Attachment #261607 - Attachment is obsolete: true
Attachment #261882 - Flags: review?(mkanat)
Attachment #261882 - Flags: review?(kevin.benton)
(Reporter)

Comment 31

12 years ago
Comment on attachment 261882 [details] [diff] [review]
patch, v2

Works as expected.  URL format is required (http://hostname:port) though the trailing / is optional.  Ran tests against Bugzilla 3.0rc1 CVS checkout.  Param was created and UI updated without incident.
Attachment #261882 - Flags: review?(kevin.benton) → review+

Comment 32

12 years ago
Comment on attachment 261882 [details] [diff] [review]
patch, v2

Make sure you pecify *which* environment variable you're talking about, or just not mention anything about environment variables at all.
Attachment #261882 - Flags: review?(mkanat) → review+
(Assignee)

Comment 33

12 years ago
My locally updated patch mentions HTTP_PROXY explicitly.
Flags: approval3.0+
Flags: approval+
(Assignee)

Comment 34

12 years ago
tip:

Checking in Bugzilla/Update.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Update.pm,v  <--  Update.pm
new revision: 1.7; previous revision: 1.6
done
Checking in Bugzilla/Config/Core.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Config/Core.pm,v  <--  Core.pm
new revision: 1.8; previous revision: 1.7
done
Checking in template/en/default/index.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/index.html.tmpl,v  <--  index.html.tmpl
new revision: 1.36; previous revision: 1.35
done
Checking in template/en/default/admin/params/core.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/admin/params/core.html.tmpl,v  <--  core.html.tmpl
new revision: 1.6; previous revision: 1.5
done


3.0 RC1:

Checking in Bugzilla/Update.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Update.pm,v  <--  Update.pm
new revision: 1.5.2.2; previous revision: 1.5.2.1
done
Checking in Bugzilla/Config/Core.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Config/Core.pm,v  <--  Core.pm
new revision: 1.7.2.1; previous revision: 1.7
done
Checking in template/en/default/index.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/index.html.tmpl,v  <--  index.html.tmpl
new revision: 1.33.2.3; previous revision: 1.33.2.2
done
Checking in template/en/default/admin/params/core.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/admin/params/core.html.tmpl,v  <--  core.html.tmpl
new revision: 1.5.2.1; previous revision: 1.5
done
Status: ASSIGNED → RESOLVED
Last Resolved: 12 years ago
Keywords: relnote
Resolution: --- → FIXED
Whiteboard: [relnote about the new 'proxy_url' parameter]

Comment 35

12 years ago
Added to relnotes in bug 379777.
Keywords: relnote
You need to log in before you can comment on or make changes to this bug.