224577 - (webservices) Bugzilla could use a web services interface

Status: RESOLVED FIXED

(Bugzilla :: WebService, enhancement, P1)

Description

[Matthew Mastracci]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5) Gecko/20030916
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5) Gecko/20030916

Having a web services interface to Bugzilla would be a nifty tool to allow
remote users to enter bugs via a client-side, native interface.  The current
method used by tools like Bugxula works, but doesn't provide the interface
stability that a web service would and requires a bit of server-side setup to
generate the .rss files required for use.

Having a native plugin in Visual Studio would be a great addition for those of
us who use it daily - this would make such an addition trivial.

I could take a look at this issue when I have a chance.  


Reproducible: Always

Steps to Reproduce:

Comment 1

[Bishop Clark (LC957)]

(Caveat:  I'm in the kiddie-wading end of the SOAP/ilk pool)

Will a web services component allow me to set a dependency on a bug so that it
will track the status (if permitted by ACL) of a bug on a friendly bz server?  I
want to say

> Bug 913 depends on:  	914  	   			 Show dependency tree
> Bug 913 blocks: 	house.steve.org:2947	   	 Show dependency graph
                  	  ^-- that.

That way, steve knows not to work on his part until my bz says so.  

This has enormous potential for me in several areas:
 - Steve owes me a document.  I need to process a doc and build a proposal for
my buddy in atlanta.  We all use (separate {,versions of}) bz dbs to track our
stuff.  It'd be good if we knew moderately easily (one step;  not transitive)
what was keeping the proposal in Atlanta or the Docs in Vancouver
 - I need to track and refer to bugs in a non-bz server, for that bug z529878/1
blocks my z529878/2 and /3 (bz's 739, 740).  I would like to have it all
available on my 'show dependency graph' so that I know what I can work on next
from the click of a mouse.

So.  Is this interface going to provide enough tools for me to sample the state
of bugs and also refer to offsite bugs, or am I logging another bz? 

(..which I'd like to track on my own bzdb so I can code up my end when it's
available, but that's putting the egg before the chicken)

Thanks.  Sorry for the long post;  I'm banking on the MIRV approach of asking
questions of people much more knowledgeable than I.

Comment 2

[Teemu Mannermaa (:wicked)]

*** Bug 260223 has been marked as a duplicate of this bug. ***

Comment 3

[Michael]

Hi ! 
 
How about trying to develop the minimum requirements for such a protocol 
to enable bugZilla interaction with any application ? 
 
like: 
	connect to the individual bugZilla installation 
	retrieve available projects via HTTP/SOAP 
	support those actions that are likely to be required by most people 
using mozilla directly. 
 
etc. 
 
Personally, I would LOVE to see this added - and I'm extremely confident that 
such a protocol would boost bugZilla even more, simply because ANY web-enabled 
application - regardless of the programming language that was used to create 
it, could make use to provide bugZilla feedback - usually, it's like this 
here: 
I notice a bug in an application, or have an idea for an application - like 
e.g. KDE's kate, kdevelop or whatever ... but I simply lack the time to really 
go KDE's webpage, sign up for a bugZilla account and then look if there was 
already such an idea/bug reported - so finally, I end up NOT reporting my 
suggestion/bug report - simply because it's so tiresome. 
 
IF I now had a very simple dialog within my application to simply report an 
issue/suggestion, I would much rather like to really make use of the feature, 
of course the "dialog" thing would be specific to a GUI application, but 
regardless whether CLI or GUI app: both would need a standard protocol to 
interface with bugZilla directly. 
 
Then it would also be useful for bugZilla admins to be able to set whether 
they 
want to enable anonymous posting of bugs/suggestions via such an interface - 
so that there's no sign up process involved. 
 
Probably, some kind of spam-check would also make sure, to customize how 
many requests can be sent using such an interface per user account/IP within 
a certain amount of time, so that it is not used to spam BG. 
 
Bugs posted via this way, would not even have to necessarily show up using 
the standard view, but could rather have a precondition to be re-viewed before 
acceptance. 
 
So, ultimately it would be up to you guys to specify an open protocol that 
supports most of BG's features, this protocol could then be made as either a 
general library for C/C++ application, or as a perl script, so that shell 
utilities can also utilize such functionality. 
 
So, a KDE application would then simply load the library, specify its name 
and the absolute URL to KDE's bugzilla installation. 
 
Then it fetches all available categories for this projects, as well as the 
options that are enabled and accordingly the library would fill in a standard 
dialog with the values and strings provided by your protocol. 
 
I think this would also help mozilla itself - even though something like this 
could in that case already be implemented using their chrome technology and 
doing some clever interfacing with javascript/http ... 
 
 

Comment 4

[Bishop Clark (LC957)]

CAUTION: I have no idea what I'm talking about.

I think we should cut up this bug a bit, and only set requirements then.  How
about this:

BZ can read  ------------>  Read-only
foreign WSS                 WSS IFace
                           /
                          /
224577   ----->  Read-Write
  \              WSS IFace
   \            /
    BZ can R/W /
    foreign WSS


Reqs:
RO WSS IFace:
 - expose regular db atts to (authed) clients
 - complications
   - has_dependencies() may be nifty
   - other virtual attributes of bugs that stand out as really-cool?

BZ Reads Foreign WSS:
 - allow bugs to depend on foreign bugs
 - launch wss 'hit' on every bug view or by [update] button on bugs?

Read/Write WSS IFace backend work
 - keep this R/W bug separate from the R-O work.
 - simply allow (authed) update to bugs along some (plan this!) format
   or via same options available via current webform
 - may need to implement some cache on remote of current bug state, reduce 
   unnecessary hits (use [update] button above for O-C people), so update 
   remote cache of this->state on state.change() for all remote states, 
   similar to current action where local list of bug dependencies are 
   run through.
 - loop-detection with foreign servers could get interesting.

BZ can read/write to foreign WSS
 - Hmm.  If the caching idea - remote.update() on state.change() to reduce
   server-server traffic to on-change vs on-access - requires BZ to write to
   other WSSs, this bug may be entirely worked by that bug instead, or be
   trivial.


Anyway, that's just my idea as to how to cut it up, and the requirements for
each sub-bit.  It's all in point form, written while I was in a telephone
meeting, and I'm still dipping my toe into the XML-RPC/SOAP/WSS idea : don't
trust I know what I'm doing, and do discuss anything that even smells odd.

But, there's some copy laid down, so the gods of XMLRPC and ComicBookGuys can
critique it/me.

Let me know and I'll make up some **** BZs for each bit.

Comment 5

[Michael]

Hi ! 
 
To be honest, personally I simply lack the necessary (detailed) perl 
knowledge, 
so I don't mind to believe what you suggest ;-)  
however, I am somewhat familiar with SOAP/XML-RPC and I would be able to 
implement 
a basic SOAP service using PHP, of course that's not an option and not really 
relevant 
in this context. 
 
 
Also, I do agree that it is certainly a good idea to split things/tasks up -  
and I would agree that one of the first things would certainly be the ability 
to  
*retrieve* structured data from a CGI that access the DB, otherwise SOAP is an  
interactive protocol - so you're making *requests*, that are responded to by 
the  
server-side CGI. 
 
So, one of the first things would be: 
 
=> send SOAP request to a CGI using an encrypted username/password combination 
 
	=> have the CGI validate the data using the local database 
	=> if the access data is right, the CGI should respond with a listing 
	   of those functions that are available for RPC, respectively with 
those 
	   functions that were enabled by the administrator. 
	    
	=> One of these functions would then be something like 
retrieveAvailableProjects() 
	 
	=> This would return those projects/sub-projects that are available 
for RPC handling 
	 
	=> One would then decide for a specific project, which makes the 
server respond again 
	   with a set of available functions for that specific (sub-)project 
	    
	=> depending on the set of available functions, a library could offer 
the corresponding 
	   frontend, either as a GUI library (GTK, QT ...) or - which is 
probably going to happen 
	   as a start anyway: by using a simple command line interface. 
	    
	=> so the CGI would have to be able to reply with a unified set of 
available user interface 
	   controls, like "text", "text input", "combo box", "buttons" 
	    
	=> That way the client-side library would only serve as a layer to a 
user interface that 
	   could be implemented in pretty much any programming language, no 
matter whether  
	   C++, perl, tcl/tk, python - so the bugZilla installation would 
merely serve as 
	   the logical backend that stores the data and processes the 
requests, while the 
	   exact graphical implementation and presentation of the available 
options would be 
	   left open to the author of the corresponding software. 
	    
	   This would mean that the CGI responds to an imaginary 
"getAvailableProjects()" request 
	   like this: 
	    
	   	<availableProjects> 
			<text>Please choose a (sub-)project:</text> 
			<project>mozilla</project> 
			<project>bugZilla</project> 
		</availableProjects> 
 
	   While a GUI implementation of the frontend would present these as a 
combo box, 
	   a command line frontend would simply echo something like the 
following to the 
	   console: 
	    
	   	Please choose a (sub-)project: 
		 
		[1] mozilla 
		[2] bugZilla 
		 
 
So, you see: I envision a pretty dynamic protocol, so that it is not 
statically 
bound to anything, but rather all relevant details can be exchanged by using 
SOAP, that way pretty much anything would ultimately be up to the 
administrator  
of a project that uses bugZilla for bugtracking. 
 
 
As I mentioned already, I don't really know perl, nor am I particularly 
familiar  
with bugZilla's internals, but I would certainly not mind to come up with more  
ideas and suggestions. 
 
Also, taking into consideration that this request seems somewhat unusual and 
not really all that "common" or popular, I would not mind make some own coding 
attempts, but these would certainly not be "release-ready" but rather have 
"concept" character. 
 
So if you think that this would be helpful, just say so. 
 
In that context it would be interesting, whether there's any detailed 
developer documentation available for bugZilla, so that I can easily look 
up the database tables that are used for a particular purpose, or simply 
look for a listing of the functionality that's so far supported and would 
need to be converted into some kind of protocol. 

Comment 6

[Bishop Clark (LC957)]

So, should we go with a division of the bug like so, for now, and start marking
up ideas into the various sub-parts, or do we need to discuss the division of
this bug some more at all?  I'd rather it was divided further before we start
discussing the actual cutting of components, but that's just my preference, and,
lacking the depth and the time to actually code on it, my preference hasn't much
weight.

It seemed like the division of the parts, and schedule of work, is approved by
at least one person.  Anyone with some Perl && BZ && SOAP experience available
to comment?

Comment 7

[Michael]

> So, should we go with a division of the bug like so, for now, and start 
> marking up ideas into the various sub-parts, or do we need to discuss the 
> division of this bug some more at all?   
 
I don't mind - probably any way would work ... ultimately it will be up to 
the person who really addresses this 'feature' request. 
 
> I'd rather it was divided further before we start 
> discussing the actual cutting of components, but that's just my preference, 
 
I think, it's not really THAT important, an overview and some suggestions were 
provided, so it's pretty obvious now, what we mean ... 
 
> and, lacking the depth and the time to actually code on it,  
> my preference hasn't much weight. 
 
it has at least given a good overview about what we mean, obviously we 
are only a minority - few other people have voted for this "bug", so to 
be honest, it seems unrealistic, that this is going to be addressed anytime 
soon - and even if someone did really care for it, we did so far not receive 
any feedback about what the bugZilla maintainers think about this whole idea. 
 
So, to start actually writing code might very well turn out to be a waste of 
time, if any resulting patch or rather modification is going to be refused. 
 
 
> It seemed like the division of the parts, and schedule of work, is approved 
> by at least one person.   
 
ONE person more, that's not going to be able to address this feature all 
alone. 
And even if I was able: it really doesn't seem to be THAT important to many 
people. 
 
I think, it is going to implemented at some time in the future - simply 
because 
it would ultimately improve bugZilla a lot, but so far not everybody seems 
to realize the potential. 
 
So, personally I can wait ... and simply use GET encoded requests to the CGI 
directly - of course it's a bit tiresome to parse the returned HTML ... 
 
> Anyone with some Perl && BZ && SOAP experience available to comment? 
 
So far I can only offer the latter, but if I should indeed find some simple 
docs about BZ's current database structure, I might consider doing the 
SOAP thing using merely PHP & mySQL. 
 
FOR ME, this would be a lot easier ... 
 
Don't know though, what the developers think about such an extension being 
developed in php ...otherwise it doesn't need to be really tightly integrated 
into BZ itself. 
 

Comment 8

[David Lawrence [:dkl]]

I currently have written a crude XML-RPC interface to Bugzilla that does a few
basic things already. Currently my code can create a new bug, query for bugs,
get single bug information, get product/component description information, and
add a comment to a bug report. It is currently working with 2.17.1 which is what
we are using in house but I am working on porting it to work with 2.18 at the
moment. Also I want to finish out the rest of the other functions such as making
changes to existing bug report, create an attachment, etc. If you want to look
at what I have (unfortunately I dont have a patch yet) you can download our
tarball at http://bugzilla.redhat.com/bugzilla.

Comment 9

[Michael]

Thanks ... it might make sense to consider directly ATTACHING any specific 
files/sources, just in case that this "bug" (feature request) should again 
rest for almost one year. 
 
I would not mind creating diff patches against the original sources -if that 
helps. 
 
Otherwise, if it is really planned to enhance that functionality in the 
redhat BZ port, I would not mind consider switching to redhat's BZ ... 

Comment 10

[Bishop Clark (LC957)]

David can't attach his bz code.  I know for a fact that it's too big. -- bish

Comment 11

[Michael]

Hi ! 
 
Me again, just wanted to say that I've meanwhile had some time to look into BZ  
now and that I'd personally still prefer starting with a PHP-based approach, 
my  
perl skills are certainly not sharp enough to be of much help otherwise ...  
 
But the database structure doesn't seem to be all that complicated, and as a 
start one could begin with simple things: So I would mainly directly 
manipulate  
the tables and wouldn't use any pre-existing perl wrappers :-/ 
 
However, I wonder now whether such an approach is encouraged at all - at  
least, I would like to know whether you'd consider using a php script for  
such a purpose - taking into account that BZ is exclusive perl software. 
 
Thanks 

Comment 12

[Jeff Bailey]

Taking this bug - I've started chatting with David Lawrence and we'll try to
work together on this.

Comment 13

[Jeff Bailey]

Just a status update on this - I'll have some code to present very shortly.

What I've got so far is that I'm using SOAP::LITE for this so that it can be
easily shared with a SOAP implementation.  Most of the code in the bugzilla
snapshot that I'm working from (2.19.2) assumes that it's running in a CGI
environment and that it's free to deal with the environment.  The most annoying
of these is global.pl.

I've moved a few functions from global.pl into Bugzilla::RPC - but the right
solution in the end is to merge these together, either in one of the bugzilla
classes, or to make global.pl not touch the environment in any way (so that I
can use it in this code)

Aside from that, I've got bug retrieval, new bugs being added, and comments
being added.  If anyone wants to see what I've got, I can cheerfully post it, or
actually get around to learning TLA and make my repository public.

Tks,
Jeff Bailey

Comment 14

[Max Kanat-Alexander]

As far as the globals.pl stuff goes, add blockers to bug 87411, called "Move xxx
out of globals.pl" where xxx is the name of each function. Note that you need
them moved out in order for them to interact with Bugzilla::RPC correctly. (I
think it should be Rpc, by the way -- otherwise you get into trouble with
appending acronyms to each other. It's not real important either way, though.)

I'll then get around to moving those functions out much faster than I would
otherwise.

Comment 15

[Max Kanat-Alexander]

Also, do post your code -- that way we can look over it and let you know early
on if there are any major changes that would be necessary for it to work in
Bugzilla.

You should be aware that Bugzilla does run in taint mode, though, so certain
environment variables definitely will not be available.

Also, if you could post it as a patch against the CVS tip, that would be ideal. 
:-) But a patch against 2.19.2 is better than no patch at all.

Comment 16

[Michael]

Hey, glad to hear that someone's actually making a first stab at it, please feel
free to post a diff patch, I'd love to take a look at it.

By the way: now that the ideas seem to keep coming, how about making this whole
thing an official subcomponent within bugZilla, so that it'd be easier to make
new suggestions and feature requests using bugzilla's functionality, instead of
having to post all the details as one single "bug" - taking the length and
details of some posts into account, it's getting a bit chaotic meanwhile, I think.

There were so many different ideas and feature requests mentioned for this
particular extension, I think it might definitely make sense, to consider adding
a particular "bugZilla RPC Service" category, so that things could be split up.

I am not sure whether bugZilla already supports some sort of "concept"
categories/sub-projects, I mean some new subproject category that isn't actually
active yet, but rather "in development" or at least "in design". So, one could
make that sub-category as a "concept category".

That way all these great thoughts would at least be saved in some orderly form
and it would be easier to catch up with the design status of this feature
requests for those people who actually want to take a look at it.

Sounds like a new feature request ;-)

Comment 17

[Max Kanat-Alexander]

(In reply to comment #16)
> I am not sure whether bugZilla already supports some sort of "concept"
> categories/sub-projects, I mean some new subproject category that isn't 
> actually active yet, but rather "in development" or at least "in design". So, 
> one could make that sub-category as a "concept category".

  That's what we use "meta" bugs for, like bug 87411 or bug 278579.

Comment 18

[Jeff Bailey]

Attachment: Initial xmlrpc patch

This is the first round of the xmlrpc bit for bugzilla.  It's about half
implemented - right now it's the half that I need for submitting bugs.

Layout wise, the xmlrpc.cgi is pretty simple, and just chains to the rest of
the functions.	The idea here is that we could easily cook up an wsdl and do
soap with this library instead.  I see that as a big win.

The RPC.pm contains a pile of functions that should really be elsewhere.  I've
tried to be clear with the comments.  Some of it is functions that are in
global.pl that need to be merged in.  Those are duplicated in their entirety,
with the idea that once they live some place that doesn't much with the CGI
environment, we can use them directly.	The last part of the file is functions
I've had to cook together.  The post one is extracted from post_bugs.cgi.  It's
incomplete but good enough to play with.  Adding other bits to it isn't that
hard.

In the RPC directory are the actual RPC functions.  The interface is taken
exactly from RedHat's most recent snapshot.  (Newer than what they have posted
on their website).  I've preserved that for now, since it's been working for
them without too many issues.  I realise that the interface will probably want
a make over, but I don't recommend doing it for 2.2[02].  The only thing that
should probably be added is a way of getting the current interface revision
number so that programs can, in the future, figure out whether or not the
xmlrpc interface is compatible with them.

HTH.  I should be doing more hacking on this soon.  I've gotten a little caught
up in other things, (like getting a useful C xmlrpc client).

Tks,
Jeff Bailey

Comment 19

[Max Kanat-Alexander]

Comment on attachment 177403 [details]
Initial xmlrpc patch

OK, I'mma ask myself for review.

In the future, it's better to just post it as a patch, if at all possible. It
makes review MUCH easier. :-)

Comment 20

[Jeff Bailey]

Attachment: Initial xmlrpc patch.

Per mkanat's request, submit patch as a collection of diffs against /dev/null
rather than a .tar.gz to be unpacked in the tree.

Comment 21

[Max Kanat-Alexander]

Comment on attachment 177563 [details] [diff] [review]
Initial xmlrpc patch.

>--- /dev/null	2005-03-14 07:01:33.250049848 -0500
>+++ ./bugzilla/xmlrpc.cgi	2005-03-14 15:30:30.000000000 -0500
>@@ -0,0 +1,32 @@
>+#!/usr/bin/perl -t

  Of course, we run under -wT, not just -t.

>+XMLRPC::Transport::HTTP::CGI
>+  -> dispatch_with({
>+  	'Bugzilla' => 'Bugzilla::RPC',
>+	})
>+  -> handle
>+;
>+
>+
>--- /dev/null	2005-03-14 07:01:33.250049848 -0500
>+++ ./bugzilla/Bugzilla/RPC.pm	2005-03-14 15:30:44.000000000 -0500

  Let's call it WebService instead of RPC -- I think that's clearer and more
specific.

>@@ -0,0 +1,343 @@
>+
>+# Only list functions here that you want to be available by RPC
> [snip]

  Thinking about it... let's start off with a read-only service, and then add
in the writing/changing functions later. I thin kthat will make it a lot easier
to do the review.

>+
>+# Bug Methods
>+
>+use Bugzilla::RPC::getBug;

  Usually we name functions_like_this... and ModulesLikeThis, so that's the way
I'd like to do it. If you'd like, we could perhaps keep around the old names as
aliases to the new ones, somehow...

>+List of possible fault codes:
>+
>+=over 4
>+
>+=item 5
>+
>+Generic Fault.  Doesn't tell you much.  Fault message should give more
>+information.

  Use constants for these, but also include their numbers (since remote callers
need to know the numbers).

>+package Bugzilla::RPC;

  I think this actually should go at the top, above the POD docs, yes?

>--- /dev/null	2005-03-14 07:01:33.250049848 -0500
>+++ ./bugzilla/Bugzilla/RPC/addAttachment.pm	2005-03-14 15:30:57.000000000 -0500
>@@ -0,0 +1,35 @@
>+#!/usr/bin/perl -t

  For a .pm file, you don't need that line.

>+
>+use strict;
>+
>+our ($userid);

  Nit: You don't really need the parens, there.

>+package Bugzilla::RPC;
>+
>+# Public RPC Function
>+#
>+# addAttachment(id, mimetype, comment, data, isprivate, username, password)
>+# 
>+# Returns: attachment id

  These should probably all be POD doc'ed inside of Bugzilla::RPC (or
Bugzilla::WebService or whatever we call it) to make it easy for users to find
information on the API.

  Also, while I'm thinking about it, we should consider naming and creating the
functions in such a fashion as so they could be used with the same names and
APIs by other bug-tracking systems, even if they say they don't want to now.
Just something to think about.

  And also... why are these functions all in separate .pm files?? They should
probably just all be in one.

>+sub addAttachment {
>+  shift if UNIVERSAL::isa($_[0] => __PACKAGE__);
>+
>+  my ($product, $username, $password) = @_;
>+
>+  my ($authres, $userid, $extra, $info) =
>+    Bugzilla::Auth->authenticate($username, $password);
>+
>+  if ($authres == Bugzilla::Constants::AUTH_LOGINFAILED) {
>+    die SOAP::Fault->faultcode(811)
>+                   ->faultstring('Bad Username or Password');
>+  }


  We should probably encapsulate that Auth stuff in another function.

>+sub addUser {
>+  shift if UNIVERSAL::isa($_[0] => __PACKAGE__);
>+
>+  my ($product, $username, $password) = @_;
>+
>+  my ($authres, $userid, $extra, $info) =
>+    Bugzilla::Auth->authenticate($username, $password);
>+
>+  if ($authres == Bugzilla::Constants::AUTH_LOGINFAILED) {
>+    die SOAP::Fault->faultcode(811)
>+                   ->faultstring('Bad Username or Password: ' . $userid);
>+  }

  You'll also need a fault for a user not having the correct permissions. Same
for a few other places.

>+sub getBug {
>+  shift if UNIVERSAL::isa($_[0] => __PACKAGE__);
>+
>+  my ($bugid, $username, $password) = @_;
>+  
>+  my ($authres, $userid, $extra, $info) =
>+    Bugzilla::Auth->authenticate($username, $password);
>+
>+  if ($authres == Bugzilla::Constants::AUTH_LOGINFAILED) {
>+    $userid = 0;
>+  }
>+
>+  my $bug = new Bugzilla::Bug($bugid, $userid);
>+  
>+  return $bug;

  There's a chance that the internal structure of a bug object will change over
time, but our API shouldn't... perhaps we should be more granular than getBug.

  At this point in the file, my thoughts are that it might be good to pass a
Bugzilla::User object into some of these functions, perhaps, but... I'm not
sure how or if it would be useful.

>+sub getLastBug {

  What's this supposed to do?

>+sub getProductInfo

  Once again, I think we'll need a more specific API.

  These are just some initial comments. Also look at bug 254400 comment 5, and
consider implementing all the read-only functions that Kristis needs (any write
functions that he needs can come later), because he's a definite future
consumer of XML-RPC. :-)

  Also, I'd like to know if you have any ideas on what would make life easier
for bug-buddy, another known consumer.

Comment 22

[Max Kanat-Alexander]

*** Bug 221342 has been marked as a duplicate of this bug. ***

Comment 23

[Jeff Bailey]

An FYI that this isn't dead, I'm just a bit swamped with the breezy release atm.
 Will get this all cleaned up right after.

Comment 24

[Max Kanat-Alexander]

Hey Jeff. Any new version for us yet? :-)

Comment 25

[Mik Kersten]

This would be very helpful for the Eclipse-based Mylar Bugzilla Client, which currently screen scrapes and formulates posts.  Fyi a flash video of the it in action is at: http://eclipse.org/mylar/doc.php 

Comment 26

[Douglas Calvert]

> changes to existing bug report, create an attachment, etc. If you want to look
> at what I have (unfortunately I dont have a patch yet) you can download our
> tarball at http://bugzilla.redhat.com/bugzilla.

Is this still available?

Comment 27

[Olav Vitters]

Comment on attachment 177563 [details] [diff] [review]
Initial xmlrpc patch.

>--- /dev/null	2005-03-14 07:01:33.250049848 -0500
>+++ ./bugzilla/Bugzilla/RPC/addAttachment.pm	2005-03-14 15:30:57.000000000 -0500
[..]
>+
>+  my ($authres, $userid, $extra, $info) =
>+    Bugzilla::Auth->authenticate($username, $password);

Should be:
  my ($authmethod, $authres, $userid, $extra, $info) =
    Bugzilla::Auth->authenticate($username, $password);

This is wrong in all the Bugzilla/RPC/*pm files.


I think that maybe using basic http authentication is perhaps easier/cleaner. See Users Guide on http://soaplite.com/ for some thoughts on authentication. It also describes how to implement access control.

Comment 28

[Igor Foox]

Hi Jeff,

I was wondering if there is any progress on this patch? If there isn't, how far along is it? I'm very interested in seeing this go through, so if there's some particular areas where it's necessary I could try and help out. Please let me know if there's anything I can do.

Comment 29

[Mik Kersten]

I would like to emphasize the importance of having a web service interface for Bugzilla.  Mylar (http://eclipse.org/mylar) now provides an Eclipse-based UI for JIRA in addition to Bugzilla, and that's all happening via the JIRA SOAP interface whic insulates us from implementation details.  Other issue trackers (e.g. Rally) are now starting to provide similar APIs.

1500 people installed/updated the 0.4.8 release of Mylar support for Bugzilla, so there is a rapidly growing user community of people who use Bugzilla through an Eclipse-integrated UI.  We currently have far better support for Bugzilla than for JIRA (e.g. offline reports), but the effort of maintaining code and tests that accurately do screen scraping on versions 2.16 through 2.2x is high, and without a WS interface on the horizon I worry that Bugzilla support will eventually fall behind, even though we are doing what we can to foucs our efforts there.

What are the plans and/or ETA for this support?

Comment 30

[Max Kanat-Alexander]

(In reply to comment #29)
> What are the plans and/or ETA for this support?

  We'd like to see it in our next release (which will probably be Bugzilla 3.0), but it doesn't currently have an assignee who is working on it. If nobody else gets to it, I will. But I have a lot of things to do on the roadmap before that.

Comment 31

[Max Kanat-Alexander]

I will be working on this, as I have a client who wants it within the year.

Comment 32

[Eugene Kuleshov]

(In reply to comment #31)
> I will be working on this, as I have a client who wants it within the year.

AFAIK, Red Heat already done some work to expose XMLRPC for Bugzilla.

Comment 33

[Max Kanat-Alexander]

(In reply to comment #32)
> AFAIK, Red Hat already done some work to expose XMLRPC for Bugzilla.

  Yes. If you read the rest of the bug, you'll notice that some of the current patch is based on Red Hat's work.

Comment 34

[David Lawrence [:dkl]]

I have been meaning to port my changes to HEAD but have found little time as of late due to other projects to get to it. I am hoping within the next week or so to have something for all to look at. Until then, I could send Max a tarball of our current 2.18 code that has the pieces integrated.

Comment 35

[Marc Schumann [:Wurblzap]]

Attachment: Patch

Taking a little stab at this. I'm ready to take this bug if we're talking about a scope like the limited functionality given in my patch.

Comment 36

[Marc Schumann [:Wurblzap]]

Attachment: Demo script

Here's a demo and testing script to go along with the patch.

Comment 37

[Mik Kersten]

Following up on my comment#29, we have been able to remove all of the Mylar Bugzilla Connector's HTML parsing and support Bugzilla 2.18 and later by relying entirely on the XML bug report format and the query and configuration RDF format (dropping 2.16 support allowed us to do this).

So what we need now is to run the following operations via SOAP/XMLRPC (currently mocking up a form POST):
1) retrieve bug contents***
2) retrieve attachments
3) retrieve all operations possible for this user on a bug (e.g. ability to reassign)
4) submit changes to bug, including attachments
5) retrieve configuration information (status and priority options, products
etc.)

Marc, which of these does your patch support?

*** A related question: with the conversion to RDF we lost the ability to get real names (in addition to email addresses).  Is there any way we could get these beyond mocking up a query for a bug that person is included on and caching names them on our end?  

Comment 38

[Marc Schumann [:Wurblzap]]

(In reply to comment #37)
> Marc, which of these does your patch support?

The first one only: there is a method returning a bug object. Plus, you can log in and out.

The patch is designed for extensibility. I think further interfaces should be added in patches on separate bugs.

Comment 39

[Vlad Dascalu]

Comment on attachment 220807 [details] [diff] [review]
Patch

Sadly, the recent auth rewrite made this bitrotten.

+++ ./Bugzilla/Auth/Login/WWW/CGI.pm	2006-05-04 19:54:15.039731200 +0200

This file no longer exists on trunk.

I can't apply the patch for testing.

Comment 40

[Marc Schumann [:Wurblzap]]

Attachment: Patch 1.0.1

Unrotted.

Comment 41

[Vlad Dascalu]

Comment on attachment 225338 [details] [diff] [review]
Patch 1.0.1

The test suite was failing on the trunk with the patch applied, because checkesetup.pl didn't report the newly introduced Perl module dependency that I was missing.

So the patch needs to modify checksetup.pl as well, and let the user know that he needs SOAP-Lite or what's its name in order to do the WebService thing.

Also, the PPM hash at the beginning of checksetup.pl needs an entry as well for the Windows PPM name of the module (usually replacing "::" with "-" does the trick).

+    if (Bugzilla->usage_mode == Bugzilla::Constants::USAGE_MODE_WEBSERVICE) {
+        die SOAP::Fault
+            ->faultcode(ERROR_AUTH_NODATA)
+            ->faultstring('Login required');
+    }

Unless we are in batch mode, we no longer use die to indicate errors due to security-related installation path disclosure (see bug 340253 ). Batch was acceptable since it implied local usage, but SOAP is intrinsecally a remote thing.


Nit: a paragraph in the XML docs mentioning xmlrpc.cgi as our SOAP proxy would be nice.


The attached script doesn't display the help information as it advertises:

[vladd@abacron-l Bugzilla-trunk]$ ../../bz_webservice_demo.pl
Error: URI unspecified
Usage:
    "bz_webservice_demo.pl [options]"

    "bz_webservice_demo.pl --help" for detailed help

[vladd@abacron-l Bugzilla-trunk]$ ../../bz_webservice_demo.pl --help
[vladd@abacron-l Bugzilla-trunk]$                                         

so I didn't get to test much stuff.

Comment 42

[Michael]

> Also, while I'm thinking about it, we should consider naming 
> and creating the functions in such a fashion as so they could 
> be used with the same names and APIs by other bug-tracking 
> systems, even if they say they don't want to now.
> Just something to think about.

FYI, the following is manti's approach: http://www.futureware.biz/mantisconnect/

Comment 43

[Max Kanat-Alexander]

*** Bug 342075 has been marked as a duplicate of this bug. ***

Comment 44

[Marc Schumann [:Wurblzap]]

Attachment: Patch 1.1

(In reply to comment #41)
> So the patch needs to modify checksetup.pl as well, and let the user know that
> he needs SOAP-Lite or what's its name in order to do the WebService thing.

Fixed.

> +    if (Bugzilla->usage_mode == Bugzilla::Constants::USAGE_MODE_WEBSERVICE) {
> +        die SOAP::Fault
> +            ->faultcode(ERROR_AUTH_NODATA)
> +            ->faultstring('Login required');
> +    }
> 
> Unless we are in batch mode, we no longer use die to indicate errors due to
> security-related installation path disclosure (see bug 340253 ). Batch was
> acceptable since it implied local usage, but SOAP is intrinsecally a remote
> thing.

Using "die SOAP::Fault" is the correct way to report errors when using SOAP::Lite. The surrounding handle() function does everything for us: it uses an eval, catches the die() and wraps it up in a correct way for the XMLRPC::Transport in use so that the caller receives the error in a nice way. It's something like a remote die(). Very neat indeed.

> Nit: a paragraph in the XML docs mentioning xmlrpc.cgi as our SOAP proxy would
> be nice.

I agree, but I'd like to keep it out of this checkin.

> The attached script doesn't display the help information as it advertises:

This works for me. Apparently the pod2usage call the script does fails for you. Maybe we can fix this together. Please refer to the embedded POD for now.

Comment 45

[Vlad Dascalu]

Thanks! I'll look at this tomorrow as I have a deadline today that I must take care of first.

Comment 46

[Vlad Dascalu]

Comment on attachment 226294 [details] [diff] [review]
Patch 1.1

Looks like we're passing "undef" to the Bugzilla::Bug constructor, with this patch applied. Which causes a crash.

In order to reproduce it:

-> turn aliases on in your Bugzilla installation.
-> modify the testing script so it allows strings for bug IDs ('bug_id:s' in the GetOptions thing).

Then I get:

[vladd@abacron-l Bugzilla-trunk]$ ../../bz_webservice_demo.pl --uri=http://localhost/Bugzilla-trunk/xmlrpc.cgi --bug_id=test

Server Undef to trick_taint at Bugzilla/Util.pm line 65
        Bugzilla::Util::trick_taint('undef') called at Bugzilla/Bug.pm line 706
        Bugzilla::Bug::bug_alias_to_id('undef') called at Bugzilla/Bug.pm line 106
        Bugzilla::Bug::initBug('Bugzilla::Bug=HASH(0x99681c4)', 'undef', 0) called at Bugzilla/Bug.pm line 83
        Bugzilla::Bug::new('Bugzilla::Bug', 'undef', 0) called at Bugzilla/WebService/Bug.pm line 29
        Bugzilla::WebService::Bug::get_bug('Bugzilla::WebService::Bug', 'test') called at /usr/lib/perl5/site_perl/5.8.8/SOAP/Lite.pm line 2322
        eval {...} called at /usr/lib/perl5/site_perl/5.8.8/SOAP/Lite.pm line 2310
        eval {...} called at /usr/lib/perl5/site_perl/5.8.8/SOAP/Lite.pm line 2282
        SOAP::Server::handle('Bugzilla::WebService::XMLRPC::Transport::HTTP::CGI=HASH(0x975...', '<?xml version="1.0" encoding="UTF-8"?><methodCall><methodName...') called at /usr/lib/perl5/site_perl/5.8.8/SOAP/Transport/HTTP.pm line 286
        SOAP::Transport::HTTP::Server::handle('Bugzilla::WebService::XMLRPC::Transport::HTTP::CGI=HASH(0x975...') called at /usr/lib/perl5/site_perl/5.8.8/SOAP/Transport/HTTP.pm line 369
        SOAP::Transport::HTTP::CGI::handle('Bugzilla::WebService::XMLRPC::Transport::HTTP::CGI=HASH(0x975...') called at /home/vladd/mozilla/Bugzilla-trunk/xmlrpc.cgi line 29

The error seems to be located in WebService/Bug.pm, in the get_bug thing. We do there:

+    detaint_natural($bug_id);
+    return new Bugzilla::Bug($bug_id, Bugzilla->user->id);

but detaint_natural simply returns 0 or 1 as return value based on unsuccessful/successful taint. If it's not a number, it probably leaves undef in $bug_id and carries on, and then we pass undef to Bugzilla::Bug, which causes the crash above.

We should probably check the return value of detaint_natural, or maybe allow bug-aliases to be passed via the WebServices interface as well.

Not sure if User or Product have similar errors, probably they should be fixed if they do.


Since we're at it, probably you'd like to enjoy my (optional) nit police as well :)  :

+    [% admindocslinks = {'installation.html#install-perlmodules' => 'Installing perl modules'} %]

s/perl/Perl/

-    if (Bugzilla->batch) {
+    if (Bugzilla->usage_mode) {

I don't like to assume that USAGE_MODE_BROWSER has the particular value of 0. So maybe we should add != USAGE_MODE_BROWSER here and in all the other similar places. (Actually here I think we can end up with only COMMAND_LINE, due to the sub in which we're located).

+use strict;
+eval 'use base qw(XMLRPC::Transport::HTTP::CGI)';
+

I haven't understood why we do the eval thing here instead of using use directly. There's no +$@ && ThrowCodeError('soap_not_installed'); after it, but it could be due to another reason which currently escapes me, so just asking about it, looked strange for me.

+        ->faultstring('Service unimplemented');

Not sure about this, but we might want to do s/unimplemented/Unimplemented/ . (or maybe put that error string in a template so it's localizable :P)

Looks good otherwise. Even if it offers for now functions available via XML output, it's a nice start and commiting it would allow others to start to work in parallel at extending WebService based on a common structure. Cool!

Comment 47

[Marc Schumann [:Wurblzap]]

Attachment: Patch 1.2

Thanks Vlad!

(In reply to comment #46)
> Looks like we're passing "undef" to the Bugzilla::Bug constructor, with this
> patch applied. Which causes a crash.

Fixed.

> +    [% admindocslinks = {'installation.html#install-perlmodules' =>
> 'Installing perl modules'} %]
> 
> s/perl/Perl/

Fixed.

> I don't like to assume that USAGE_MODE_BROWSER has the particular value of 0.
> So maybe we should add != USAGE_MODE_BROWSER here and in all the other similar
> places. (Actually here I think we can end up with only COMMAND_LINE, due to the
> sub in which we're located).

Fixed.

> +use strict;
> +eval 'use base qw(XMLRPC::Transport::HTTP::CGI)';
> +
> 
> I haven't understood why we do the eval thing here instead of using use
> directly. There's no +$@ && ThrowCodeError('soap_not_installed'); after it, but

This appeases runtests.pl if the module is not installed. The ThrowCodeError thing is done by xmlrpc.cgi.

> +        ->faultstring('Service unimplemented');
> 
> Not sure about this, but we might want to do s/unimplemented/Unimplemented/ .

Done. This is how Apache talks to us by default, so let's adopt it.

> (or maybe put that error string in a template so it's localizable :P)

New bug please, if at all.

> Looks good otherwise. Even if it offers for now functions available via XML
> output, it's a nice start and commiting it would allow others to start to work
> in parallel at extending WebService based on a common structure. Cool!

Yeah, extending is the idea.

Comment 48

[Vlad Dascalu]

Thanks Marc. I'll get it reviewed this weekend (around 1st of July).

Comment 49

[Frédéric Buclin]

Comment on attachment 226968 [details] [diff] [review]
Patch 1.2

>+++ ./Bugzilla/WebService/Bug.pm	2006-06-25 11:28:14.143731200 +0200

>+sub get_bug {

>+    return new Bugzilla::Bug($bug_id, Bugzilla->user->id);

How can this work without use'ing Bugzilla::Bug?



>+++ ./Bugzilla/WebService/Product.pm	2006-05-04 19:51:32.295716800 +0200

>+sub get_product {

>+    return new Bugzilla::Product({'name' => $product_name});

Same remark here about Bugzilla::Product.

Comment 50

[Marc Schumann [:Wurblzap]]

(In reply to comment #49)
> How can this work without use'ing Bugzilla::Bug?

The code refers to Bugzilla::Bug, so it works. As soon as we've entered sub new() of Bugzilla::Bug, we can see Bugzilla::Bug's namespace, so we're all set. Try it.

Comment 51

[Frédéric Buclin]

Comment on attachment 226968 [details] [diff] [review]
Patch 1.2

>+++ ./Bugzilla/WebService.pm	2006-06-25 11:06:00.405910400 +0200

>+package Bugzilla::WebService;

>+package Bugzilla::WebService::XMLRPC::Transport::HTTP::CGI;

Can you really have two definitions of package in the same .pm module???


>+eval 'use base qw(XMLRPC::Transport::HTTP::CGI)';

Newbie question: Why not using SOAP::Transport::HTTP::CGI ?

Comment 52

[Marc Schumann [:Wurblzap]]

(In reply to comment #51)
> Can you really have two definitions of package in the same .pm module???

Sure thing, why not? It's a namespace definition. You could even have stuff like this in .pl files.

> Newbie question: Why not using SOAP::Transport::HTTP::CGI ?

I didn't try this, but I don't really mind. XMLRPC::Transport::HTTP::CGI works for me. Do you think SOAP::Transport::HTTP::CGI would be more suitable?

Comment 53

[Frédéric Buclin]

(In reply to comment #52)
> I didn't try this, but I don't really mind. XMLRPC::Transport::HTTP::CGI works
> for me. Do you think SOAP::Transport::HTTP::CGI would be more suitable?

No idea as I never used any of them. I was just curious. But as they are both available in my SOAP::Lite RPM, I don't care. :)

Comment 54

[Vlad Dascalu]

Comment on attachment 226968 [details] [diff] [review]
Patch 1.2

Marc: this is bitrotten due to mkanat's request_cache thing implemented in Bugzilla.pm.

I was about to r+ it, could you update it to the tip since I don't know enough about that?

Comment 55

[Marc Schumann [:Wurblzap]]

Attachment: Patch 1.2.1

Unrotted.

Comment 56

[Max Kanat-Alexander]

Comment on attachment 229290 [details] [diff] [review]
Patch 1.2.1

  Since I'm probably going to be doing a lot of work on the XML-RPC code once the framework is done, I thought I'd do a little review, also.

>+++ ./Bugzilla/Token.pm	2006-07-14 08:25:38.629344000 +0200
>@@ -94,7 +94,6 @@
>     my $vars = {};
> 
>     # Retrieve the user's ID from the database.
>-    trick_taint($loginname);

  I'm pretty sure $loginname can be tainted, if entered from the reset password page. Subs should not require the user to untaint their variables.

>+package Bugzilla::WebService::Bug;
>+
>+use strict;
>+use base qw(Bugzilla::WebService);
>+use Bugzilla::Util qw(detaint_natural);

  It's true that you don't *need* to "use Bugzilla::Bug" here, but it would be a good idea, and there's no reason not to.

>+@Bugzilla::WebService::Constants::EXPORT = qw(
>+    ERROR_AUTH_NODATA
>+
>+    ERROR_UNIMPLEMENTED
>+
>+    ERROR_GENERAL

  Nit: You don't really need extra newlines between those.

>+package Bugzilla::WebService::Product;
>+
>+use strict;
>+use base qw(Bugzilla::WebService);
>+use Bugzilla::Util qw(detaint_natural);

  This package never actually uses detaint_natural.

>+    # Bugzilla::Product doesn't do permissions checks, so we can't do the call
>+    # to Bugzilla::Product::new until a permissions check happens here.
>+    $self->fail_unimplemented();

  I'd rather just not check in the code at all, if it doesn't work yet.

>+package Bugzilla::WebService::User;
>+
> [snip]
>+use Bugzilla::Config;

  You won't need this anymore.

>+    else {
>+        # Use Bugzilla's default if $remember is not supplied.
>+        $remember = Param('rememberlogin') eq 'defaulton'? 'on': '';

  Bugzilla->params now--Param doesn't exist anymore.

>+++ ./Bugzilla.pm	2006-07-12 21:29:49.192622700 +0200
>@@ -262,13 +262,14 @@
>     return request_cache()->{dbh};
> }
> 
>-sub batch {
>+sub usage_mode {

  This sub could actually figure out automatically if it's a CGI or a command-line, with i_am_cgi() from Bugzilla::Util. Then it would only have to be set by the WebService.

>+++ ./checksetup.pl	2006-07-14 08:44:23.566924800 +0200
>@@ -248,6 +248,7 @@
> my $gdtextalign = have_vers("GD::Text::Align",0);
> my $patchreader = have_vers("PatchReader","0.9.4");
> my $imagemagick = have_vers("Image::Magick",0);
>+my $webservices = have_vers("SOAP::Lite",0);

  If bug 344617 goes in before this, by any chance, make sure to move this code into the right place.

>+if (!$webservices && !$silent) {
>+    print "If you want your Bugzilla installation to be accessible via its\n",
>+          "Web Service interface, you will need to install SOAP::Lite by\n",
>+          "running (as $::root):\n\n",
>+    "   " . install_command("SOAP::Lite") . "\n\n";

  The install_command calls usually are preceded immediately by the name of the module, to make it easy for people to quickly scan the output of checksetup.pl.

>+++ ./docs/xml/Bugzilla-Guide.xml	2006-07-12 21:33:16.476114500 +0200
>@@ -67,6 +67,7 @@
> <!ENTITY min-gd-text-align-ver "any">
> <!ENTITY min-chart-base-ver "1.0">
> <!ENTITY min-xml-parser-ver "any">
>+<!ENTITY min-soap-ver "any">

  This just got rotted today. But thankfully, it can just go away, now! :-)

>+++ ./docs/xml/installation.xml	2006-07-13 22:12:06.234547200 +0200
>@@ -417,6 +417,13 @@
> 
>         <listitem>
>           <para>
>+            <link linkend="install-modules-soap">SOAP::Lite</link>
>+            (&min-soap-ver;) for the web service interface

  That will become min-soap-lite-ver

>@@ -520,6 +527,14 @@
>         </para>
>       </section>
> 
>+      <section id="install-modules-soap">
>+        <title>SOAP::Lite (&min-soap-ver;)</title>

  Also here.

>+        <para>Installing SOAP::Lite enables your Bugzilla installation to be
>+        accessible at a standardized Web Service interface (SOAP/XMLRPC)

  Nit: XML-RPC

>+++ ./template/en/default/global/code-error.html.tmpl	2006-06-25 11:41:13.053748800 +0200
>@@ -104,7 +104,7 @@
>     Unable to open the chart datafile <tt>[% filename FILTER html %]</tt>.
>   
>   [% ELSIF error == "chart_lines_not_installed" %]
>-    [% admindocslinks = {'installation.html#install-perlmodules' => 'Installing perl modules necessary for Charting'} %]
>+    [% admindocslinks = {'installation.html#install-perlmodules' => 'Installing Perl modules necessary 

  That change is unrelated to this bug, but that's not so horrible.

  Is "perl" always capitalized? I almost never capitalize it. I suppose it's a proper noun...


  Everything else looks great! :-) The demo script is very nice, too--are we going to check that in to contrib/?

  Also, I was thinking--why don't we just name the functions Bug.get, Product.get, and User.get? That way it could be consistent.

Comment 57

[Max Kanat-Alexander]

Slight correction: The docs part hasn't rotted yet, but bug 344731 will rot it, and it already has a+.

Comment 58

[Max Kanat-Alexander]

Oh also, instead of xmlrpc.cgi, let's just call the file rpc.cgi. Then we could do ?type=soap to get a SOAP RPC instead, later.

Comment 59

[Max Kanat-Alexander]

Oh, and I changed my mind about calling them Bug.get, Product.get, etc. Don't do that--I want to create those functions, but for something else entirely.

Comment 60

[Max Kanat-Alexander]

Oh, and one final question (my apologies for these all being separate comments!): Does this point about mod_perl apply to how we're using SOAP::Lite?

http://search.cpan.org/dist/SOAP-Lite/lib/SOAP/Transport.pm#EXAMPLE_APACHE%3A%3AREGISTRY_USAGE

Comment 61

[Max Kanat-Alexander]

*** Bug 345098 has been marked as a duplicate of this bug. ***

Comment 62

[Max Kanat-Alexander]

*** Bug 345030 has been marked as a duplicate of this bug. ***

Comment 63

[Frédéric Buclin]

*** Bug 327525 has been marked as a duplicate of this bug. ***

Comment 64

[Robert Elves]

Glad to see this feature is well under way!  I'd just like to re-iterate the need for the capabilities addressed in comment#37. 
In particular, after reading through this thread I'm not entirely sure if the following are being address (or are planned):

3) retrieve all operations possible for a given user on a particular bug
5) retrieve all configuration information (repository version, status and priority options, products etc.)

The Bugzilla connector of the Mylar project (http://eclipse.org/mylar) would greatly benefit from these capabilities as would others wishing to integrate with Bugzilla.

Comment 65

[Marc Schumann [:Wurblzap]]

Robert, as per comment 38, this bug will provide a web service framework, so that functionality may be added. I think we should handle each functionality in (more or less) separate follow-up bugs.

Comment 66

[Marc Schumann [:Wurblzap]]

Attachment: Patch 1.3

Thanks for the review, Max.

(In reply to comment #56)
>   I'm pretty sure $loginname can be tainted, if entered from the reset password

Yes. This seems to be a mixup with bug @@@. I'm sorry, this modification doesn't belong here. Removed.

> page. Subs should not require the user to untaint their variables.

Not true!! Untainting is entirely unrelated to whether we're in a sub or not, or a module or not or whatever. All that counts is whether we're close to validation or not. Otherwise there is no point in taint mode! I'll assume what you really mean is that subs need to do their own validation ;)
I really need to come up some kind of a taint mode howto or something.

>   It's true that you don't *need* to "use Bugzilla::Bug" here, but it would be
> a good idea, and there's no reason not to.

Added.
You might want to explain why, though (other than it being "a good idea").

> >+    ERROR_GENERAL
> 
>   Nit: You don't really need extra newlines between those.

Yeah, they were intended to be some kind of error class groups in the beginning, but I didn't shoot as far in the end. Fixed.

> >+package Bugzilla::WebService::Product;
> 
>   This package never actually uses detaint_natural.

Fixed.

>   I'd rather just not check in the code at all, if it doesn't work yet.

I'd like to keep it because it shows basic issues when adding functionality (permissions+security and controlling failure).

> >+use Bugzilla::Config;
> 
>   You won't need this anymore.

Removed.

>   Bugzilla->params now--Param doesn't exist anymore.

Unrotted.

> >-sub batch {
> >+sub usage_mode {
> 
>   This sub could actually figure out automatically if it's a CGI or a
> command-line, with i_am_cgi() from Bugzilla::Util. Then it would only have to
> be set by the WebService.

Maybe; new bug, please, though. Bug 174039 abuses this, and if we start breaking this abusage, I'd like to do it in a separate bug.

> >+my $webservices = have_vers("SOAP::Lite",0);
> 
>   If bug 344617 goes in before this, by any chance, make sure to move this code
> into the right place.

Unrotted.

>   The install_command calls usually are preceded immediately by the name of the
> module, to make it easy for people to quickly scan the output of checksetup.pl.

I don't know what you mean here – this is copied from one of the other places doing this.

> >+<!ENTITY min-soap-ver "any">
> 
>   This just got rotted today. But thankfully, it can just go away, now! :-)

Unrotted.

> >+        accessible at a standardized Web Service interface (SOAP/XMLRPC)
> 
>   Nit: XML-RPC

Ok, I don't mind.

>   That change is unrelated to this bug, but that's not so horrible.

It was so easy to do, so I didn't want to leave the file being inconsistent in this regard.

>   Is "perl" always capitalized? I almost never capitalize it. I suppose it's a
> proper noun...

I don't care. Make up your minds. The Perl man pages capitalize the word (from what my random lookings-at tell).

>   Everything else looks great! :-) The demo script is very nice, too--are we
> going to check that in to contrib/?

It was kinda intended to get put there at some time, so if you're asking me, then the answer is yes.

(In reply to comment #58)
> Oh also, instead of xmlrpc.cgi, let's just call the file rpc.cgi. Then we could
> do ?type=soap to get a SOAP RPC instead, later.

I didn't get this to work. Is suspect this is because this would mean mixing up HTTP GET and POST badly. Skipped.

Comment 67

[Max Kanat-Alexander]

(In reply to comment #66)
> I'll assume what you really mean is that subs need to do their own validation ;)

  Right. :-) That is what I meant. :-)

> >   It's true that you don't *need* to "use Bugzilla::Bug" here, but it would be
> > a good idea, and there's no reason not to.
> 
> Added.
> You might want to explain why, though (other than it being "a good idea").

  Because there's a chance that the calling page won't actually have used Bugzilla::Bug, and if nothing actually uses it, then it's not loaded and perl will error out. It's better to do the "use" closer to where things are actually used, module-wise.

> >   I'd rather just not check in the code at all, if it doesn't work yet.
> 
> I'd like to keep it because it shows basic issues when adding functionality
> (permissions+security and controlling failure).

  I suppose that's fair enough...

> Maybe; new bug, please, though. Bug 174039 abuses this, and if we start
> breaking this abusage, I'd like to do it in a separate bug.

  Okay, agreed.

> >   The install_command calls usually are preceded immediately by the name of the
> > module, to make it easy for people to quickly scan the output of checksetup.pl.
> 
> I don't know what you mean here – this is copied from one of the other places
> doing this.

  Example:

 print "GD:          " . install_command("GD") ."\n" if !$have_mod{'GD'};

> I don't care. Make up your minds. The Perl man pages capitalize the word (from
> what my random lookings-at tell).

  Okay. If the man pages capitalize it, then it should be capitalized. But we don't need to fix the other locations in this bug, really...

> (In reply to comment #58)
> > Oh also, instead of xmlrpc.cgi, let's just call the file rpc.cgi. Then we could
> > do ?type=soap to get a SOAP RPC instead, later.
> 
> I didn't get this to work. Is suspect this is because this would mean mixing up
> HTTP GET and POST badly. Skipped.

  Oh, I didn't mean that you should make ?type=soap actually work! I just mean rename the file so that we can do it in the future. It's a pain to rename a file in CVS after it's already in there.

Comment 68

[Marc Schumann [:Wurblzap]]

Attachment: Demo script

Slightly tweaked.

(In reply to comment #60)
> Oh, and one final question (my apologies for these all being separate
> comments!): Does this point about mod_perl apply to how we're using SOAP::Lite?

I don't know as I know neither mod_perl nor Apache::Registry. Seems to me we'd want to configure all of Bugzilla in one mod_perl thingy. But I may be wrong.

(In reply to comment #67)
> > I don't know what you mean here – this is copied from one of the other places
> > doing this.
> 
>   Example:
> 
>  print "GD:          " . install_command("GD") ."\n" if !$have_mod{'GD'};

I see. It sort of clashes with saying "... by running (as root)" before this because the module name at the beginning of the following line is not part of the command to run. Hrm. If you really want it, I can add it on check-in. Alternatively, you could have a separate bug making all install_command places consistent.

> > > Oh also, instead of xmlrpc.cgi, let's just call the file rpc.cgi. Then we could
> > > do ?type=soap to get a SOAP RPC instead, later.
> > 
> > I didn't get this to work. Is suspect this is because this would mean mixing up
> > HTTP GET and POST badly. Skipped.
> 
>   Oh, I didn't mean that you should make ?type=soap actually work! I just mean
> rename the file so that we can do it in the future. It's a pain to rename a
> file in CVS after it's already in there.

Well I didn't try to make ?type=soap work, but I tried to prepare the script so that it expects ?type=xmlrpc. This doesn't work, and I suspect this is because the HTTP GET style ?type=xmlrpc doesn't agree with the HTTP POST stuff XML-RPC does in its belly. Still skipping.

Comment 69

[Enrico Weigelt]

I'm currently working on an java API for accessing bugzilla (and similar tools):

http://nibiru.borg.metux.de:7000/wiki/index.php/Java-issuetracker

Comment 70

[Mik Kersten]

Enrico, note that we provide a well-tested Java API for Bugzilla (and other trackers) on the Mylar project: http://wiki.eclipse.org/index.php/Mylar_Architecture and http://eclipse.org/mylar/  We're very open to contributions and requests if you're missing anything.

Comment 71

[Enrico Weigelt]

Where can I get it ? 
I only need the bugzilla API, nothing more.

Comment 72

[Mik Kersten]

I just added a few notes on how to get it and use it to: http://wiki.eclipse.org/index.php/Mylar_Integrator_Reference

If you would like us to ship a JAR with just the API in it please file a new bug report: http://www.eclipse.org/mylar/bugs.php 

Comment 73

[Marc Schumann [:Wurblzap]]

Attachment: Patch 1.3.1

Unrotted.

Comment 74

[Marc Schumann [:Wurblzap]]

Attachment: Patch 1.3.2

Unrotted. It seems to me we need to put best-before dates no later than a day in the future on patches these days...

Comment 75

[Frédéric Buclin]

mkanat, could you review this one, please?

Comment 76

[Marc Schumann [:Wurblzap]]

Attachment: Wrong patch

Unrotted.
Vlad, should I refrain from asking you for review these days?

Comment 77

[Marc Schumann [:Wurblzap]]

Attachment: Patch 1.3.3

Comment 78

[Max Kanat-Alexander]

Comment on attachment 233632 [details] [diff] [review]
Patch 1.3.3

One thing occurred to me the other day. Instead of calling it usage_mode, we should just call it error_mode, because that's what it really is, and then we could have ERROR_MODE_DIE, ERROR_MODE_THROW, and ERROR_MODE_WEBSERVICE.

There's no need to detaint the bug_id in Bugzilla::Webservice::Bug--Bugzilla::Bug::new will do that. Also, Bugzilla::Bug::new no longer takes a second argument. So, you'll have to remove that and do a $user->can_see_bug check instead.

One function I'd like to see is Bugzilla.version -- to get the current version of Bugzilla, so people can know what version of the API they're using. I'd like to see this in there from the very start, so that it will never fail.

In Bugzilla::WebService::User, rememberlogin can also be "on" (in addition to "defaulton") so you'll want to check for that, I think, yes?

Add a comment that explains why you do the "eval 'use base'" thing.

Everything else looks pretty good.

Make sure that users don't get a whole HTML page of an error when they get an error. That is, they should just get the error text. I've seen some scripts throw a whole HTML page (like importxml.pl).

Everything else looks fine, so with these things fixed (mostly the Bug thing) I should be able to r+ it.

Comment 79

[Vlad Dascalu]

Comment on attachment 233632 [details] [diff] [review]
Patch 1.3.3

> Vlad, should I refrain from asking you for review these days?

Yes, I can't seem to find the time for those :-( .

Comment 80

[Max Kanat-Alexander]

(In reply to comment #78)
> So, you'll have to remove that and do a $user->can_see_bug check instead.

  It also occurred to me that you could just call ValidateBugID before calling Bugzilla::Bug->new. That would probably be even better.

Comment 81

[Marc Schumann [:Wurblzap]]

Attachment: Patch 1.4

Comment 82

[Marc Schumann [:Wurblzap]]

Comment on attachment 234385 [details] [diff] [review]
Patch 1.4

(In reply to comment #78)
> One thing occurred to me the other day. Instead of calling it usage_mode, we
> should just call it error_mode, because that's what it really is, and then we
> could have ERROR_MODE_DIE, ERROR_MODE_THROW, and ERROR_MODE_WEBSERVICE.

Hmmmm... It's two different things, really. Imagine checksetup.pl being callable from the web at some time -- usage_mode would change the way how to ask for the admin's e-mail, for example, while error_mode (as called by post_bug.cgi) wants to influence the error mode only.

I introduced error_mode and let usage_mode call error_mode.

> There's no need to detaint the bug_id in
> Bugzilla::Webservice::Bug--Bugzilla::Bug::new will do that. Also,
>
> Bugzilla::Bug::new no longer takes a second argument. So, you'll have to remove
> that and do a $user->can_see_bug check instead.
>
>   It also occurred to me that you could just call ValidateBugID before calling
> Bugzilla::Bug->new. That would probably be even better.

Yes. Done. I needed to fix detaint_natural, though... For some reason, it didn't untaint when called in non-void context. I fixed trick_taint and detaint_signed along the way, and I took care not to rip open bug 297928 again.

> One function I'd like to see is Bugzilla.version -- to get the current version
> of Bugzilla, so people can know what version of the API they're using. I'd like
> to see this in there from the very start, so that it will never fail.

Good idea. Done.

> In Bugzilla::WebService::User, rememberlogin can also be "on" (in addition to
> "defaulton") so you'll want to check for that, I think, yes?

Some guy wrote a new auth code which takes care of this for us.

> Add a comment that explains why you do the "eval 'use base'" thing.

Done.

> Make sure that users don't get a whole HTML page of an error when they get an
> error. That is, they should just get the error text. I've seen some scripts
> throw a whole HTML page (like importxml.pl).

Error.pm sends the error identifier only, no HTML.

Comment 83

[Max Kanat-Alexander]

(In reply to comment #82)
> Hmmmm... It's two different things, really. [snip]

  I agree, but at this time the only thing we have is error_mode. That is, usage_mode means nothing to Bugzilla but error_mode, so I'd like to just have that. At some point if we have some need for usage_mode, we can introduce it then.

  As it stands, usage_mode() and batch() both hide what's really going on, which is just that we're only changing the error mode.

> Yes. Done. I needed to fix detaint_natural, though... [snip]

  That's weird. Well, the changes you made look fine, even though I'm not sure why they were needed.

> Error.pm sends the error identifier only, no HTML.
 
  Oh...it might be better to actually send the text, since the text involves variables and so on. But I suppose that's not something critical that we have to decide right now. Still, I don't see any reason to make the client re-implement all our error messages... perhaps we should send both somehow.

Comment 84

[Max Kanat-Alexander]

Comment on attachment 234385 [details] [diff] [review]
Patch 1.4

Okay, other than the usage_mode/error_mode thing everything looks fine in this patch.

I just think it's going to confuse developers when they should use usage_mode and when they should use error_mode. I don't think developers should have to read docs in-depth to understand things, and I think error_mode is a lot clearer just from reading the code.

Comment 85

[Max Kanat-Alexander]

Comment on attachment 234385 [details] [diff] [review]
Patch 1.4

Okay, actually, I'll have a use for usage_mode in the near future (for the email interface), so I suppose we might as well keep it.

It *is* going to be confusing. Perhaps we should mention that most scripts should just use usage_mode, and error_mode would rarely be called directly.

Comment 86

[Max Kanat-Alexander]

I'm also requesting approval to check in the demo script to contrib/.

Comment 87

[Marc Schumann [:Wurblzap]]

(In reply to comment #85)
> It *is* going to be confusing. Perhaps we should mention that most scripts
> should just use usage_mode, and error_mode would rarely be called directly.

I don't think it's very confusing. There is a simple rule of thumb -- you'll use usage_mode at the beginning of your script, and it'll take care of your woes. You'll use error_mode in the middle of your script or in modules so that you can control how to handle errors.

Comment 88

[Marc Schumann [:Wurblzap]]

Thanks Vlad and Max for the reviews!

Now, all, let them follow-up bugs and patches flow to fill this framework with life.

Checking in Bugzilla.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla.pm,v  <--  Bugzilla.pm
new revision: 1.45; previous revision: 1.44
done
Checking in collectstats.pl;
/cvsroot/mozilla/webtools/bugzilla/collectstats.pl,v  <--  collectstats.pl
new revision: 1.57; previous revision: 1.56
done
Checking in importxml.pl;
/cvsroot/mozilla/webtools/bugzilla/importxml.pl,v  <--  importxml.pl
new revision: 1.67; previous revision: 1.66
done
Checking in post_bug.cgi;
/cvsroot/mozilla/webtools/bugzilla/post_bug.cgi,v  <--  post_bug.cgi
new revision: 1.161; previous revision: 1.160
done
RCS file: /cvsroot/mozilla/webtools/bugzilla/xmlrpc.cgi,v
done
Checking in xmlrpc.cgi;
/cvsroot/mozilla/webtools/bugzilla/xmlrpc.cgi,v  <--  xmlrpc.cgi
initial revision: 1.1
done
Checking in Bugzilla/Constants.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Constants.pm,v  <--  Constants.pm
new revision: 1.48; previous revision: 1.47
done
Checking in Bugzilla/Error.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Error.pm,v  <--  Error.pm
new revision: 1.18; previous revision: 1.17
done
Checking in Bugzilla/Util.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Util.pm,v  <--  Util.pm
new revision: 1.52; previous revision: 1.51
done
RCS file: /cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService.pm,v
done
Checking in Bugzilla/WebService.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService.pm,v  <--  WebService.pm
initial revision: 1.1
done
Checking in Bugzilla/Auth/Login/CGI.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Auth/Login/CGI.pm,v  <--  CGI.pm
new revision: 1.7; previous revision: 1.6
done
Checking in Bugzilla/Install/Requirements.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Install/Requirements.pm,v  <--  Requirements.pm
new revision: 1.16; previous revision: 1.15
done
RCS file: /cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService/Bug.pm,v
done
Checking in Bugzilla/WebService/Bug.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService/Bug.pm,v  <--  Bug.pm
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService/Bugzilla.pm,v
done
Checking in Bugzilla/WebService/Bugzilla.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService/Bugzilla.pm,v  <--  Bugzilla.pm
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService/Constants.pm,v
done
Checking in Bugzilla/WebService/Constants.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService/Constants.pm,v  <--  Constants.pm
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService/Product.pm,v
done
Checking in Bugzilla/WebService/Product.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService/Product.pm,v  <--  Product.pm
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService/User.pm,v
done
Checking in Bugzilla/WebService/User.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/WebService/User.pm,v  <--  User.pm
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/webtools/bugzilla/contrib/bz_webservice_demo.pl,v
done
Checking in contrib/bz_webservice_demo.pl;
/cvsroot/mozilla/webtools/bugzilla/contrib/bz_webservice_demo.pl,v  <--  bz_webservice_demo.pl
initial revision: 1.1
done
Checking in docs/xml/installation.xml;
/cvsroot/mozilla/webtools/bugzilla/docs/xml/installation.xml,v  <--  installation.xml
new revision: 1.128; previous revision: 1.127
done
Checking in template/en/default/global/code-error.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/global/code-error.html.tmpl,v  <--  code-error.html.tmpl
new revision: 1.76; previous revision: 1.75
done
Checking in template/en/default/global/user-error.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/global/user-error.html.tmpl,v  <--  user-error.html.tmpl
new revision: 1.181; previous revision: 1.180
done

Comment 89

[Frédéric Buclin]

I fixed the bustage due to an unfiltered directive in code-error.html.tmpl.

Checking in template/en/default/global/code-error.html.tmpl;
/cvsroot/mozilla/webtools/bugzilla/template/en/default/global/code-error.html.tmpl,v  <--  code-error.html.tmpl
new revision: 1.77; previous revision: 1.76
done

Comment 90

[Colin Ogilvie [:cso]]

Docs bustage fix:

Checking in docs/xml/installation.xml;
/cvsroot/mozilla/webtools/bugzilla/docs/xml/installation.xml,v  <--  installation.xml
new revision: 1.129; previous revision: 1.128
done

Comment 91

[Bill Barry]

I just updated my local bugzilla tip (using "cvs update") and when I run ./checksetup.pl it fails with:
Can't locate Bugzilla/WebService/Constants.pm in @INC (...)

It appears that this patch is causing this with the following line:
+use Bugzilla::WebService::Constants;

As far as I know the only Constants.pm in the bugzilla installation is Bugzilla/Constants.pm. Is there another I am suppossed to have? 

Is something wrong with my bugzilla (is anyone else getting this)?

Comment 92

[Olav Vitters]

Use:
  cvs update -dP

Comment 93

[Alexandre Michetti Manduca]

  I started working with this webservice interface today but I got confused.
  I should implement a client that exchange simple XML-RPC messages or one that exchanges SOAP messages?

  I read some comments on this bug and, for me, it looks like that you talk about XML-RPC and SOAP as if they are the same thing (SOAP/XML-RPC), but they are different protocols.

  For example, I tryied to create a php client (using NuSOAP) to login and get some information about one bug, but the Bugzilla code couldn't deserialize the message.

* This is the request that contrib/bz_webservice_demo.pl creates when you try to login (which is a XML-RPC request):

<?xml version="1.0" encoding="UTF-8"?>
<methodCall>
  <methodName>User.login</methodName>
  <params>
    <param><value><string>[e-mail]</string></value></param>
    <param><value><int>123456</int></value></param>
    <param><value /></param>
  </params>
</methodCall>


* This is the request NuSOAP creates (which is a default SOAP message, since we are not using WSDL):

<?xml version="1.0" encoding="ISO-8859-1"?>
<SOAP-ENV:Envelope
  SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" 
  xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" 
  xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"      
  xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/">

  <SOAP-ENV:Body>
    <ns1266:User.login xmlns:ns1266="http://tempuri.org">
    <__numeric_0 si:type="xsd:string">[e-mail]</__numeric_0>
    <__numeric_1 xsi:type="xsd:string">123456</__numeric_1></ns1266:User.login>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

  I didn't read all the comments on this bug (too big), so sorry if I missed something. Just trying to help.

Comment 94

[Max Kanat-Alexander]

(In reply to comment #93)
>   I started working with this webservice interface today but I got confused.

  If you use this webservice interface, you should know that it will change completely before the release of Bugzilla 3.0. Once 3.0rc1 is out, you should be quite safe to use the webservice interface.

  It's XML-RPC. XML-RPC is not SOAP.

Comment 95

[Marc Schumann [:Wurblzap]]

(In reply to comment #94)
>   If you use this webservice interface, you should know that it will change
> completely before the release of Bugzilla 3.0. Once 3.0rc1 is out, you should

For implementers, the result is quite indistinguishable, but I think "completely" should be replaced by "fundamentally" or something more to this effect.

Comment 96

[Alexandre Michetti Manduca]

(In reply to comment #95)
> (In reply to comment #94)
> >   If you use this webservice interface, you should know that it will change
> > completely before the release of Bugzilla 3.0. Once 3.0rc1 is out, you should
> 
> For implementers, the result is quite indistinguishable, but I think
> "completely" should be replaced by "fundamentally" or something more to this
> effect.
> 

  I was just testing this webservice interface because I need to integrate Bugzilla with other tools in my university project, so it would be much better if I could use some infrasctrucute supported by the Bugzilla community than starting something completely new. The difference is that we are already using SOAP in all the other tools.

  Anyway, I think that this webservice interface would work with SOAP if we just change XMLRPC::Transport::HTTP to SOAP::Transport::HTTP, right? So it would try to deserialize a SOAP protocol instead of a XML-RPC one?

  I will need to implement a SOAP interface anyway, so if you have plans to provide a SOAP interface, I would like to contribute.

Comment 97

[Max Kanat-Alexander]

Added to the relnotes currently attached to bug 349423.

Comment 98

[Frédéric Buclin]

We have various test scripts for each method defined here -> testcase+