559721 - [BUGZILLA] Mozbot fails to collect data from Bugzilla 3.6 (bugzilla.dtd related)

Status: RESOLVED INACTIVE

(Webtools Graveyard :: Mozbot, defect)

Description

[Tanner M. Young [:tmyoung]]

Something has regressed between Bugzilla 3.4.x and 3.6 in the api that causes Mozbot to not collect data from Bugzilla pages.

Example:

<mozbot>	New General - General bug 2769 filed by [user].
[No data]
<tmyoung>	mozbot: Bug 2769
[No data]

I will investigate further, to see if I can find anything.

Comment 1

[Tanner M. Young [:tmyoung]]

2010-04-16 02:42:41 UTC (15531) Told: #ays <tmyoung> Bug 2768
2010-04-16 02:42:41 UTC (15531) Module Bugzilla: spawned 15533 (wget --quiet --passive --user-agent="Mozilla/5.0 (compatible; mozbot)" --output-document=- http://bugzilla.[install]/show_bug.cgi?ctype=xml&excludefield=long_desc&excludefield=attachmentdata&excludefield=cc&id=2768)
2010-04-16 02:42:42 UTC (15531) child 15533 completed URI
2010-04-16 02:42:42 UTC (15531) ERROR!!!
2010-04-16 02:42:42 UTC (15531) Empty String at (eval 33) line 401

2010-04-16 02:42:44 UTC (15531) Told: #ays <tmyoung> Bug 2769
2010-04-16 02:42:44 UTC (15531) Module Bugzilla: spawned 15537 (wget --quiet --passive --user-agent="Mozilla/5.0 (compatible; mozbot)" --output-document=- http://bugzilla.[install]/show_bug.cgi?ctype=xml&excludefield=long_desc&excludefield=attachmentdata&excludefield=cc&id=2769)
2010-04-16 02:42:45 UTC (15531) child 15537 completed URI
2010-04-16 02:42:45 UTC (15531) ERROR!!!
2010-04-16 02:42:45 UTC (15531) Empty String at (eval 33) line 401

Comment 2

[Tanner M. Young [:tmyoung]]

This may be totally unrelated, but when mozbot starts, there is an warning about Net::IRC::Connection.pm, which discusses line 480 of that module

  if (exists $self->{_handler}->{$ev}) {

The error states that "an undefined value was used as if it were already defined." "Use of uninitialized value $ev in exists at /usr/local/share/perl/5.10.0/Net/IRC/Connection.pm line 480 #1."

I doubt this has anything to do with this problem, because it appears to happen before any queries are sent to the server, but I still thought it worth mentioning.

-------

Another note: the errors above are all that mozbot lists.

Here is the output of the url that mozbot queries

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugzilla.alyoung.com/bugzilla.dtd">
<bugzilla version="3.6"
          urlbase="https://bugzilla.alyoung.com/"
          maintainer="[e-mail]"
          exporter="[e-mail]"
>
    <bug>
          <bug_id>2769</bug_id>
          <creation_ts>2010-04-15 20:28:00 -0600</creation_ts>
          <short_desc>Test</short_desc>
          <delta_ts>2010-04-15 20:28:59 -0600</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>6</classification_id>
          <classification>General</classification>
          <product>General</product>
          <component>General</component>
          <version>unspecified</version>
          <rep_platform>All</rep_platform>
          <op_sys>All</op_sys>
          <bug_status>NEW</bug_status>          
          <priority>--</priority>
          <bug_severity>normal</bug_severity>
          <target_milestone>---</target_milestone>
          <everconfirmed>1</everconfirmed>
          <reporter name="Tanner M. Young [:tmyoung]">[e-mail]</reporter>
          <assigned_to name="Nobody; OK to take it and work on it">[e-mail]</assigned_to>
          <estimated_time>0.00</estimated_time>
          <remaining_time>0.00</remaining_time>
          <actual_time>0.00</actual_time>
          <qa_contact>[e-mail]</qa_contact>
          <cf_browsers>---</cf_browsers>
          <cf_nagios_state>---</cf_nagios_state>
          <cf_ngaios_downtime>---</cf_ngaios_downtime>
          <token>1271386284-5762b6004adbda0965dd7a1d3d721d32</token>
    </bug>
</bugzilla>

Here is an output from BMO that works for comparison:

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugzilla.mozilla.org/bugzilla.dtd">
<bugzilla version="3.4.6+"
          urlbase="https://bugzilla.mozilla.org/"
          maintainer="[e-mail]"
          exporter="[e-mail]"
>
    <bug>
          <bug_id>2769</bug_id>
          <creation_ts>1998-11-02 08:00:00 -0800</creation_ts>
          <short_desc>[LDAP] Auth with bogus credentials does not bind anonymously</short_desc>
          <delta_ts>2009-03-13 19:05:24 -0700</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>3</classification_id>
          <classification>Components</classification>
          <product>MailNews Core</product>
          <component>LDAP Integration</component>
          <version>Trunk</version>
          <rep_platform>All</rep_platform>
          <op_sys>All</op_sys>
          <bug_status>NEW</bug_status>
          <keywords>helpwanted</keywords>
          <priority>P1</priority>
          <bug_severity>normal</bug_severity>
          <target_milestone>---</target_milestone>
          <blocked>148891</blocked>
          <votes>1</votes>
          <everconfirmed>1</everconfirmed>
          <reporter>[e-mail]</reporter>
          <assigned_to name="Nobody; OK to take it and work on it">[e-mail]</assigned_to>
          <qa_contact>[e-mail]</qa_contact>
          <cf_blocking_193>---</cf_blocking_193>
          <cf_status_193>---</cf_status_193>
          <cf_blocking_192>---</cf_blocking_192>
          <cf_status_192>---</cf_status_192>
          <cf_blocking_191>---</cf_blocking_191>
          <cf_status_191>---</cf_status_191>
          <cf_blocking_thunderbird31>---</cf_blocking_thunderbird31>
          <cf_status_thunderbird31>---</cf_status_thunderbird31>
          <cf_blocking_thunderbird30>---</cf_blocking_thunderbird30>
          <cf_status_thunderbird30>---</cf_status_thunderbird30>
          <token>1271386409-985030c93ffabb2ec7f06cb2915ee715</token>
    </bug>
</bugzilla>

Comment 3

[Tanner M. Young [:tmyoung]]

The bad news is that one of my bots works with 3.6 on landfill, but the good news is that the error

2010-04-16 02:42:45 UTC (15531) child 15537 completed URI
2010-04-16 02:42:45 UTC (15531) ERROR!!!
2010-04-16 02:42:45 UTC (15531) Empty String at (eval 33) line 401

does not appear when communicating with landfill, which indicates the above error is a sign of the problem.

Comment 4

[Tanner M. Young [:tmyoung]]

Line 401 of the Bugzilla module with the error is

        my $tree = $parser->parse_string($output);

Also, this is not tied to mozbot version it appears both in 2.6 and Trunk.

Comment 5

[Tanner M. Young [:tmyoung]]

The problem stems from $output being blank, I will continue investigating.

Comment 6

[Tanner M. Young [:tmyoung]]

This is because of a bug in Bugzilla's DTD ( Bug 538428 ).

However, we should probably move to LWP::UserAgent, which I have a patch for (though that would be another bug).

Wolf: Would you approve?  That would prevent forking for mozbot, and it would mean that mozbot could be one step closer to working on Windows.

Comment 7

[Wolf [:wolf]]

> Wolf: Would you approve?  That would prevent forking for mozbot, and it would
> mean that mozbot could be one step closer to working on Windows.

Yes. Its a major change though, but I would be interested. (In a new bug of course.)

Comment 8

[Tanner M. Young [:tmyoung]]

Filed Bug 560450 for LWP::UserAgent support.

Once Bug 538428 is landed and we confirm that it fixes the problem for everyone--not just me, then we can close this bug.

Comment 9

[Wolf [:wolf]]

BMO is 3.6.3+ and Firebot has no issues with it.

Is Bug 538428 applied to BMO or is something else going on here?

Comment 10

[Wolf [:wolf]]

Hmm. spoke too soon.

2011-01-18 09:18:00 UTC (1724) Module Bugzilla: spawned 6916 (wget --quiet --pas
sive --user-agent="Mozilla/5.0 (compatible; mozbot)" --output-document=- https:/
/bugzilla.mozilla.org/show_bug.cgi?ctype=xml&excludefield=long_desc&excludefield
=attachmentdata&excludefield=cc&id=5800)
Handler for 'msg' called.

2011-01-18 09:18:00 UTC (1724) child 6916 completed URICan't exec "mozbot.pl": N
o such file or directory at mozbot.pl line 1884 (#2)
    (W exec) A system(), exec(), or piped open call could not execute the
    named program for the indicated reason.  Typical reasons include: the
2011-01-18 09:18:00 UTC (1724) ERROR!!!
2011-01-18 09:18:00 UTC (1724) Empty String at (eval 1027) line 400

Comment 12

[Tanner M. Young [:tmyoung]]

I think this has been fixed since the Bugzilla DTD was fixed.  I don't have any way of testing this at the moment.  Wolf, can you (or someone else) confirm that Bug 538428 cleared this up?

I think if it did fix it, since there is nothing for us to do on our end, we will call it a duplicate of the other bug.

Comment 13

[:glob ✱]

Mozbot/Firebot is in maintenance-only mode; no further changes are planned.