848244 - Deal with fact that BMO 4.2 returns "depends_on" and "blocks" fields in search results

Status: RESOLVED FIXED

(Webtools Graveyard :: BzAPI, defect)

Description

[Axel Hecht [:Pike]]

In a URL like https://api-dev.bugzilla.mozilla.org/1.2/bug?id=846806%2C+496109%2C+590164%2C+590165%2C+590166%2C+590167%2C+590169%2C+590170%2C+590171%2C+775487%2C636983%2C+846788&resolution=---&include_fields=id%2Cproduct%2Ccomponent%2Csummary%2Clast_change_time%2Ckeywords%2Cblocks, the entry for blocks is a single number instead of an array with one entry.

That was working yesterday, as I had written that code then. Marking as bugzilla update regression blocking bug 848218

Not sure if the comments in bug 782967, which sound similar.

To be precise, in the plaintext output

--- 
bugs: 
  - 
    blocks: 636982
    component: zu / Zulu
    id: 636983
    keywords: 
      - productization
    last_change_time: 2012-08-30T09:25:56Z
    product: Mozilla Localizations
    summary: "[zu] Search engine setup for Firefox 4.0 for Zulu."

should look like

--- 
bugs: 
  - 
    blocks:
      - 636982
    component: zu / Zulu
    id: 636983
    keywords: 
      - productization
    last_change_time: 2012-08-30T09:25:56Z
    product: Mozilla Localizations
    summary: "[zu] Search engine setup for Firefox 4.0 for Zulu."

Comment 1

[Gervase Markham [:gerv]]

I think the problem here is as follows:

- After 11 years, someone has finally fixed bug 102048, which adds "depends_on" and "blocks" 
  to the bug list
- We took this patch into the bmo-4.2 branch in rev 8525
- These fields therefore now show up when you ask for "columnlist=all"
- In the past, because these fields were not in the bug list, BzAPI did an extra behind-
  the-scenes query for the individual bug XML to get them so it could return them
- That is no longer necessary; the data now comes from the bug list
- In the XML, the data was normally an array, as each number was a separate element. We had 
  fixup code (which required a typo fix in bug 782967) to force an array in the single 
  element case.
- However, the CSV call returns the data as a string, and there is no magic fixup code 
  because blocks/depends_on data is not expected. (In fact, up to now, every field returned 
  by a buglist has been single-valued.)
- Therefore, you are now seeing blocks and depends_on data (both when there is one value and 
  when there are multiple values) as a string rather than an array.

To fix it, I need to add code to the CSV data fixup path to split these strings into arrays. 

Gerv

Comment 2

[Axel Hecht [:Pike]]

That explains why I got the string when asking for cf_blocking_b2g.

It'd be great if we could get this compatible with what we get for the individual bug, aka the ForceArray code.

Comment 3

[Gervase Markham [:gerv]]

Axel: cf_blocking_b2g is a single-valued field - so you'd expect a string, right?

The only multiple-valued field I can see in b.m.o. is "cf_locale". Irritatingly (although I suppose there's no reason it should) its name does not begin with cf_multi, so I will have to special-case it. I can't find a bug which returns multiple values for that field, though - do you know of one? Or do I have to file a test bug?

The ForceArray code is:

    my $xs = XML::Simple->new(ForceArray => 
                                ['cc', 'see_also', 'attachment', 'dependson', 
                                 'blocked', 'comment', 'flag', 'group', 'bug',
                                 qr/^cf_multi/],
                              KeyAttr => { },
                              SuppressEmpty => 1
                              );

The fields on that list that I think appear in buglists are: "dependson", "blocked", "flag", plus multiply-valued custom fields. Is that what you think?

Gerv

Comment 4

[Axel Hecht [:Pike]]

Sorry, I misremembered that, I think it was actually bug history that shows array-valued changes as simple strings. Sounds like a bug to be filed?

Comment 5

[Gervase Markham [:gerv]]

No; bug history entries are strings by design, and stored as such in the database. I looked at one point at fixing that in the API, but it would have been a nightmare of special-cases and may not be possible in the general case. I'm afraid you have to do your own history parsing for the fields that you care about.
 
See https://wiki.mozilla.org/Bugzilla:REST_API:Objects#Change .

Gerv

Comment 6

[Gervase Markham [:gerv]]

I have updated the code in /latest and restarted Apache. The URL in comment 0, when switched from 1.2 to latest, now correctly returns an array for the "blocks" field. This fix will be included in the next release, which I will make soon.

Gerv