958559 - Attached file got corrupted via Bugzilla XML-RPC API

Status: RESOLVED INVALID

(bugzilla.mozilla.org :: API, defect)

Description

[Khokhar]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36

Steps to reproduce:

Initially this bug was created as comment#33 of bug#412074 and not its moved here as a new fresh bug.

By using add_attachment method I can upload files with different extensions (.zip, .jpg etc) , but uploaded files got corrupted. 

I've google it and found that many people have this problem with add_attachment method of Bugzilla API.

Ref: 
http://stackoverflow.com/questions/7131197/uploading-attachments-to-bugzilla-using-the-web-services-api-and-perl

http://stackoverflow.com/questions/13516982/add-an-attachment-to-bugzilla-using-xml-rpc-in-vba

My Bugzilla version is 4.2.3

I am using a python module for interacting with Bugzilla over XMLRPC
https://git.fedorahosted.org/git/python-bugzilla.git


I am just copying here the attachfile function from base.py
Its very straightforward python code. 

    def attachfile(self, idlist, attachfile, description, **kwargs):

        if isinstance(attachfile, str):
            f = open(attachfile)
        elif hasattr(attachfile, 'read'):
            f = attachfile
        else:
            raise TypeError("attachfile must be filename or file-like object")

        # Back compat
        if "contenttype" in kwargs:
            kwargs["content_type"] = kwargs.pop("contenttype")
        if "ispatch" in kwargs:
            kwargs["is_patch"] = kwargs.pop("ispatch")
        if "isprivate" in kwargs:
            kwargs["is_private"] = kwargs.pop("isprivate")
        if "filename" in kwargs:
            kwargs["file_name"] = kwargs.pop("filename")

        kwargs['summary'] = description
        if 'file_name' not in kwargs:
            kwargs['file_name'] = os.path.basename(f.name)

        if 'content_type' not in kwargs:
            kwargs['content_type'] = 'application/octet-stream'

        kwargs['data'] = self._attachment_encode(f)
        kwargs['ids'] = self._listify(idlist)

        ret = self._proxy.Bug.add_attachment(kwargs)

        if "attachments" in ret:
            # Up to BZ 4.2
            ret = [int(k) for k in ret["attachments"].keys()]
        elif "ids" in ret:
            # BZ 4.4+
            ret = ret["ids"]

        if type(ret) is list and len(ret) == 1:
            ret = ret[0]
        return ret

And here is function call:

        bug_id = 15
        my_file = "abc.zip"
        att_file_desc = "Attached file description"                

        kwards = {
            'contenttype':'application/octet-stream',
            'filename': my_file 
        } 

        #   attachfile method will return the id of attached file 
        bz.attachfile( bug_id, my_file, att_file_desc, **kwards) 



Actual results:

By using add_attachment method I can upload files with different extensions (.zip, .jpg etc) , but uploaded files got corrupted. 



Expected results:

Uploaded files should not get corrupted.

Comment 1

[Khokhar]

Attachment: Original attachment.jpg

Comment 2

[Khokhar]

Attachment: Original config-file.zip

Comment 3

[Khokhar]

Attachment: corrupted_attachment.jpg

Comment 4

[Khokhar]

Attachment: corrupted_config-file.zip

Comment 5

[Khokhar]

I got a question about _attachment_encode(f) from  David Lawrence in https://bugzilla.mozilla.org/show_bug.cgi?id=412074#c34

And Answer is Yes, 

Here is body of that method.

    def _attachment_encode(self, fh):
        chunksize = 3072
        data = ''
        chunk = fh.read(chunksize)

        while chunk:
            data = data + base64.b64encode(chunk)
            chunk = fh.read(chunksize)
        return data


You can also explore python-bugzilla.git (see the path in description of bug)

For me this issue is Critical because all uploading files getting corrupt..

Comment 6

[Frédéric Buclin]

(In reply to Khokhar from comment #5)
>             data = data + base64.b64encode(chunk)

It's not enough to encode the data as Base64. As said in the documentation, you must also declare your data as Base64 encoded. Else it's passed as a string and is not decoded by Bugzilla.

Comment 7

[Samip Banker]

I am facing same problem. even after declaring content_type as base64 created file using add_attachment is junk...

May be I am doing something very obvious that I can't seem to realize... 

Usecase is that I am reading the file data from source BZ and creating a bug in another BZ. 
I am reading data from source straight from by calling
 my %params = (ids => $source_bug_id);
 my $bugsAttachements = BZ::Client::API->api_call($clientSource,"Bug.attachments",\%params);



Below is snippet of my code.. Please help...
    $fileData = MIME::Base64::decode_base64($fileAttach->{data});
    %params = (ids => $destination_bug_id,
               data => $fileData,
               file_name => $fileAttach->{file_name},
               summary => $fileAttach->{summary},
               content_type => $fileAttach->{content_type},
               is_obsolete => $fileAttach->{is_obsolete},
               is_patch => $fileAttach->{is_patch});
    
   
   my $bugAttachementAdd = BZ::Client::API->api_call($clientDestination,"Bug.add_attachment",\%params);

Comment 8

[:glob ✱]

(In reply to Samip Banker from comment #7)
> Below is snippet of my code.. Please help...
>     $fileData = MIME::Base64::decode_base64($fileAttach->{data});

you should be calling encode_base64(..) not decode.

Comment 9

[Samip Banker]

Byron:

Thanks for the feedback. encode_base64 is also generating junk data in the file. 

Question... 
http://www.bugzilla.org/docs/tip/en/html/api/Bugzilla/WebService/Bug.html states this under "add_attachment"...

"data
Required base64 or string The content of the attachment. If the content of the attachment is not ASCII text, you must encode it in base64 and declare it as the base64 type."

How do we declare that attachement is a base64 type?

Comment 10

[:glob ✱]

(In reply to Samip Banker from comment #9)
> How do we declare that attachement is a base64 type?

sorry, but i'm not familiar with the BZ::Client package.

using XMLRPC::Lite you'd do: XMLRPC::Data->type('base64', $content)
there's a full sample script at: https://gist.github.com/globau/8798382