1180409 - sdk/system/child_process stdout stream bugs

Status: RESOLVED INCOMPLETE

(Add-on SDK Graveyard :: General, defect)

Description

[Alervd]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:39.0) Gecko/20100101 Firefox/39.0
Build ID: 20150630154324

Steps to reproduce:

I spawn a process:
var child_process = require("sdk/system/child_process");
this.brhlpr_process = child_process.spawn (this.fdmAppInfo.brhlpr_path);

I start to listen for data:
this.brhlpr_process.stdout.on ('data', function (data_from_exe) {});

2 bugs I have for now:

1. If my exe sends data which contains 0 bytes, this API drops out all the data starting from this 0 byte. E.g. for 44 byte buffer sent by my EXE data_from_exe will contain only the first byte. The remaining 43 bytes will be lost forever.

2. OK. I can make a workaround for 1st bug and make my buffer to contain strings only. But. Here is another HUGE bug. I send a JSON string from my EXE. This JSON contains a list of strings (localized strings of my APP). If these strings are in English - works just fine. If not - things becomes worse. No notification come about data from my EXE! Strings are encoding using UTF-8.
I've tried using this:
this.brhlpr_process.stdout.setEncoding ('utf8');
Did not help.

Comment 1

[Alervd]

Regarding the 2nd bug: description is incorrect, my fault.
I get the data notification, but the data is ruined.

EXE data length: 393.
Data length in notification (data_from_exe.length): 286.

Is there a way for force this API to give me the buffer contents w/o any pre-processing?
I want it to be exactly the same the EXE sent it to me.

Comment 2

[Alervd]

This does not help:
this.brhlpr_process.stdout.setEncoding ('ascii');

Comment 3

[Alervd]

This does not help:
this.brhlpr_process.stdout.setEncoding ('binary');

Comment 4

[Alervd]

Writing a data to stdin works in a similar way: serialized JSON with non-English strings came to my EXE completely damaged. It even is not in UTF8. It just damaged (e.g. for my 4-chars non-English string it sends me 4-chars English string with some random letters). I use event/core emit to send a data.
Because stdin.write API just does not exists.
(https://nodejs.org/api/stream.html#stream_writable_write_chunk_encoding_callback)
How can you propose the developers to use such a buggy API instead of binary components?

Comment 5

[Alervd]

Recently I decided to use URI encoding to force addon sdk api to send data between extension and exe "as is". But now I've discovered one yet bug: stdin stream gets closed and my exe exists in case the original buffer contains already uri encoded data in some parts of json.
I.e.:
send_data (uri_encode(uri_encode(data)).
Now exceptions thrown. Just stream closed and nothing can be send to exe anymore.
The only way remains for me currently: use base64 encoding. 
A very buggy part of the addon sdk.

Comment 6

[Alervd]

Some typos... :
my exe exists --> my exe exits
Now exceptions thrown --> NO exceptions thrown

Comment 7

[gorf4673]

Wonder if this is an option/configuration issue.

I have just been finding that binary stream  wasn't working for me either by using stream.setEncoding
Looking through source, spawn/Child calling subprocess and it has doc recommending charset null for binary.

 * charset:     Output is decoded with given charset and a string is returned.
 *              If charset is undefined, "UTF-8" is used as default.
 *              To get binary data, set this to null and the returned string
 *              is not decoded in any way.
 *

spawn creates options with default encoding:'UTF-8'
Child class which creates option charset: options.encoding,  to pass to subprocess.

    let options = {
      encoding: null
    };// encoding:binary
    child_process = spawn(executable,null,options);

Comment 8

[Alervd]

Encoding has no effect at all.
Current version of my extension works in current stable version of Firefox.
But it does not in Developer version. 
Extension sends 00 00 00 B5. My EXE receives 00 00 C2 B5.

Comment 9

[Duan Yao]

In Firefox 46 on Windows, this works for me:
  var proc = child_process.spawn(cmd, args, {encoding: 'GBK'});

However

  var proc = child_process.spawn(cmd, args);
  proc.stdout.setEncoding('GBK');

Does not work.

In Firefox 52.0a2 (2017-01-11) on Windows, both don't work -- seems a regression.

Comment 10

[Alervd]

It worked fine until 51 version of Firefox.

Comment 11

[BMO Automation]

Add-on SDK is no longer supported so resolving bugs as INCOMPLETE