559856 - Inefficient use of the sql array in xref-scripts/dxr.js

Status: RESOLVED WORKSFORME

(Webtools Graveyard :: DXR, defect)

Description

[Jean-Marc Desperrier]

Trying to use DXR on a non-mozilla source, I ended up with an OOM error at dxr.js:379 when compiling a large auto-generated file.

Maybe the only reason it doesn't happen on the Mozilla machines is that they have a lot of memory, because some files in the Mozilla source are very large.

dxr.js:379 is this code :
function input_end() {
  // This assumes |sort -u| will be called to get INSERT and UPDATES in right 
  // order
  write_file (sys.aux_base_name + ".sql", sql.join("\n") + "\n");

dxr.js defines at start :
  var sql = [];

Then (if I'm not mistaken) the only way sql is accessed is by doing a number of :
   sql.push( some string content )

And at the end, as seen above, when input_end() is called, a join operation is made on the whole content of sql, and the result is then written to disk.

This is very inefficient for large files. I suppose the only reason the code is written like that is that initially the sql array was sorted before writing it to disk.

I think it would be much better to directly write the generated strings to the output file, and simply get rid of the sql array.

Comment 1

[Jean-Marc Desperrier]

Hum, maybe the other reason is that Dehydra does not provide the functions needed to open a file handle, make several write call to that handle, and close it when you need ?

I think it just needs to be added. Or then we should use JS-ctypes.

Comment 2

[(dormant account)]

DXR got rewritten since this was filed. Reopen if it's still an issue