588557 - minimize json2 memory usage

Status: RESOLVED FIXED

(Webtools Graveyard :: Tinderbox, defect)

Description

[Robert Helmer [:rhelmer]]

json2 was backed out after being put into production due to excessive memory usage (bug 588315). Let's see where it's taking so much memory and make it better.

Comment 1

[Robert Helmer [:rhelmer]]

Attachment: print each entry instead of building data structure

Print out the data as we go instead of building a (potentially very large) data structure.

Unfortunately this means we're back to building JSON output by hand but only for the begin and end, still using JSON->encode() as much as possible.

Comment 2

[Robert Helmer [:rhelmer]]

To confirm this is fixed, I am thinking we could just sample memory usage on tinderbox-stage while hitting it, something like:

while true; do sleep 0.5; ps -e -o rss,cmd; done

We could get more hardcore if we really want to get peak memory usage for each process, and make sure the process is really the one we want etc

However if this is really what caused a problem on production, it should be quite obvious from sampling before/after any fixes while load is happening on tinderbox-stage, whether the problem is getting better or not.

Comment 3

[Robert Helmer [:rhelmer]]

Checking in tbglobals.pl;
/cvsroot/mozilla/webtools/tinderbox/tbglobals.pl,v  <--  tbglobals.pl
new revision: 1.80; previous revision: 1.79
done
Checking in showbuilds.pl;
/cvsroot/mozilla/webtools/tinderbox/showbuilds.pl,v  <--  showbuilds.pl
new revision: 1.44; previous revision: 1.43
done

Comment 4

[Robert Helmer [:rhelmer]]

Attachment: only print leading comma if buildname is set

I just noticed this testing production (which has attachment 467212 [details] [diff] [review]); I haven't been able to reproduce this in dev yet but I am pretty sure this is because load_buildlog is putting in these fake entries that we don't want, and adding the comma should be after we check for this and call "next;" not before.

Comment 5

[Robert Helmer [:rhelmer]]

Checking in tbglobals.pl;
/cvsroot/mozilla/webtools/tinderbox/tbglobals.pl,v  <--  tbglobals.pl
new revision: 1.81; previous revision: 1.80
done