update avmdiff.py, if all configs match return the output, otherwise return the diff there are still a few cases of scrubbing: (1) strip runtime errors after Error: Error #+ since release does not show error detail (2) strip stack traces from runtime errors, only Debugger builds show stacktraces (3) strip all lines '^metric', all time related or changing output can be prefixed with metric
avmdiff.py does not pass all vm args to each vm, only passes the .abc name e.g. avmdiff.py -memstats test.abc should pass -memstats to each vm
Created attachment 465668 [details] [diff] [review] have avmdiff pass command line options to avmshell There is a slight issue with patch when running avmdiff from the command line, if passing in a value for buildfile, the buildfile switch must use an =: avmdiff --buildfile=file.txt using a space will pass file.txt to avmshell
Assignee: nobody → cpeyer
Status: NEW → ASSIGNED
Attachment #465668 - Flags: review?(dschaffe)
Comment on attachment 465668 [details] [diff] [review] have avmdiff pass command line options to avmshell looks ok assuming all of the differential testing pass in the sandbox. We will still have to skip tests when the vm args are not supported on all shells. For example debugger shells have many vm args not supported on release shells. The release shells will return known argument errors. There are probably several testcases where these errors are hidden since currently the vm args are ignored by avmdiff. running a sandbox build will show these problems.
Attachment #465668 - Flags: review?(dschaffe) → review+
Patch pushed to redux: http://hg.mozilla.org/tamarin-redux/rev/723838f07125
OS: Mac OS X → Windows 7
Created attachment 466718 [details] [diff] [review] Followup fix for --buildfile usage Previous patch didn't work well with the buildfile option. This patch actually removes the vmargs var since the loop beforehand extracts out all of the vm args (and testname) into avmshell_args. Also adds ability to now correctly pass application arguments to the testcase (using " -- ").
Attachment #466718 - Flags: review?(brbaker)
Comment on attachment 466718 [details] [diff] [review] Followup fix for --buildfile usage '-f' switch does not work. The switch is currently not in use via the build system so the short form could either be removed or fixed, makes no difference to me. Currently using -f will fail to work and will just return the usage - update the usage output to show the required equals sign in the buildfile switch '--buildfile='
Attachment #466718 - Flags: review?(brbaker) → review-
Created attachment 467046 [details] [diff] [review] update avmdiff patch based on brents feedback Removed -f option. Added comments regarding what options are allowable in avmdiff. Added = to end of buildfile switch. Added a missing continue in for args loop for test_args.
Created attachment 467047 [details] [diff] [review] One more update duh - no need for continue, just branch on if / elif
pushed to redux: http://hg.mozilla.org/tamarin-redux/rev/724973cb001f
Status: ASSIGNED → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.