1116229 - talos compare.py safe guardings and cleanup

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

with the recent updates in compare.py we have done a lot of great cleanup.  Some future changes could pass different parameters to functions.  Here is a list of things to adjust:
* remove comment: http://hg.mozilla.org/build/talos/file/tip/talos/compare.py#l32
* ensure test exists before removing: http://hg.mozilla.org/build/talos/file/tip/talos/compare.py#l87 (for both conditions in this function, maybe some refactoring could be done to make it more extensible)
* add tp5n as a valid test, we reference it here: http://hg.mozilla.org/build/talos/file/tip/talos/compare.py#l200. just verify calling with tp5n and no results coming back, it is needed for xperf related data.

Comment 1

[Vaibhav (:vaibhav1994)]

Attachment: cleanup.patch

These are the outputs after making the changes in the file:

1) tp5n is back and media_tests not run for OSX64

vaibhav@vaibhav:~/talos/talos$ python compare.py --rev 8991815dd086 --branch Firefox --masterbranch Firefox --platform OSX64 --verbose
   test			min.	->	max.	rev.
OSX64:

   tresize           	   20.5	->	   21.5	   20.9
   kraken            	 2324.4	->	 2389.8	 2342.5
   v8_7              	12385.3	->	12786.8	12558.7
   dromaeo_css       	 4187.0	->	 4575.5	 4454.6
   dromaeo_dom       	  759.0	->	  831.0	  792.3
   a11yr             	  337.0	->	  356.0	  352.0
   ts_paint          	 1200.9	->	 1238.3	 1217.0
   tpaint            	  382.5	->	  411.3	  392.6
   tsvgr_opacity     	  443.5	->	  458.0	  445.5
   tp5n              	No data for platform
   tp5o              	  317.3	->	  327.1	  320.6
   tart              	   11.6	->	   17.6	   11.9
   tcanvasmark       	 5808.0	->	 5995.0	 5993.5
   tsvgx             	  523.5	->	  527.9	  524.3
   tscrollx          	    6.7	->	    6.9	    6.7
:) sessionrestore    	 2281.0	->	 2348.6	 2273.6
   sessionrestore_no_auto_restore	  983.6	->	 1026.3	  999.2
   glterrain         	    7.3	->	    8.2	    7.7
   cart              	   48.2	->	   67.3	   49.0
   tp5o_scroll       	    7.8	->	    8.0	    7.9

2) kraken, v8_7, dromaeo_css, dromaeo_dom not run on WinXP

vaibhav@vaibhav:~/talos/talos$ python compare.py --rev 8991815dd086 --branch Firefox --masterbranch Firefox --platform WinXP --verbose
   test			min.	->	max.	rev.
WinXP:

   tresize           	   11.7	->	   12.9	   12.6
   a11yr             	  307.0	->	  332.5	  313.5
   ts_paint          	  770.5	->	  799.9	  779.5
   tpaint            	  189.6	->	  205.0	  190.7
:( tsvgr_opacity     	  443.0	->	  461.0	  465.0
   tp5n              	No data for platform
:( tp5o              	  273.0	->	  283.2	  284.3
   tart              	    6.2	->	    9.3	    6.2
   tcanvasmark       	 6420.0	->	 6679.0	 6579.5
   tsvgx             	  576.0	->	  613.0	  590.8
   tscrollx          	    2.4	->	    3.1	    2.4
   sessionrestore    	 1604.6	->	 1642.0	 1621.8
   sessionrestore_no_auto_restore	  622.9	->	  638.0	  624.9
   glterrain         	   21.9	->	   22.2	   22.2
   cart              	   38.5	->	   58.8	   39.8
   tp5o_scroll       	    3.6	->	    3.8	    3.7

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8545263 [details] [diff] [review]
cleanup.patch

Review of attachment 8545263 [details] [diff] [review]:
-----------------------------------------------------------------

very nice, thanks!

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

https://hg.mozilla.org/build/talos/rev/3921f6afbd84