If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

talos compare.py safe guardings and cleanup

RESOLVED FIXED

Status

Testing
Talos
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: jmaher, Assigned: vaibhav1994)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

3 years ago
with the recent updates in compare.py we have done a lot of great cleanup.  Some future changes could pass different parameters to functions.  Here is a list of things to adjust:
* remove comment: http://hg.mozilla.org/build/talos/file/tip/talos/compare.py#l32
* ensure test exists before removing: http://hg.mozilla.org/build/talos/file/tip/talos/compare.py#l87 (for both conditions in this function, maybe some refactoring could be done to make it more extensible)
* add tp5n as a valid test, we reference it here: http://hg.mozilla.org/build/talos/file/tip/talos/compare.py#l200. just verify calling with tp5n and no results coming back, it is needed for xperf related data.
(Assignee)

Comment 1

3 years ago
Created attachment 8545263 [details] [diff] [review]
cleanup.patch

These are the outputs after making the changes in the file:

1) tp5n is back and media_tests not run for OSX64

vaibhav@vaibhav:~/talos/talos$ python compare.py --rev 8991815dd086 --branch Firefox --masterbranch Firefox --platform OSX64 --verbose
   test			min.	->	max.	rev.
OSX64:

   tresize           	   20.5	->	   21.5	   20.9
   kraken            	 2324.4	->	 2389.8	 2342.5
   v8_7              	12385.3	->	12786.8	12558.7
   dromaeo_css       	 4187.0	->	 4575.5	 4454.6
   dromaeo_dom       	  759.0	->	  831.0	  792.3
   a11yr             	  337.0	->	  356.0	  352.0
   ts_paint          	 1200.9	->	 1238.3	 1217.0
   tpaint            	  382.5	->	  411.3	  392.6
   tsvgr_opacity     	  443.5	->	  458.0	  445.5
   tp5n              	No data for platform
   tp5o              	  317.3	->	  327.1	  320.6
   tart              	   11.6	->	   17.6	   11.9
   tcanvasmark       	 5808.0	->	 5995.0	 5993.5
   tsvgx             	  523.5	->	  527.9	  524.3
   tscrollx          	    6.7	->	    6.9	    6.7
:) sessionrestore    	 2281.0	->	 2348.6	 2273.6
   sessionrestore_no_auto_restore	  983.6	->	 1026.3	  999.2
   glterrain         	    7.3	->	    8.2	    7.7
   cart              	   48.2	->	   67.3	   49.0
   tp5o_scroll       	    7.8	->	    8.0	    7.9

2) kraken, v8_7, dromaeo_css, dromaeo_dom not run on WinXP

vaibhav@vaibhav:~/talos/talos$ python compare.py --rev 8991815dd086 --branch Firefox --masterbranch Firefox --platform WinXP --verbose
   test			min.	->	max.	rev.
WinXP:

   tresize           	   11.7	->	   12.9	   12.6
   a11yr             	  307.0	->	  332.5	  313.5
   ts_paint          	  770.5	->	  799.9	  779.5
   tpaint            	  189.6	->	  205.0	  190.7
:( tsvgr_opacity     	  443.0	->	  461.0	  465.0
   tp5n              	No data for platform
:( tp5o              	  273.0	->	  283.2	  284.3
   tart              	    6.2	->	    9.3	    6.2
   tcanvasmark       	 6420.0	->	 6679.0	 6579.5
   tsvgx             	  576.0	->	  613.0	  590.8
   tscrollx          	    2.4	->	    3.1	    2.4
   sessionrestore    	 1604.6	->	 1642.0	 1621.8
   sessionrestore_no_auto_restore	  622.9	->	  638.0	  624.9
   glterrain         	   21.9	->	   22.2	   22.2
   cart              	   38.5	->	   58.8	   39.8
   tp5o_scroll       	    3.6	->	    3.8	    3.7
Attachment #8545263 - Flags: review?(jmaher)
(Assignee)

Updated

3 years ago
Assignee: nobody → vaibhavmagarwal
(Reporter)

Comment 2

3 years ago
Comment on attachment 8545263 [details] [diff] [review]
cleanup.patch

Review of attachment 8545263 [details] [diff] [review]:
-----------------------------------------------------------------

very nice, thanks!
Attachment #8545263 - Flags: review?(jmaher) → review+
(Reporter)

Comment 3

3 years ago
https://hg.mozilla.org/build/talos/rev/3921f6afbd84
(Reporter)

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.