Henrik, i had a discussion with Dave Hunt about these devices and there will be an upgrade in Q1 and we can see more clearly if the new hardware will solve these issues. Here is another discussion from yesterday, (#perftest on slack) : davehunt : @jmaher happy new year! do you know how many Apple Mac Minis we have for perf testing? jmaher : @davehunt about 450 it is a shared pool with the unittests @davehunt on a related note, the mac mini pool is out of warranty and there is new hardware to replace it- this is upgraded specs as that is what is available; timeline is Q1 to replace- probably February --------------------------------------------- I also ran the youtube-playback-tests as "--power-test" to force the MAC jobs to run on MacBook Pro, - more details here: https://bugzilla.mozilla.org/show_bug.cgi?id=1604207#c0 - there are some errors for Chrome, because the app is not found: https://searchfox.org/mozilla-central/rev/4537228c0a18bc0ebba2eb7f5cbebb6ea9ab211c/testing/mozharness/mozharness/mozilla/testing/raptor.py#414-420 - i think i could add a new condtition like "elif 'mbpro' in self.platform_name():" But at least we have some results for Firefox: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&tier=1%2C2%2C3&revision=2b7e94a872ff9b92dddd25ecce10383d9d85464f The workerID contains "mbpro", so i assume the tests are running on MacBook Pro: (Selected an ytp job -> Task link -> Worker Type: https://firefox-ci-tc.services.mozilla.com/provisioners/releng-hardware/worker-types/gecko-t-osx-1014-power) The ytp score is not displayed under Performance Tab for ytp power tests, but i did these steps to get the results: From "OS X 10.14 Shippable opt" selected the ytp jobs Clicked on "Job Details" tab at the bottom Clicked on perfherder-data.json Clicked on "Collapse All" The values for suites[0]['value'] field are: 1. 35.64 2. 34.84 3. 33.85 4. 36.9 5. 33.42 6. 35.92 7. 36.58 8. 34.57 The results are still higher than other platforms but lower compared to regular "mac OS 10" on Firefox, (https://firefox-ci-tc.services.mozilla.com/provisioners/releng-hardware/worker-types/gecko-t-osx-1014) where we have 50 - 60 values: https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&series=mozilla-central,2054686,1,10&series=mozilla-central,2044136,1,10&series=mozilla-central,2044425,1,10&series=mozilla-central,2045003,1,10&series=mozilla-central,2043847,1,10&series=mozilla-central,2044714,1,10&series=mozilla-central,2043558,1,10&timerange=2592000
Bug 1576107 Comment 16 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Henrik, i had a discussion with Dave Hunt about these devices and there will be an upgrade in Q1 and we can see more clearly if the new hardware will solve these issues. Here is another discussion from yesterday, (#perftest on slack) : davehunt : @jmaher happy new year! do you know how many Apple Mac Minis we have for perf testing? jmaher : @davehunt about 450 it is a shared pool with the unittests @davehunt on a related note, the mac mini pool is out of warranty and there is new hardware to replace it- this is upgraded specs as that is what is available; timeline is Q1 to replace- probably February --------------------------------------------- I also ran the youtube-playback-tests as "--power-test" to force the MAC jobs to run on MacBook Pro, - more details here: https://bugzilla.mozilla.org/show_bug.cgi?id=1604207#c0 - there are some errors for Chrome, because the app is not found: https://searchfox.org/mozilla-central/rev/4537228c0a18bc0ebba2eb7f5cbebb6ea9ab211c/testing/mozharness/mozharness/mozilla/testing/raptor.py#414-420 - i think i could add a new condtition like "elif 'mbpro' in self.platform_name():" But at least we have some results for Firefox: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&tier=1%2C2%2C3&revision=2b7e94a872ff9b92dddd25ecce10383d9d85464f The workerID contains "mbpro", so i assume the tests are running on MacBook Pro: (Selected an ytp job -> Task link -> Worker Type: https://firefox-ci-tc.services.mozilla.com/provisioners/releng-hardware/worker-types/gecko-t-osx-1014-power) The ytp score is not displayed under Performance Tab for ytp power tests, but i did these steps to get the results: From "OS X 10.14 Shippable opt" selected the ytp jobs Clicked on "Job Details" tab at the bottom Clicked on perfherder-data.json Clicked on "Collapse All" The values for suites[0]['value'] field are: 1. 35.64 2. 34.84 3. 33.85 4. 36.9 5. 33.42 6. 35.92 7. 36.58 8. 34.57 The results are still higher than other platforms but lower compared to regular "mac OS 10" on Firefox, (https://firefox-ci-tc.services.mozilla.com/provisioners/releng-hardware/worker-types/gecko-t-osx-1014) where we have values between 50 - 60: https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&series=mozilla-central,2054686,1,10&series=mozilla-central,2044136,1,10&series=mozilla-central,2044425,1,10&series=mozilla-central,2045003,1,10&series=mozilla-central,2043847,1,10&series=mozilla-central,2044714,1,10&series=mozilla-central,2043558,1,10&timerange=2592000