Open Bug 1466525 Opened 2 years ago Updated 1 year ago
Can't compare try pushes using `mozharness: --gecko
When pushing to try with `mozharness: --geckoProfile`, DAMP no longer uploads `perfherder-data.json` artifact, nor generates PERFHERDER_DATA in its output. Thus we can no longer compare builds that are profiled. This isn't handy as it doesn't allow to know if a regression that we are tracking without the profiler, also reproduces with the profiler. Here is two example try pushes: https://treeherder.mozilla.org/#/jobs?repo=try&revision=0b5b8629a272b5b763f5a9c8b7cafacc5bf4c841&selectedJob=181693848 https://treeherder.mozilla.org/#/jobs?repo=try&revision=d8e543e21997254f9cedf6f86145a403c39839dc
this is by design- a profiled build will produce misleading data and can skew our results for alerts and trends. Why are you trying to compare results with --geckoProfile?
(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #1) > this is by design- a profiled build will produce misleading data and can > skew our results for alerts and trends. Why are you trying to compare > results with --geckoProfile? I totally agree that the data would be misleading, especially when comparing with a non-profiled one. But it could also help ensuring that a particular regression also reproduces with the profiler turned on. Honza originaly reported that to me when looking at a hard to debug regression on DAMP, on a "settle" subtest regression that involved spurious asynchronous code overlapping between two subtests.
maybe for try runs we could allow it to upload data with profile on
(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #3) > maybe for try runs we could allow it to upload data with profile on Yes, this would help. The perf/damp issue I am debugging is only reproducible on Try and I wanted to also get profile data to analyze it using perf.html Honza
Priority: -- → P3
You need to log in before you can comment on or make changes to this bug.