(In reply to :Gijs (he/him) from comment #6) > (In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #5) > > (In reply to :Gijs (he/him) from comment #4) > > > https://treeherder.mozilla.org/#/jobs?repo=autoland&tochange=b5656f3353165e991e563acd3a0b276df4de7e65&searchStr=g5%2Cshippable%2Clinux&fromchange=ddb7366f09a390b17b25984be1681026143ac679 > > > > > > https://treeherder.mozilla.org/perf.html#/graphs?series=autoland,1927254,1,1&zoom=1558723638479.2905,1558736569901.2415,1.5094566727599597,1.8011370161509457&selected=autoland,1927254,482399,819892708,1 > > > > > > This looks to me like it's bug 1542746 instead, which landed right before the backout. That would make a bunch more sense. Ionuț, can you confirm? > > > > Hmmm... I agree with you. Especially since this regression didn't affect any other platforms. Could we conclude that this so called "regression" was actually caused by a temporary misbehaving PGO build process? And close the bug as WONTFIX? > > I'm confused - has the regression disappeared again? I thought it was still there. And bug 1542746 was a deliberate change to how we do PGO, and it's possible there's a real regression caused by that, right? No, the regression is still there & it's real. But bug 1542746 didn't cause it directly. As you said:i t only changed the way we do PGO. With this in mind, when bug 1552565 got backed out, somehow the updated PGO build process produced binaries which regress on **tp5o_webext responsiveness**. From my experience, PGO-induced regressions like these are just temporary. At some point, the produced binaries come back to original baseline by themselves. Situations like these happen rarely, but constantly (every 1-2 months); me & :jmaher call these **broken PGO builds**. Once we're pretty sure about them, we close bugs as WONTFIX.
Bug 1555268 Comment 7 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
(In reply to :Gijs (he/him) from comment #6) > (In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #5) > > (In reply to :Gijs (he/him) from comment #4) > > > https://treeherder.mozilla.org/#/jobs?repo=autoland&tochange=b5656f3353165e991e563acd3a0b276df4de7e65&searchStr=g5%2Cshippable%2Clinux&fromchange=ddb7366f09a390b17b25984be1681026143ac679 > > > > > > https://treeherder.mozilla.org/perf.html#/graphs?series=autoland,1927254,1,1&zoom=1558723638479.2905,1558736569901.2415,1.5094566727599597,1.8011370161509457&selected=autoland,1927254,482399,819892708,1 > > > > > > This looks to me like it's bug 1542746 instead, which landed right before the backout. That would make a bunch more sense. Ionuț, can you confirm? > > > > Hmmm... I agree with you. Especially since this regression didn't affect any other platforms. Could we conclude that this so called "regression" was actually caused by a temporary misbehaving PGO build process? And close the bug as WONTFIX? > > I'm confused - has the regression disappeared again? I thought it was still there. And bug 1542746 was a deliberate change to how we do PGO, and it's possible there's a real regression caused by that, right? No, the regression is still there & it's real. But bug 1542746 didn't cause it directly. As you said: it only changed the way we do PGO. With this in mind, when bug 1552565 got backed out, somehow the updated PGO build process produced binaries which regress on **tp5o_webext responsiveness**. From my experience, PGO-induced regressions like these are just temporary. At some point, the produced binaries come back to original baseline by themselves. Situations like these happen rarely, but constantly (every 1-2 months); me & :jmaher call these **broken PGO builds**. Once we're pretty sure about them, we close bugs as WONTFIX.
(In reply to :Gijs (he/him) from comment #6) > (In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #5) > > (In reply to :Gijs (he/him) from comment #4) > > > https://treeherder.mozilla.org/#/jobs?repo=autoland&tochange=b5656f3353165e991e563acd3a0b276df4de7e65&searchStr=g5%2Cshippable%2Clinux&fromchange=ddb7366f09a390b17b25984be1681026143ac679 > > > > > > https://treeherder.mozilla.org/perf.html#/graphs?series=autoland,1927254,1,1&zoom=1558723638479.2905,1558736569901.2415,1.5094566727599597,1.8011370161509457&selected=autoland,1927254,482399,819892708,1 > > > > > > This looks to me like it's bug 1542746 instead, which landed right before the backout. That would make a bunch more sense. Ionuț, can you confirm? > > > > Hmmm... I agree with you. Especially since this regression didn't affect any other platforms. Could we conclude that this so called "regression" was actually caused by a temporary misbehaving PGO build process? And close the bug as WONTFIX? > > I'm confused - has the regression disappeared again? I thought it was still there. And bug 1542746 was a deliberate change to how we do PGO, and it's possible there's a real regression caused by that, right? No, the regression is still there & it's real. But bug 1542746 didn't cause it directly. As you said: it only changed the way we do PGO. With this in mind, when bug 1552565 got backed out, somehow the updated PGO build process produced binaries which regress on **tp5o_webext responsiveness**. From my experience, PGO-induced regressions like these are just temporary. At some point, the produced binaries come back to original baseline by themselves. Situations like these happen rarely, but constantly (every 1-2 months); me & :jmaher call these **broken PGO builds**. Once we're pretty sure about them, we close these bugs as WONTFIX.
(In reply to :Gijs (he/him) from comment #6) > (In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #5) > > (In reply to :Gijs (he/him) from comment #4) > > > https://treeherder.mozilla.org/#/jobs?repo=autoland&tochange=b5656f3353165e991e563acd3a0b276df4de7e65&searchStr=g5%2Cshippable%2Clinux&fromchange=ddb7366f09a390b17b25984be1681026143ac679 > > > > > > https://treeherder.mozilla.org/perf.html#/graphs?series=autoland,1927254,1,1&zoom=1558723638479.2905,1558736569901.2415,1.5094566727599597,1.8011370161509457&selected=autoland,1927254,482399,819892708,1 > > > > > > This looks to me like it's bug 1542746 instead, which landed right before the backout. That would make a bunch more sense. Ionuț, can you confirm? > > > > Hmmm... I agree with you. Especially since this regression didn't affect any other platforms. Could we conclude that this so called "regression" was actually caused by a temporary misbehaving PGO build process? And close the bug as WONTFIX? > > I'm confused - has the regression disappeared again? I thought it was still there. And bug 1542746 was a deliberate change to how we do PGO, and it's possible there's a real regression caused by that, right? No, the regression is still there & it's real. But bug 1542746 didn't cause it directly. As you said: it only changed the way we do PGO. With this in mind, when bug 1552565 got backed out, somehow the updated PGO build process produced binaries which regress on **tp5o_webext responsiveness**. From my experience, PGO-induced regressions like these are just temporary. At some point, the produced binaries come back to original baseline by themselves. Situations like these happen rarely, but constantly (every 1-2 months); me & :jmaher call these **broken PGO builds**. Once we're confident that this is the case, we close these bugs as WONTFIX.
(In reply to :Gijs (he/him) from comment #6) > (In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #5) > > (In reply to :Gijs (he/him) from comment #4) > > > https://treeherder.mozilla.org/#/jobs?repo=autoland&tochange=b5656f3353165e991e563acd3a0b276df4de7e65&searchStr=g5%2Cshippable%2Clinux&fromchange=ddb7366f09a390b17b25984be1681026143ac679 > > > > > > https://treeherder.mozilla.org/perf.html#/graphs?series=autoland,1927254,1,1&zoom=1558723638479.2905,1558736569901.2415,1.5094566727599597,1.8011370161509457&selected=autoland,1927254,482399,819892708,1 > > > > > > This looks to me like it's bug 1542746 instead, which landed right before the backout. That would make a bunch more sense. Ionuț, can you confirm? > > > > Hmmm... I agree with you. Especially since this regression didn't affect any other platforms. Could we conclude that this so called "regression" was actually caused by a temporary misbehaving PGO build process? And close the bug as WONTFIX? > > I'm confused - has the regression disappeared again? I thought it was still there. And bug 1542746 was a deliberate change to how we do PGO, and it's possible there's a real regression caused by that, right? No, the regression is still there & it's real. But bug 1542746 didn't cause it directly. As you said: it only changed the way we do PGO. With this in mind, when bug 1552565 got backed out, somehow the updated PGO build process produced binaries which regress on **tp5o_webext responsiveness**. From my experience, PGO-induced regressions like these are just temporary. At some point, the produced binaries come back to original baseline by themselves. Situations like these happen rarely, but constantly (every 1-2 months); me & :jmaher call these **broken PGO builds**. Once we're confident that this is the case, we close these bugs as WONTFIX. In this case, I'm confident.
(In reply to :Gijs (he/him) from comment #6) > (In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #5) > > (In reply to :Gijs (he/him) from comment #4) > > > https://treeherder.mozilla.org/#/jobs?repo=autoland&tochange=b5656f3353165e991e563acd3a0b276df4de7e65&searchStr=g5%2Cshippable%2Clinux&fromchange=ddb7366f09a390b17b25984be1681026143ac679 > > > > > > https://treeherder.mozilla.org/perf.html#/graphs?series=autoland,1927254,1,1&zoom=1558723638479.2905,1558736569901.2415,1.5094566727599597,1.8011370161509457&selected=autoland,1927254,482399,819892708,1 > > > > > > This looks to me like it's bug 1542746 instead, which landed right before the backout. That would make a bunch more sense. Ionuț, can you confirm? > > > > Hmmm... I agree with you. Especially since this regression didn't affect any other platforms. Could we conclude that this so called "regression" was actually caused by a temporary misbehaving PGO build process? And close the bug as WONTFIX? > > I'm confused - has the regression disappeared again? I thought it was still there. And bug 1542746 was a deliberate change to how we do PGO, and it's possible there's a real regression caused by that, right? No, the regression is still there & it's real. But bug 1542746 didn't cause it directly. As you said: it only changed the way we do PGO. With this in mind, when bug 1552565 got backed out, somehow the updated PGO build process produced binaries which regress on **tp5o_webext responsiveness**. From my experience, PGO-induced regressions like these are just temporary. At some point, the produced binaries come back to original baseline by themselves. Situations like these happen rarely, but constantly (every 1-2 months); me & :jmaher call these **broken PGO builds**. Once we're confident that this is the case, we close these bugs as WONTFIX. In our case, I'm confident.