Bugzilla

Hi Alex! Sorry for the multiple NIs, this is my first rodeo with performance regressions. I wanted to confirm if I'm going about this correctly and clarify if I'm looking at the correct metrics.

I've done several try runs with `browsertime-first-install-firefox-welcome` removing bits of the code and comparing it to the regressions chart you created above. 

I then compared one of the cases `linux1804-64-shippable-qr` with the corresponding before load time values. Each reversion still showed similar after values i.e.:
```
134.8	ms	Lower	
welcome fcp opt cold fission webrender
```

But what's confusing is that when comparing the completely reverted patch with the original patch, I'm not seeing the same before ratios. As a sanity check, I did another try run with the original patch and then with it completely reverted.

[original patch](https://treeherder.mozilla.org/jobs?repo=try&author=nsauermann%40mozilla.com&selectedTaskRun=LjImoJK0Tneia4heawJ5WQ.0)
```
127.9	ms	Lower	
welcome fcp opt cold fission webrender
```

[completely reverted try run](https://treeherder.mozilla.org/jobs?repo=try&author=nsauermann%40mozilla.com&selectedTaskRun=fjPvA2O0RIya5UxGVVQ3Dg.0)
```130.2	ms	Lower	
welcome fcp opt cold fission webrender```



So I'm wondering if I'm debugging this incorrectly and if you have any pointers? Or if there's an explanation for the difference in ratios for the before/afters.

Hi Alex! Sorry for the multiple NIs, this is my first rodeo with performance regressions. I wanted to confirm if I'm going about this correctly and clarify if I'm looking at the correct metrics.

I've done several try runs with `browsertime-first-install-firefox-welcome` removing bits of the code and comparing it to the regressions chart you created above. 

I then compared one of the cases `linux1804-64-shippable-qr` with the corresponding before load time values. Each reversion still showed similar after values i.e.:
```
134.8	ms	Lower	
welcome fcp opt cold fission webrender
```

But what's confusing is that when comparing the completely reverted patch with the original patch, I'm not seeing the same before ratios. As a sanity check, I did another try run with the original patch and then with it completely reverted.

[original patch](https://treeherder.mozilla.org/jobs?repo=try&author=nsauermann%40mozilla.com&selectedTaskRun=LjImoJK0Tneia4heawJ5WQ.0)

```
127.9	ms	Lower	
welcome fcp opt cold fission webrender
```

[completely reverted try run](https://treeherder.mozilla.org/jobs?repo=try&author=nsauermann%40mozilla.com&selectedTaskRun=fjPvA2O0RIya5UxGVVQ3Dg.0)

```130.2	ms	Lower	
welcome fcp opt cold fission webrender
```



So I'm wondering if I'm debugging this incorrectly and if you have any pointers? Or if there's an explanation for the difference in ratios for the before/afters.

Hi Alex! Sorry for the multiple NIs, this is my first rodeo with performance regressions. I wanted to confirm if I'm going about this correctly and clarify if I'm looking at the correct metrics.

I've done several try runs with `browsertime-first-install-firefox-welcome` removing bits of the code and comparing it to the regressions chart you created above. 

I then compared one of the cases `linux1804-64-shippable-qr` with the corresponding before load time values. Each reversion still showed similar after values i.e.:
```
134.8	ms	
welcome fcp opt cold fission webrender
```

But what's confusing is that when comparing the completely reverted patch with the original patch, I'm not seeing the same before ratios. As a sanity check, I did another try run with the original patch and then with it completely reverted.

[original patch](https://treeherder.mozilla.org/jobs?repo=try&author=nsauermann%40mozilla.com&selectedTaskRun=LjImoJK0Tneia4heawJ5WQ.0)

```
127.9	ms	
welcome fcp opt cold fission webrender
```

[completely reverted try run](https://treeherder.mozilla.org/jobs?repo=try&author=nsauermann%40mozilla.com&selectedTaskRun=fjPvA2O0RIya5UxGVVQ3Dg.0)

```130.2	ms	
welcome fcp opt cold fission webrender
```



So I'm wondering if I'm debugging this incorrectly and if you have any pointers? Or if there's an explanation for the difference in ratios for the before/afters.

Hi Alex! Sorry for the multiple NIs, this is my first rodeo with performance regressions. I wanted to confirm if I'm going about this correctly and clarify if I'm looking at the correct metrics.

I've done several try runs with `browsertime-first-install-firefox-welcome` removing bits of the code and comparing it to the regressions chart you created above. 

I then compared one of the cases `linux1804-64-shippable-qr` with the corresponding before load time values. Each reversion still showed similar after values i.e.:
```
134.8	ms	
welcome fcp opt cold fission webrender
```

But what's confusing is that when comparing the completely reverted patch with the original patch, I'm not seeing the same before ratios. As a sanity check, I did another try run with the original patch and then with it completely reverted.

[original patch](https://treeherder.mozilla.org/jobs?repo=try&author=nsauermann%40mozilla.com&selectedTaskRun=LjImoJK0Tneia4heawJ5WQ.0)

```
127.9	ms	
welcome fcp opt cold fission webrender
```

[completely reverted try run](https://treeherder.mozilla.org/jobs?repo=try&author=nsauermann%40mozilla.com&selectedTaskRun=fjPvA2O0RIya5UxGVVQ3Dg.0)

```
130.2	ms	
welcome fcp opt cold fission webrender
```



So I'm wondering if I'm debugging this incorrectly and if you have any pointers? Or if there's an explanation for the difference in ratios for the before/afters.