The bugbot-auto-crash regressor-identification sometimes misfires and picks a regressor that landed substantially after the crash-volume-signal that it's trying to account for
Categories
(Developer Infrastructure :: Source Code Analysis, task)
Tracking
(Not tracked)
People
(Reporter: dholbert, Unassigned)
References
Details
BugBot did something kind of silly when filing bug 1863178. (Not sure to-what-extent-we-can-address-this, but I figured I'd file... Also note, I'm trying to read between the lines about its reasoning; it's possible I'm mis-stating something here.)
Here's what BugBot did:
- It decided that 4 crashes on Nightly (for a signature that previously had no nightly crash volume) was an interesting signal. (So far, this seems reasonable.)
- It identified a "potential regressor" that landed right in the middle of that small cluster of 4 crashes, and flagged that as the regressor bug. (This part is a bit silly).
i.e. the potential regressor could hypothetically account for the latter two crashes, but not the former two which occurred before the supposed regressor had landed.
Maybe there's something we could do to improve the bugbot regression-analysis here? (I'm not sure what sorts of heuristics it's using to identify the regressor, but it might be worth tuning them to be sure they can account for the start of whatever signal that BugBot is identifying.)
|
||
Comment 1•2 years ago
•
|
||
Thank you for the detailed feedback. I already filed an issue to follow up on this. The bot has some volume thresholds before filing a bug, but these thresholds do not apply when the bug has an identified regressor. Fixing #2265 should mitigate this.
|
|
||
Comment 2•2 years ago
|
||
Since the landing of #2265, I haven't noticed any similar cases. If you do notice anything wrong, please feel free to file an issue in the bugbot repository.
Description
•