So bug 823711 made the trychooser web ui default to opt-only, which is a fairly awful default, except that defaulting to debug-only would skip some tests. But opt-only also skips tests (OS X ASan? I don't even know.) So neither is right. We should make something that *is* right. (Well, philor would say only -b do is right. But apparently we're shooting for something less.) Filing bug in the trychooser component. Probably ought to be split into try_parser and trychooser bugs. (For the record, I am not working on this. Just filing.)
The problem is that there is no sane default short of -b do -p all, other than applying specialized knowledge. You should virtually never skip debug tests if you are affecting tests at all in any way, because debug tests will count your assertions and fail while opt will not. So, -b d. If you do -b d and build Android or b2g, you will run no tests at all on those platforms. We could be weird, default to -b d but for platforms which do not do debug tests, treat -b d as -b do. That would still be a bad default, because skip-if(debug), and because opt-only failures are common. Well, and debug-only failures are common too. There is no sane default other than -b do.
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2019] → [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2026]
Component: Tools → General
Product: Release Engineering → Release Engineering
Status: NEW → RESOLVED
Last Resolved: 11 months ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.