941183 - Implement a sane trychooser string and default to that

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[Steve Fink [:sfink] [:s:]]

So bug 823711 made the trychooser web ui default to opt-only, which is a fairly 
awful default, except that defaulting to debug-only would skip some tests. But opt-only also skips tests (OS X ASan? I don't even know.) So neither is right.

We should make something that *is* right. (Well, philor would say only -b do is right. But apparently we're shooting for something less.)

Filing bug in the trychooser component. Probably ought to be split into try_parser and trychooser bugs.

(For the record, I am not working on this. Just filing.)

Comment 1

[Phil Ringnalda (:philor)]

The problem is that there is no sane default short of -b do -p all, other than applying specialized knowledge.

You should virtually never skip debug tests if you are affecting tests at all in any way, because debug tests will count your assertions and fail while opt will not. So, -b d.

If you do -b d and build Android or b2g, you will run no tests at all on those platforms.

We could be weird, default to -b d but for platforms which do not do debug tests, treat -b d as -b do. That would still be a bad default, because skip-if(debug), and because opt-only failures are common. Well, and debug-only failures are common too.

There is no sane default other than -b do.