1784509 - Decouple error reporting builds from builds which are used for tests and benchmarks.

Status: NEW

(Developer Infrastructure :: Try, enhancement)

Description

[Nicolas B. Pierron [:nbp]]

Steps to reproduce:

• Make a mistake which does not prevent building and running but would only be reported as an error on CI.
• Push to try.
• Schedule Linux browsertime benchmarks.
• Wait 4 hours (because you know it is going to be a lengthy process)

Expected:

• Benchmarks were run, and result are available.

Seen:

• The build failed because struct/class might cause compilations error on Windows.
• I have to make another push and wait 4 hours again …

Comment 1

[Mike Hommey [:glandium]]

The build failed because struct/class might cause compilations error on Windows.

Can you detail this?

Comment 2

[Nicolas B. Pierron [:nbp]]

(In reply to Mike Hommey [:glandium] from comment #1)

The build failed because struct/class might cause compilations error on Windows.

Can you detail this?

Yes:

1. struct/class pre-declaration and class/struct declaration mix-up causes build failures and prevents getting benchmarking results: https://treeherder.mozilla.org/jobs?repo=try&revision=76d8f5fea6cec3acb83761cce21f01f489c20450
2. -Werror prevents getting test suite results: https://treeherder.mozilla.org/jobs?repo=try&revision=232b4d216b1fcc4d64b309e42dfaf7f1c31a3016

I do understand that -Werror and Windows builds are important. However these errors are not important to me at this precise moment, when my goal is to get benchmark / test suite results.