Open Bug 1571617 Opened 7 months ago Updated 6 months ago

Tasks that don't have rust-size are reporting empty section sizes

Categories

(Firefox Build System :: General, defect)

defect
Not set

Tracking

(Not tracked)

People

(Reporter: glandium, Assigned: glandium)

References

Details

I think it's fair that some tasks don't care to report section sizes. I'd argue we only really care about the shippable builds, but let's say we care about that those currently report valid values. The problem is that some builds currently report invalid values (0) because they don't have rust-size. Rather than force them to install rust-size, let's say it's fine if they don't have it.

One worry that I have is what happens if a task that legitimately should report values stops doing so. Does that raise an alarm?

Flags: needinfo?(igoldan)

(In reply to Mike Hommey [:glandium] from comment #0)

I think it's fair that some tasks don't care to report section sizes. I'd argue we only really care about the shippable builds, but let's say we care about that those currently report valid values. The problem is that some builds currently report invalid values (0) because they don't have rust-size. Rather than force them to install rust-size, let's say it's fine if they don't have it.

One worry that I have is what happens if a task that legitimately should report values stops doing so. Does that raise an alarm?

I'm not sure I get your question. If you're worried about side effects from removing section size reports, you don't have too.

If you're worried your work could somehow turn off the valid data also (which could prove usefull), I'm afraid we don't have an alarm system in place for situations like these.
We already discussed about a feature like this for Perfherder & even defined a plan for implementing it. But it's not officially on our long/mid-term roadmap. Thanks to your question, I'll bump its importance.

Dave, FYI.

Flags: needinfo?(igoldan) → needinfo?(dave.hunt)

A test that starts reporting a 0 value when previously the value was non-zero should alert if this is outside of the alert threshold (defaults to 2% but configurable per test). A test that ceases to submit values does not alert. As Ionut says we have discussed a feature for alerting whenever test data ceases, or when new test data is seen. This will also help when we have tests that change name for whatever reason.

:igoldan I had a look but couldn't find a bug on file for this feature. Could you file one? I think we should consider it for Q4.

Flags: needinfo?(dave.hunt) → needinfo?(igoldan)
See Also: → 1581046

(In reply to Dave Hunt [:davehunt] [he/him] ⌚️UTC from comment #2)

:igoldan I had a look but couldn't find a bug on file for this feature. Could you file one? I think we should consider it for Q4.

Done.

Flags: needinfo?(igoldan)
You need to log in before you can comment on or make changes to this bug.