1348266 - Coverage builds should never be artifact builds

Status: NEW

(Testing :: Code Coverage, enhancement, P3)

Description

[James Graham [:jgraham]]

I presume we aren't uploading artifacts for coverage builds, so a try run with -p linux64-ccov should never default to artifact mode, irrespective of the mozconfig.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Marco, is that true? From my perspective it would be great to have artifact support for coverage builds so that I'm not forced having to run a complete build. Is it something we could do?

Comment 2

[Marco Castelluccio [:marco]]

I'm OK with allowing code coverage builds to be artifact builds, but I don't know what we need to do to support it.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Andrew, could you help out here, or at least know someone who can? It would be good to know how complex it will be to allow artifact builds.

James, I assume when we would have valid artifact builds the problem you were facing won't exist anymore, right?

Comment 4

[James Graham [:jgraham]]

Sure, if there are artifact builds for ccov that invalidates this bug. I assume that actually doing that work is going to be non-trivial though.

Comment 5

[Andrew Halberstadt [:ahal]]

Looks like the CI logic for determining artifact builds sources valid jobs from here:
https://searchfox.org/mozilla-central/source/python/mozbuild/mozbuild/artifact_builds.py#9

Though I think you are asking for the opposite of this bug :p. Why would you want to run ccov with artifact builds? Wouldn't it be better to not select them in the first place?

I guess as long as ccov is being pointlessly selected anyway, it's better to run it with an artifact build rather than not.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Andrew Halberstadt [:ahal] from comment #5)

Though I think you are asking for the opposite of this bug :p. Why would you want to run ccov with artifact builds? Wouldn't it be better to not select them in the first place?

Maybe we mix up bugs now, but in my case I would love to not have to do full ccov builds locally, but just benefit from already build artifacts.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Andrew, could you explain what would be the downside of having ccov builds as artifact builds available? Would this put some extra overhead on CI? Or what's the reason why we shouldn't have them as artifact builds?

Comment 8

[Andrew Halberstadt [:ahal]]

I assume that any coverage artifacts produced by an artifact coverage build would only be relevant to the commit that produced the artifacts and not your current commit. So what would be the point of running them on your current commit? Are you worried about breaking them or something?

Comment 9

[Nick Alexander :nalexander [he/him]]

(In reply to Andrew Halberstadt [:ahal] from comment #8)

I assume that any coverage artifacts produced by an artifact coverage build would only be relevant to the commit that produced the artifacts and not your current commit. So what would be the point of running them on your current commit? Are you worried about breaking them or something?

Granted that I know nothing about coverage builds, but: it's my belief that we compute coverage for multiple layers of the stack. Traditionally, C++/Rust, but now also JS (and maybe even Python). So the natural divide is that artifact builds get "up to date" coverage information for the layers up the stack, say, JS. Which in turn can invoke C++/Rust that wasn't previously invoked.

I don't know how well we can represent those layers in the stack, i.e., how well this use-case/distinction can be catered to, but it sounds valuable if it's cheap.

Comment 10

[Marco Castelluccio [:marco]]

(In reply to Nick Alexander :nalexander [he/him] from comment #9)

Granted that I know nothing about coverage builds, but: it's my belief that we compute coverage for multiple layers of the stack. Traditionally, C++/Rust, but now also JS (and maybe even Python). So the natural divide is that artifact builds get "up to date" coverage information for the layers up the stack, say, JS. Which in turn can invoke C++/Rust that wasn't previously invoked.

I don't know how well we can represent those layers in the stack, i.e., how well this use-case/distinction can be catered to, but it sounds valuable if it's cheap.

Yes, this is correct. An artifact build could be used to collect coverage (for all languages) in the case where you only changed JS (or HTML, etc.) code.
The only thing you need in addition to a normal build are the gcno files (the "target.code-coverage-gcno.zip" artifact), which are required to parse coverage artifacts.