Open Bug 1350310 Opened 2 years ago Updated 2 years ago

Ensure necessary artifacts are uploaded so symbolic links and generated code can be resolved

Categories

(Testing :: Code Coverage, enhancement)

enhancement
Not set

Tracking

(Not tracked)

People

(Reporter: ekyle, Assigned: sparky)

References

(Blocks 1 open bug)

Details

After code coverage artifacts are generated, we can inspect each source file (the "full path" to the source file) and what lines were covered.

Unfortunately, the "full path" can include a symbolic link, or the source file can be generated from yet-other-sources. This is a problem because the "full paths" can not be found in the repository.

I am certain that symbolic links can be translated; if the ETL pipeline has access to the information necessary to resolve those symbolic links, then the coverage artifacts can be fixed before they go into the database. There is a similar story for the generated sources, but the specifics are unclear: Are the generated sources coming from code templates? Are the generated sources a result of translating another language? Do we care about generated source coverage?

This bug will be used to ensure each of the tasks responsible for codecoverage will upload the artifact(s) required to translate the symbolic links.
Assignee: nobody → gmierz2
This is not a high priority, as we can resolve the symbolic links quite easily (just need to clone mozilla-central and build "pre-export" and "export", which is pretty fast: https://github.com/mozilla-releng/services/blob/master/src/shipit_code_coverage/shipit_code_coverage/codecov.py#L121). Bug 1224691 is quite more important.
Blocks: 1365591
Marco, do you know where I could find more information about |mach build pre-export| and |mach build export|? I haven't found any newly created, or modified, files after running it which is why I am looking for more info on what these commands do.
Flags: needinfo?(mcastelluccio)
(In reply to Greg Mierzwinski [:gmierz] from comment #2)
> Marco, do you know where I could find more information about |mach build
> pre-export| and |mach build export|? I haven't found any newly created, or
> modified, files after running it which is why I am looking for more info on
> what these commands do.

About export:
http://gecko.readthedocs.io/en/latest/build/buildsystem/build-targets.html

To generate the coveralls and codecov reports, I'm doing this (https://github.com/mozilla-releng/services/blob/master/src/shipit_code_coverage/shipit_code_coverage/codecov.py#L121):
> mach configure
> mach build pre-export
> mach build export
Flags: needinfo?(mcastelluccio)
Thanks Marco, now I'm guessing that what ever is produced by those three commands is used by grcov. Is that right? Otherwise, could you please clarify what you need those commands for? 

Kyle, would you have any sample output that I could look at?
Flags: needinfo?(mcastelluccio)
Flags: needinfo?(klahnakoski)
(In reply to Greg Mierzwinski [:gmierz] from comment #4)
> Thanks Marco, now I'm guessing that what ever is produced by those three
> commands is used by grcov. Is that right? Otherwise, could you please
> clarify what you need those commands for? 

The commands are used to generate the symbolic links and compile IDL to C++.
The gcda files contain paths like "BUILD_DIR/dist/include/gfxMatrix.h", which are actually symbolic links (e.g. to "gfx/thebes/gfxMatrix.h"). In the reports, we don't want to show the path of the symbolic link, but we want to show the path the symbolic link resolves to.
grcov automatically resolves the symbolic links when generating the report, but obviously it needs the symbolic links to be there in order to resolve them, so I'm using the commands for that.

To summarize, this bug is already resolved for the codecov and coveralls reports.
Flags: needinfo?(mcastelluccio)
Greg,

I will get you a list of files found, files almost found, and files not found. Please point to a specific TH run of codecoverage so I may use it to generate the list.
(In reply to Marco Castelluccio [:marco] from comment #5)

> To summarize, this bug is already resolved for the codecov and coveralls
> reports.

Marco,

Does this mean there is a piece of code that maps the paths in the gcno files to paths in hg?  May you point to the code that does this? 

Thank you.
Flags: needinfo?(klahnakoski) → needinfo?(mcastelluccio)
Thanks for the great explanation Marco!

Kyle, you can use this run: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=6dfa56094f0cc291945dd3c24d0a4c2682d80ec7

It's the nightly code coverage run if that's fine with you. Otherwise, I'll start another test.
(In reply to Kyle Lahnakoski [:ekyle] from comment #7)
> (In reply to Marco Castelluccio [:marco] from comment #5)
> 
> > To summarize, this bug is already resolved for the codecov and coveralls
> > reports.
> 
> Marco,
> Does this mean there is a piece of code that maps the paths in the gcno
> files to paths in hg?  May you point to the code that does this? 

The code is what I pointed at in comment 1 and comment 3.

The gcno files contain paths to symlinks.

The mach commands are generating the symlinks, which point to the source files.

grcov is resolving the symlinks when generating the report, so it will get the correct paths to the source files.
Flags: needinfo?(mcastelluccio)
You need to log in before you can comment on or make changes to this bug.