Open Bug 1597903 Opened 1 year ago Updated 3 months ago

Improve error reporting for the diffoscope tasks

Categories

(Firefox Build System :: Task Configuration, task)

task
Not set
normal

Tracking

(Not tracked)

People

(Reporter: glandium, Assigned: glandium)

References

Details

(Keywords: leave-open)

Attachments

(3 files)

cf. bug 1597901, the thing that pops to sheriffs is not really helpful (Downloaded image is corrupted: (HTTP code 404) no such image) because it's actually irrelevant to the actual error which is that the builds differ.

So we should add an error message that shows up on treeherder, possibly with a link to the diff artifact if possible.

Let's start with just an error message. I'll leave the bug open to think more about maybe ways we could munge diff.txt to generate something useful for sheriffs to copy/paste when filing bugs.

Assignee: nobody → mh+mozilla
Keywords: leave-open
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/cd6b79c74b23
Add an error message when differences are found in diffoscope. r=froydnj

Somehow parse the diff output to print out slightly more unique error messages.

With this change, the error for bug 1597901 becomes:
TEST-UNEXPECTED-FAIL | firefox/libmozavutil.so differs. See the diff.html or diff.txt artifact

And the error for bug 1601150 becomes:
TEST-UNEXPECTED-FAIL | firefox/libxul.so differs. See the diff.html or diff.txt artifact

Which is not great, but better than the status quo.

With this change, the error for bug 1601150 becomes:
TEST-UNEXPECTED-FAIL | i686-unknown-linux-gnu/release/build/cranelift-codegen-37dab135a3db8b7a/out/inst_builder.rs differs. See the generated-files.diff.html or generated-files.diff.txt artifact
TEST-UNEXPECTED-FAIL | i686-unknown-linux-gnu/release/build/cranelift-codegen-37dab135a3db8b7a/out/opcodes.rs differs. See the generated-files.diff.html or generated-files.diff.txt artifact
TEST-UNEXPECTED-FAIL | i686-unknown-linux-gnu/release/build/cranelift-codegen-e483dcad1bdf8c4a/out/inst_builder.rs differs. See the generated-files.diff.html or generated-files.diff.txt artifact
TEST-UNEXPECTED-FAIL | i686-unknown-linux-gnu/release/build/cranelift-codegen-e483dcad1bdf8c4a/out/opcodes.rs differs. See the generated-files.diff.html or generated-files.diff.txt artifact

Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/6435902a44ef
Improve error reporting on diffoscope tasks. r=froydnj
https://hg.mozilla.org/integration/autoland/rev/ae2503b461f3
When a build differs, check if the generated-files differ. r=froydnj
Regressions: 1615150
Depends on: 1654158
You need to log in before you can comment on or make changes to this bug.