1518170 - Always upload profile output

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[Michael Shal [:mshal]]

Bug 1514288 made it so that during PGO builds we save the terminal output of Firefox into files and save them as artifacts. However, the files don't get uploaded if the task fails. We should upload them regardless so we can see why the profile generation step failed, and perhaps print them out if FirefoxRunner returns a failure code.

Comment 1

[(not currently active) Ted Mielczarek]

You should be able to just write the log to `$UPLOAD_PATH`, which is where we stick artifacts. I did that for the sccache log for similar reasons:
https://dxr.mozilla.org/mozilla-central/rev/c2593a3058afdfeaac5c990e18794ee8257afe99/client.mk#85-86

Comment 2

[Michael Shal [:mshal]]

Attachment: Bug 1518170 - Always upload profile run output; r?#firefox-build-system-reviewers

In bug 1514288 we started to save the output from running Firefox during
PGO builds into log files, but they aren't correctly uploaded if the run
fails. We can generate them directly into the artifacts directory so
that they are always uploaded. Additionally, the log files are displayed
if the run fails so that the error messages are displayed in the log for
easier bug categorization.

Comment 3

[Mike Hommey [:glandium]]

What's the benefit of getting the profile data from failed jobs?

Comment 4

[Mike Hommey [:glandium]]

Just saw bug 1517939 comment 2. Please mention that in the commit message.

Comment 5

[Michael Shal [:mshal]]

(In reply to Mike Hommey [:glandium] from comment #4)

Just saw bug 1517939 comment 2. Please mention that in the commit message.

Done!

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8d44d6ba5863
Always upload profile run output; r=firefox-build-system-reviewers,chmanchester#firefox-build-system-reviewers

Comment 7

[(not currently active) Ted Mielczarek]

Just in case this fails in automation--you might need to ensure that UPLOAD_PATH exists. It doesn't always get created before we hit the upload step (I put an explicit mkdir in the rules to run sccache in client.mk, you can see that in the link in comment 1).

Comment 8

[Michael Shal [:mshal]]

(In reply to Ted Mielczarek [:ted] [:ted.mielczarek] from comment #7)

Just in case this fails in automation--you might need to ensure that UPLOAD_PATH exists. It doesn't always get created before we hit the upload step (I put an explicit mkdir in the rules to run sccache in client.mk, you can see that in the link in comment 1).

Oh, interesting. It seemed to work in try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=9dce1163ad8a6cd10055e2a1a89e05f246749dd6

I guess we'll see if it sticks.

Comment 9

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/8d44d6ba5863