1976784 - Llama.cpp native peformance test

Status: RESOLVED FIXED

(Core :: Machine Learning: General, enhancement)

Description

[Aristide Tossou]

Llama.cpp performance test

Comment 1

[Aristide Tossou]

Attachment: Bug 1976784 - Llama.cpp native peformance test r?tarek,padenot

Comment 2

[Aristide Tossou]

Attachment: WIP: Bug 1976784 - Temporarily activate simd r?tarek,padenot

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/b16ea82cc615
https://hg.mozilla.org/integration/autoland/rev/3d0582773176
Llama.cpp native peformance test r=tarek,mozperftest-reviewers,kshampur

Comment 4

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/3d0582773176

Comment 5

[amarc]

Backed out for causing Hazard bustages.

Backout link: https://hg-edge.mozilla.org/integration/autoland/rev/4efe072f5aa1c69de8158337d9cddbaf7ec3ae05

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=347c59ae0bce3cde618bb738c6e630896f22cf8a&selectedTaskRun=WahxcdQjRPGMO-v-YgrlcQ.0

Failure log:
Hazard: https://treeherder.mozilla.org/logviewer?job_id=523781509&repo=autoland&task=WahxcdQjRPGMO-v-YgrlcQ.0&lineNumber=153071
Tier2 Snap: https://treeherder.mozilla.org/logviewer?job_id=523755213&repo=autoland&task=DAmutYuXQhqT2vtrOnel0g.0

Comment 6

[amarc]

Backout merged to central: https://hg.mozilla.org/mozilla-central/rev/4efe072f5aa1

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/4a3e319d3bc5
https://hg.mozilla.org/integration/autoland/rev/c63b68ef62a6
Llama.cpp native peformance test r=tarek,mozperftest-reviewers,kshampur

Comment 8

[Atila Butkovits]

Backed out for causing Hazard bustages.

Backout link: https://hg-edge.mozilla.org/integration/autoland/rev/ad97718e3c54cfa88c7e480fde786dc961594b79

Push where failures started: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&searchStr=linux%2Cdebug%2Chazard-linux64-haz%2Fdebug%2Ch&revision=4f62441a0fd8eeb63139d01174dc9435a77cccb9&selectedTaskRun=Jlsuh2WVTDmjkYKYpFL2Ww.1

Failure log: https://treeherder.mozilla.org/logviewer?job_id=524018554&repo=autoland&task=Jlsuh2WVTDmjkYKYpFL2Ww.1&lineNumber=153087

Comment 9

[Alex Finder]

Hi Aristide! There was a build_metrics regression detected on the push mentioned below, and this bug was referenced. Do you think this bug may have caused the performance regression ?

Perfherder has detected a build_metrics performance change from push 347c59ae0bce3cde618bb738c6e630896f22cf8a. As author of one of the patches included in that push, we need your help to address this regression.

Please acknowledge, and begin investigating this alert within 3 business days, or the patch(es) may be backed out in accordance with our regression policy. Our guide to handling regression bugs has information about how you can proceed with this investigation.

If you have any questions or need any help with the investigation, please reach out to a performance sheriff. Alternatively, you can find help on Slack by joining #perf-help, and on Matrix you can find help by joining #perftest.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
6%	compiler_metrics num_static_constructors	osx-cross		91.00 -> 96.50
4%	compiler_metrics num_static_constructors	linux64-shippable		102.00 -> 106.17

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a performance sheriff to do that for you.

You can run all of these tests on try with ./mach try perf --alert 46416

The following documentation link provides more information about this command.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/67b2b7da0f5f
https://hg.mozilla.org/integration/autoland/rev/c91cd7c51202
Llama.cpp native peformance test r=tarek,mozperftest-reviewers,kshampur

Comment 11

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/c91cd7c51202