1887824 - Vendor Transformers.js 2.16.1

Status: RESOLVED FIXED

(Core :: Machine Learning, task)

Description

[Tarek Ziadé (:tarek)]

This vendors Transformers.js into the ML component to perform local inference

The project is maintained by Hugging Faces and is a wrapper lib around the ONNX runtime that we will store as WASM files into Remote Settings.

The licence is APLv2

Comment 1

[Tarek Ziadé (:tarek)]

Attachment: Bug 1887824 - vendor Transformers.js - r?gregtatum

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/957de71a1cd0
vendor Transformers.js - r=gregtatum,frontend-codestyle-reviewers,Standard8

Comment 3

[Sandor Molnar[:smolnar]]

Backed out for causing bc failures @ browser_all_files_referenced.js

Backout link: https://hg.mozilla.org/integration/autoland/rev/0558e77139f0c57a342972dc9bcfa67dc55715cd

Push with failures

Failure log -> TEST-UNEXPECTED-FAIL | browser/base/content/test/static/browser_all_files_referenced.js | Uncaught exception in test bound checkAllTheFiles

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c51b8113c3f5
vendor Transformers.js - r=gregtatum,frontend-codestyle-reviewers,Standard8

Comment 5

[Aron Cseh]

https://hg.mozilla.org/mozilla-central/rev/c51b8113c3f5

Comment 6

[Alex Finder]

(In reply to Pulsebot from comment #2)

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/957de71a1cd0
vendor Transformers.js - r=gregtatum,frontend-codestyle-reviewers,Standard8

Perfherder has detected a build_metrics performance change from push 957de71a1cd0cadef70bbc0cc03810e428f7abff.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
1%	installer size	osx-cross		94,150,413.00 -> 94,991,016.33

As author of one of the patches included in that push, we need your help to address this regression.
Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run these tests on try with ./mach try perf --alert 42272

For more information on performance sheriffing please see our FAQ.

Comment 7

[Tarek Ziadé (:tarek)]

I added a new vendored lib that will be smaller on release (the dev version includes the dev flavor) -- I also did a follow up patch on Bug 1887826 that removes the map data, making it smaller already.

Comment 8

[Alex Finder]

(In reply to Tarek Ziadé (:tarek) from comment #7)

I added a new vendored lib that will be smaller on release (the dev version includes the dev flavor) -- I also did a follow up patch on Bug 1887826 that removes the map data, making it smaller already.

Hi Tarek! Do you have a pushlog with the fix you mentioned. I checked the graphs in the regression bug, but there is no size fix visible.

Comment 9

[Tarek Ziadé (:tarek)]

Hey Alex, check out this changeset https://hg.mozilla.org/mozilla-central/rev/6070f05ecffa46451670c0daa47148c3e7612a43
and in particular the change on ort-dev.js

https://hg.mozilla.org/mozilla-central/diff/6070f05ecffa46451670c0daa47148c3e7612a43/toolkit/components/ml/vendor/ort-dev.js

I removed the map data at the end of the file, reducing its size from 3.8MiB to 1.3MiB:

➜  dist git:(main) ✗ ls -lah ort-dev.js
-rw-r--r--  1 tarekziade  staff   3.8M Oct 26  1985 ort-dev.js
➜  dist git:(main) ✗ sed -i '' '1855,1859d' ort-dev.js
➜  dist git:(main) ✗ sed -i '' '$d' ort-dev.js
➜  dist git:(main) ✗ ls -lah ort-dev.js
-rw-r--r--  1 tarekziade  staff   1.3M May  2 14:14 ort-dev.js

Comment 10

[Alex Finder]

(In reply to Tarek Ziadé (:tarek) from comment #9)

Hey Alex, check out this changeset https://hg.mozilla.org/mozilla-central/rev/6070f05ecffa46451670c0daa47148c3e7612a43
and in particular the change on ort-dev.js

https://hg.mozilla.org/mozilla-central/diff/6070f05ecffa46451670c0daa47148c3e7612a43/toolkit/components/ml/vendor/ort-dev.js

I removed the map data at the end of the file, reducing its size from 3.8MiB to 1.3MiB:

➜  dist git:(main) ✗ ls -lah ort-dev.js
-rw-r--r--  1 tarekziade  staff   3.8M Oct 26  1985 ort-dev.js
➜  dist git:(main) ✗ sed -i '' '1855,1859d' ort-dev.js
➜  dist git:(main) ✗ sed -i '' '$d' ort-dev.js
➜  dist git:(main) ✗ ls -lah ort-dev.js
-rw-r--r--  1 tarekziade  staff   1.3M May  2 14:14 ort-dev.js

Hi Tarek! Thanks for the reply! It looks like even though the size increase caused by 6070f05ecffa46451670c0daa47148c3e7612a43 is smaller than the one introduced by revision 957de71a1cd0cadef70bbc0cc03810e428f7abff, the alerting system still detected a regression for the introduced changeset (as visible in the regressing defect 1893112. If the changeset is necessary and the regressing bug is set to WONTFIX, I will update the alert summary to reflect the new status.