1861626 - Use rtoml instead of toml to read test manifests

Status: REOPENED

(Firefox Build System :: Task Configuration, enhancement)

Description

[Marco Castelluccio [:marco]]

Not sure how faster it is, but it's worth investigating.

Comment 1

[Mike Hommey [:glandium]]

rtoml is a compiled module, that makes it significantly harder to deploy. (even more so for decision tasks)

Comment 2

[Mike Hommey [:glandium]]

(also, the switch from tomlkit to toml didn't really impact decision task times, despite having had a dramatic effect on configure)

Comment 3

[Marco Castelluccio [:marco]]

Time to read 7777 manifests:

import timeit
import toml
import tomlkit
import tomllib
import rtoml

with open("browser/base/content/test/popups/browser.toml", "r") as f:
    manifest = f.read()

def parsewithtoml():
    toml.loads(manifest)

def parsewithtomlkit():
    tomlkit.parse(manifest)

def parsewithtomllib():
    tomllib.loads(manifest)

def parsewithrtoml():
    rtoml.load(manifest)

print(timeit.timeit('parsewithtoml()', number=7777, globals=globals()))
print(timeit.timeit('parsewithtomlkit()', number=7777, globals=globals()))
print(timeit.timeit('parsewithtomllib()', number=7777, globals=globals()))
print(timeit.timeit('parsewithrtoml()', number=7777, globals=globals()))

toml 4.716776925000886
tomlkit 70.1370183600011
tomllib 2.5913682589998643
rtoml 0.6805894560020533

Comment 4

[Marco Castelluccio [:marco]]

Attachment: Bug 1861626 - Add rtoml as an optional dependency for mach. r=#mach-reviewers

Comment 5

[Marco Castelluccio [:marco]]

Attachment: Bug 1861626 - Use rtoml, if available, to read toml manifests. r=jmaher

Comment 6

[Marco Castelluccio [:marco]]

Attachment: Bug 1861626 - Install rtoml in the decision task image. r=#taskgraph-reviewers

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/6b58054b18dc
https://hg.mozilla.org/integration/autoland/rev/215600a840b9
Add rtoml as an optional dependency for mach. r=mach-reviewers,ahochheiden
https://github.com/mozilla-firefox/firefox/commit/f4a3d0ee0ef0
https://hg.mozilla.org/integration/autoland/rev/1baf82234776
Use rtoml, if available, to read toml manifests. r=jmaher
https://github.com/mozilla-firefox/firefox/commit/f978c8468854
https://hg.mozilla.org/integration/autoland/rev/07b1fff42c4f
Install rtoml in the decision task image. r=taskgraph-reviewers,jmaher,releng-reviewers,ahal

Comment 8

[Marco Castelluccio [:marco]]

Julien, could you help with the decision image update like you did in bug 1840829?

Comment 9

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/215600a840b9
https://hg.mozilla.org/mozilla-central/rev/1baf82234776
https://hg.mozilla.org/mozilla-central/rev/07b1fff42c4f

Comment 10

[Julien Cristau [:jcristau]]

Attachment: Bug 1861626 - use gecko_decision:5.1.0 docker image. r?#releng,marco

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/41aa2e769f03
https://hg.mozilla.org/integration/autoland/rev/b1ec2a03cee3
use gecko_decision:5.1.0 docker image. r=marco,releng-reviewers,bhearsum

Comment 12

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/b1ec2a03cee3

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/5ccf4a7d77a3
https://hg.mozilla.org/mozilla-central/rev/b50fe26f9711
Revert "Bug 1861626 - use gecko_decision:5.1.0 docker image. r=marco,releng-reviewers,bhearsum" to unbreak nightlies.

Comment 14

[Atila Butkovits]

Backed out to unbreak nightlies.

Backout link: https://hg-edge.mozilla.org/mozilla-central/rev/b50fe26f971156e4967c642fb43cf26f5e69eb1d

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/9d75bf093845
https://hg.mozilla.org/integration/autoland/rev/fc7f69cb518c
use gecko_decision:5.1.0 docker image. r=marco,releng-reviewers,bhearsum