1821199 - [meta] replace manifestparser .ini with .toml

Status: NEW

(Testing :: General, task)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

this is the master bug, a project is outlined here:
https://docs.google.com/document/d/1VV_1ao4f1mnR7n1fRqz_KbX7ybUBFieszPCShbwFWVo/edit

Comment 1

[Mark Banner (:standard8)]

I've just filed bug 1847580 which will need fixing as part of this - currently ESLint reads the test manifest .ini files to work out various details, and so that will need updating to read toml files before they ini files are made redundant/removed.

Comment 2

[Mark Banner (:standard8)]

A couple more thoughts:

I'd assume that ./mach addtest will also need fixing. Though I'm not quite sure where this should be filed.

Should we add a .toml handling extension to the recommended list for VS Code (see .vscode/extensions.json)? I would assume it would give us syntax highlighting and that might make it easier to read and edit.

Comment 3

[Mike Hommey [:glandium]]

I think each of these changes is making manifest parsing slower, and is the cause for the slow drift in time spent in configure: https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&series=autoland,4200695,1,2&timerange=5184000

Before this all started, we had:
Finished reading 2285 moz.build files in 4.72s

Now, we have:
Finished reading 2288 moz.build files in 23.25s

And a local python profile shows most of the time spent in read_toml.

Comment 4

[Mike Hommey [:glandium]]

I'll also note that bug 1779473 added tomlkit, while we already had toml, and that crudely replacing tomlkit with toml in testing/mozbase/manifestparser/manifestparser/toml.py brings down the time spent in read_toml substantially.

Comment 5

[Tom Marble]

The choice of tomlkit over toml was intentional due to the former library's ability to edit TOML files while preserving comments (to be used in future automation).

We can use Bug 1859997 to track read_toml performance

Comment 6

[Magnus Melin [:mkmelin]]

Are conversions by hand, or do you have something scripted to help?

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

they were scripted, in general you can read/write from manifestparser, so try using that for reading the .ini and then writing out a .toml.