1824183 - [meta] Improve logic around custom-data usage in Raptor

Status: NEW

(Testing :: Raptor, task, P2)

Description

[Greg Mierzwinski [:sparky]]

This bug is for improving how custom-data is handled in raptor. There's a few issues that make it difficult for developers to easily create tests without a lot of help or documentation. See this patch for a good example of many issues that we are hitting: https://phabricator.services.mozilla.com/D172467

For example:

• [bug] The custom data has to be in the first extras entry, otherwise it won't get processed.
• [bug] Flattening data with only 1-level will return no data.
• [bug] Unclear of the data structure expected in raptor results.py
• No documentation apart from prior art
• No easy way to know how to run your test to be able to properly gather your data. There's quite a bit of guessing involved at the moment in terms of how the test should be started/stopped, and when/where to add custom data.