1057210 - [mozlog] Mozlog should have a thorough set of tests establishing desired properties for its formatters

Status: NEW

(Testing :: Mozbase, defect)

Description

[Chris Manchester (limited bugmail, email directly)]

Myself and others on the A-Team have been involved in a push to make our logging more flexible and useful. This has meant transitioning away from our test harnesses logging an ad-hoc textual format for people and tools to read directly to instead logging a JSON structure containing the relevant information in a log message that can then be reformatted or otherwise interpreted for a variety of uses. The library we're using to achieve this is called mozlog. It's part of mozbase and lives in mozilla central. The mozlog docs (http://mozbase.readthedocs.org/en/latest/mozlog_structured.html) would be a good place to start to get familiar with the implementation and the data format we're using.

As we're making this transition and fine-tuning output formats, it would be invaluable to have a more complete set of tests to both sanity check our changes and serve as some way to document the requirements for log formats. I'm not sure how critical each of these will be in practice, but here are a few ideas we could refine into smaller items:

Some tests would aim to establish general safety properties of a formatter, for instance the property that a formatter doesn't access a field of a log message directly if that field is optional (raising a KeyError that might be hard to diagnose). Others would be for other measures of correctness of a formatter, such as the property that a certain formatter always produces string output that contains the 'message' field of an input message as a substring. Still others could be properties that formatted output satisfies an external requirement, such as the idea that test failure output always triggers a match with respect to a regex specified in a consumer (such as mozharness or treeherder/tbpl), or conform to another guideline such as the sheriff's visibility guidelines.

Comment 1

[Chris Manchester (limited bugmail, email directly)]

Hi Vikas, does this sound like an interesting project for you? If so, please feel free to ping me in #ateam or ask here and we can talk about specific cases to cover. Thanks!