1504200 - Validate test pings using the glean ping schema

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P3)

Description

[Alessio Placitelli [:Dexter]]

Instead of hardcoding the validation logic in tests, we should change our tests to validate the generated pings using the schema. Downloading the schema from Gradle can be achieved as:

> // Download the ping schema to the test resources directory.
> task downloadPingSchema {
>     description = "Download the glean ping schemas for validating pings in tests"
>     doLast {
>         def f = new File("$buildDir/glean_schemas/schema.json")
>         println(f.getPath().toString())
>         f.getParentFile().mkdirs()
>         if (!f.exists()) {
>             new URL('https://github.com/mdboom/mobile-telemetry-sdk-schemas/raw/master/schemas/ping.1-0-0.schema.json').withInputStream{ i -> f.withOutputStream{ it << i }}
>         }
>     }
> }
> 
> preBuild.dependsOn(downloadPingSchema)

With the schema resource being added to the project as 

>     sourceSets {
>         test {
>             resources {
>                 // We want to access the schema from the unit tests, in order
>                 // to validate pings we generate from them.
>                 srcDir "${buildDir}/glean_schemas/"
>             }
>         }
>     }

And the schema being loaded in tests as:

>         // Load the schema from the downloaded resources.
>         val classLoader = PingMakerTest::class.java.classLoader
>         val jsonContent = classLoader.getResourceAsStream("schema.json")
>             .bufferedReader()
>             .use { it.readText() }
>         assertNotNull(jsonContent)

Unfortunately, there doesn't seem to be many Java/Kotlin JSON schema validators available off the shelf: [1] doesn't work in Android Studio (due to |getNames| being missing in their implementation of org.json.JSONObject) and [2] doesn't work nicely with JSONOBject(s) and doesn't support schema Draft-6.

[1] - https://github.com/everit-org/json-schema
[2] - https://github.com/java-json-tools/json-schema-validator

Comment 1

[Michael Droettboom [:mdroettboom]]

Attachment: GitHub Pull Request

Comment 2

[Michael Droettboom [:mdroettboom]]

Attachment: GitHub Pull Request

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/328a0fdd4f80
[components] Fix #1504200: Check generated pings against the schema