Extend local validation script with totalTime & browser defaults

RESOLVED WORKSFORME

Status

Cloud Services
Metrics: Pipeline
RESOLVED WORKSFORME
3 years ago
3 years ago

People

(Reporter: gfritzsche, Assigned: gfritzsche)

Tracking

unspecified
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox43 affected)

Details

(Whiteboard: [unifiedTelemetry])

(Assignee)

Description

3 years ago
We should extend the v2/v4 comparison in this script:
https://gist.github.com/georgf/1b0831a6b81b6c9fe240

... with totalTime & browser defaults.
(Assignee)

Updated

3 years ago
Assignee: nobody → gfritzsche
(Assignee)

Comment 1

3 years ago
Any other fields?
Component: Telemetry → Metrics: Pipeline
Flags: needinfo?(kparlante)
Product: Toolkit → Cloud Services
Whiteboard: [unifiedTelemetry]
Version: Trunk → unspecified
(Assignee)

Comment 2

3 years ago
Updated the script:
* added comparisons of v2 totalTime to v4 totalTime, sessionLength, subsessionLength
* if on a "newish" profile on 42+, don't match up v2 or v4 dates if they cover roughly the same historic dates, use all the data instead
* show the cutoff date used for date matching (and when it wasn't used)

This gives fine totalTime comparisons on my first run on a new profile, but looks rather broken on my profile with long history.

Caveats:
* sessionLength was introduced relatively recently (Nightly >= 2015-07-30)
* when not on a newish profile on 42+, we have to try date matching, so certain discrepancies are expected
(Assignee)

Comment 3

3 years ago
So we can approach this from two angles:
* trying to reproduce big mismatches on a fresh profile on 42+
* for bigger mismatches in our clients, manually dig into the historic data to check this (potentially time consuming)
(Assignee)

Comment 5

3 years ago
(In reply to Georg Fritzsche [:gfritzsche] from comment #4)
> Mismatch data from Katie:
> https://docs.google.com/spreadsheets/d/1XCi_yYTb6RVO89lfnAAf1MTqBV7F2YXPXJYY-
> 6fmDU8/edit#gid=0

The sesssion chaining problems there went away after waiting for 60sec after startup (i.e. after letting Telemetry load the session data from disk).
The mismatches persisted.
(Assignee)

Updated

3 years ago
Flags: needinfo?(kparlante)
(Assignee)

Comment 6

3 years ago
Updated:
* added the current browser default
* consolidated the output in one tab
* v2 history is now shown in table form too for manual inspection
(Assignee)

Comment 7

3 years ago
Updated, mostly added the historical browser default validation.
(Assignee)

Comment 8

3 years ago
This part seems to work fine, lets collect any suspicious data and file new bugs for investigations.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.