726761 - deserializing Histogram SampleSets shouldn't check count sums

Status: RESOLVED FIXED

(Core :: IPC, defect)

Description

[Nathan Froyd [:froydnj]]

See bug 707320 comment 57 for the background here.

Comment 1

[Nathan Froyd [:froydnj]]

Attachment: patch

Deserialization shouldn't care whether the data "looks good"; that determination should be left to someone else.

Comment 2

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Comment on attachment 596781 [details] [diff] [review]
patch

AIUI this check caught a bug in our code.  Why do you want to remove it?  If you want to change the invariants on histogram state, then do that but update the check here.

Comment 3

[Nathan Froyd [:froydnj]]

We check the "invariants" on histogram state elsewhere (e.g. calling FindCorruption on histograms before reflecting them into JS and sending them off).  It's not useful to have a check in the deserialization code for this sort of thing because it lands us in the following situation:

- Histogram X existed in memory with (very rare!) differing counts and we happily included/ignored it when sending telemetry pings.
- Histogram X gets serialized at shutdown.
- Histogram X gets deserialized at startup and we think the datafile is corrupt because of this check.

Without the check, we would include/ignore histogram X just as we were doing before.

Comment 4

[Nathan Froyd [:froydnj]]

We can also run into the following scenario:

- Histogram X existed in memory with differing counts and we happily included it when sending telemetry pings (because it's not corrupt according to Histogram::FindCorruption).
- Histogram X gets serialized at shutdown.
- Firefox gets upgraded and histogram X does not exist in the new version of Firefox: we decided it was no longer useful, or we are measuring something slightly different and decided to rename it.
- When reading in histogram X's data, we notice that it's "corrupt" and therefore decide the whole file is corrupt.  We do this *even though* we don't care about the contents of the data (limitations in the current implementation dictate that we can only send serialized histograms that exist in TelemetryHistogram.h), we just want to read it off the disk.

Checking for corrupt data should be put at a higher level than this.

Comment 5

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Comment on attachment 596781 [details] [diff] [review]
patch

Just remove all this code.  Hopefully the scheme you describe in the above two comments is documented elsewhere, so we shouldn't need this comment.

r=me with that

Comment 6

[Nathan Froyd [:froydnj]]

Attachment: patch

Address review comments by deleting comments.  Carrying over r+.

Comment 7

[Mozilla RelEng Bot]

Autoland Patchset:
	Patches: 597795
	Branch: mozilla-central => try
	Destination: http://hg.mozilla.org/try/pushloghtml?changeset=898595eb9cbf
Try run started, revision 898595eb9cbf. To cancel or monitor the job, see: https://tbpl.mozilla.org/?tree=Try&rev=898595eb9cbf

Comment 8

[Mozilla RelEng Bot]

Try run for 898595eb9cbf is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=898595eb9cbf
Results (out of 212 total builds):
    exception: 1
    success: 175
    warnings: 22
    failure: 14
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-898595eb9cbf

Comment 9

[Ed Morley [:emorley]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/da1dae1b215d

Comment 10

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/da1dae1b215d