917341 - Investigate data sanitation problems in overnight runs

Status: RESOLVED INVALID

(Data Platform and Tools :: General, defect)

Description

[Lonnen :lonnen]

Unique lines written to the error log on 09-16-2013:

SimpleMeasures are missing...

index is None in map

<type 'list'> is not a value type for simpleMeasurements "addonManager_XPIDB_startup_load_reasons"

<type 'unicode'> is not a value type for simpleMeasurements "addonManager_XPIDB_lateOpen_writeList"

bucket2index is None in map

<type 'unicode'> is not a value type for simpleMeasurements "addonManager_XPIDB_async_load"

OverflowError: long too big to convert

Comment 1

[Lonnen :lonnen]

Overwhelmingly "bucket2index is None in map" errors are the most prevalent (99% of the lines)

Comment 2

[Mark Reid [:mreid]]

With the analysis script running against the validated data in S3, I don't think this should be a problem.

Jonas, can you check to see if we're still getting errors like the above using the S3-published data?

Comment 3

[Jonas Finnemann Jensen (:jonasfj)]

Yeah, considering that everything have been rewritten 2-3 times and now in C++ since this was reported... I think it's safe to say that this is now outdated...