939871 - Expose dump.largest_free_vm_block as a numeric field in supersearch

Status: RESOLVED FIXED

(Socorro Graveyard :: Middleware, defect, P3)

Description

[Benjamin Smedberg]

Bug 939141 added a new field for the JSON form of MDSW: dump.largest_free_vm_block is a string field containing a hex "0xAFFD0000" number.

I'd like to be able to supersearch on this field, including limits such as "<100MB" or even, if this can be possible in the future, "largest_free_vm_block < raw.dumpOOMAllocationSize".

Comment 1

[[DEACTIVATED] Adrian Gaudebert]

This is interesting. I don't know if we can make elasticsearch understand what a hex is, and to perform comparisons on it, but I will look. 

> "largest_free_vm_block < raw.dumpOOMAllocationSize"

I don't see that happening in SuperSearch, but that looks like a good case for a custom elasticsearch query service. There are bugs filed for this but they went down on my priority list. See bug 890942 and bug 890946.

Comment 2

[Benjamin Smedberg]

I don't particularly need the value to be hex in elasticsearch. It's hex in JSON because JSON doesn't allow for 64-bit ints as numbers. So if you want to convert it to a number before it ends up in ES, that's fine.

If the ES custom search is a longer-term project, maybe I should focus on running this against either PG or hbase. I'll be doing a lot of comparison/queries against these set of fields from the raw and processed data:

* largest_free_vm_block
* AvailablePhysicalMemory
* AvailableVirtualMemory
* AvailablePageFile

Comment 3

[[DEACTIVATED] Adrian Gaudebert]

https://crash-stats.mozilla.com/search/?product=Firefox&_facets=largest_free_vm_block&_columns=largest_free_vm_block

This field is available as a string. It will be quite hard to make it a number, and even harder to have, in Super Search, any way to do something like "largest_free_vm_block < raw.dumpOOMAllocationSize". However, I believe this will be possible in the telemetry world, and it is definitely possible to do that using the Custom Search page. 

Marking this as resolved, please reopen if that seems wrong.