1315964 - Remove number_of_processors field from SuperSearch (or use the cpu count info that is available in every crash report)

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Marco Castelluccio [:marco]]

The `number_of_processors` field uses the NumberOfProcessors annotation, which was introduced in bug 764660 only for plugin hangs.

We have the information about the number of processors in every crash report, so that annotation is not useful (and can be removed, bug 1315935).

We should either:
1) Remove the `number_of_processors` field from SuperSearch;
2) Repurpose the field (keeping the same name or renaming it) to use the data that is available in any crash report.

Comment 1

[[DEACTIVATED] Adrian Gaudebert]

I believe cpu_count is only accessible in the JSON dump. Thus it cannot be used in Super Search. Maybe we want to add ``system_info`` to the list of fields we keep in the JSON dump? This way all of these fields will become available in Super Search: 

"system_info": {
    "type": "object",
    "properties": {
        "cpu_arch": {
            "type": ["string", "null"],
            "description": "Architecture of the CPU."
        },
        "cpu_count": {
            "type": ["integer", "null"],
            "description": "Number of CPU cores."
        },
        "cpu_info": {
            "type": ["string", "null"],
            "description": "Extended name of the CPU."
        },
        "os": {
            "type": ["string", "null"],
            "description": "Operating System."
        },
        "os_ver": {
            "type": ["string", "null"],
            "description": "Operating System Version."
        }
    }
},

Comment 2

[Marco Castelluccio [:marco]]

(In reply to Adrian Gaudebert [:adrian] from comment #1)
> I believe cpu_count is only accessible in the JSON dump. Thus it cannot be
> used in Super Search.

That is correct (actually, we do have the CPU count in the `cpu_info` field, the number after '|', but it's clearly less easily searchable than a separate field).

I don't personally need the field for my correlation tool (as I can just parse `cpu_info`), but in theory it can be useful in SuperSearch.

(In reply to Adrian Gaudebert [:adrian] from comment #1)
> Maybe we want to add ``system_info`` to the list of
> fields we keep in the JSON dump? This way all of these fields will become
> available in Super Search: 
> 
> "system_info": {
>     "type": "object",
>     "properties": {
>         "cpu_arch": {
>             "type": ["string", "null"],
>             "description": "Architecture of the CPU."
>         },
>         "cpu_count": {
>             "type": ["integer", "null"],
>             "description": "Number of CPU cores."
>         },
>         "cpu_info": {
>             "type": ["string", "null"],
>             "description": "Extended name of the CPU."
>         },
>         "os": {
>             "type": ["string", "null"],
>             "description": "Operating System."
>         },
>         "os_ver": {
>             "type": ["string", "null"],
>             "description": "Operating System Version."
>         }
>     }
> },

Only cpu_count is missing, right? `os` is `platform` and `os_ver` is `platform_version`. Or are they different? If they are, we might not want to add them as it could be confusing to have different fields with the same info (unless they are adding more info compared to `platform` and `platform_version`).

Comment 3

[[DEACTIVATED] Adrian Gaudebert]

(In reply to Marco Castelluccio [:marco] from comment #2)
> Only cpu_count is missing, right? `os` is `platform` and `os_ver` is
> `platform_version`. Or are they different? If they are, we might not want to
> add them as it could be confusing to have different fields with the same
> info (unless they are adding more info compared to `platform` and
> `platform_version`).

That is correct. However we do not have to expose those other fields in Super Search. The data would be stored in Elasticsearch and not being used, but for such simple fields I'm sure that's negligible. And it's quite easier to just add a new white-listed field, and it avoids having to copy yet another field into the processed_crash. :)

Comment 4

[[DEACTIVATED] Adrian Gaudebert]

I filed bug 1316224 to take care of adding the field. Once it lands, we can change the Super Search Fields list to reflect the change.

Comment 5

[[DEACTIVATED] Adrian Gaudebert]

I have added a ``cpu_count`` field on stage and prod, which contains the actual number of processors. Now the remaining question is, what do we want to do with `number_of_processors`? Should it just be an alias of `cpu_count`?

Comment 6

[Marco Castelluccio [:marco]]

Do we have a way to tell if it is being used? If nobody is using it, I guess we can remove it.
If we don't have a way to tell, then I would make it an alias.

Comment 7

[[DEACTIVATED] Adrian Gaudebert]

As far as I know we don't have any way of knowing that. Plus I am quite reluctant at removing arguments from the SuperSearch public API. So let's go for an alias!

Comment 8

[[DEACTIVATED] Adrian Gaudebert]

Done on stage and prod.