1149085 - Raise out of index range while merge dump

Status: RESOLVED FIXED

(Toolkit :: about:memory, defect)

Description

[Paul Yang [:pyang] (away)]

Seems related to bug 1145005 
Is it possible there's no dump, even index 0, resulting to fail at line 181?

Following is traceback log:

Traceback (most recent call last):     tools.get_about_memory.get_and_show_info(memory_report_args(output_directory=mem_dir))
   File "/var/jenkins/workspace/flamekk.v2.2.moztwlab01.319.mtbf_op@5/label/moztwlab-01/tools/get_about_memory.py", line 241, in get_and_show_info
     (out_dir, merged_reports_path, dmd_files) = get_dumps(args)
   File "/var/jenkins/workspace/flamekk.v2.2.moztwlab01.319.mtbf_op@5/label/moztwlab-01/tools/get_about_memory.py", line 237, in get_dumps
     return utils.run_and_delete_dir_on_exception(do_work, out_dir)
   File "/var/jenkins/workspace/flamekk.v2.2.moztwlab01.319.mtbf_op@5/label/moztwlab-01/tools/include/device_utils.py", line 177, in run_and_delete_dir_on_exception
     return fun()
   File "/var/jenkins/workspace/flamekk.v2.2.moztwlab01.319.mtbf_op@5/label/moztwlab-01/tools/get_about_memory.py", line 226, in do_work
     merged_reports_path = merge_files(out_dir, memory_report_files)
   File "/var/jenkins/workspace/flamekk.v2.2.moztwlab01.319.mtbf_op@5/label/moztwlab-01/tools/get_about_memory.py", line 181, in merge_files
     merged_dump = dumps[0]
 IndexError: list index out of range

Comment 1

[Paul Yang [:pyang] (away)]

ni? erahm for further investigation.

Comment 2

[Eric Rahm [:erahm]]

The call to |utils.notify_and_pull_files| should have thrown an exception if not enough memory reports were found, so I'm surprised we got to the merge step. I'll see if I can reproduce something like this locally.

I don't think this is related to bug 1145005 as that had to do with causing an OOM on the host system.

Is it possible to get a larger portion of the log? There should have been some information logged about gathering memory reports before the exception occurred.

Comment 3

[Eric Rahm [:erahm]]

Oh, sorry I was thinking of a previous bug. This could certainly be the cause of bug 1145005.

Comment 4

[Paul Yang [:pyang] (away)]

I'll see if I could get more information next run, it's not easy to reproduce.

Comment 5

[Eric Rahm [:erahm]]

Through code inspection it looks like this could happen if all child processes died (unlikely, but this is mtbf so I suppose that could happen). We should probably handle this gracefully, I'll try to think of a proper solution.

Comment 6

[Eric Rahm [:erahm]]

Attachment: Handle if no memory reports are retrieved

:mccr8, do you mind taking a look at this? I test by mocking all the processes dying and it didn't crash.

Comment 7

[Andrew McCreight [:mccr8]]

Comment on attachment 8587107 [details] [review]
Handle if no memory reports are retrieved

r=me.  Somebody else will have to merge it.

Comment 8

[Eric Rahm [:erahm]]

https://github.com/mozilla-b2g/B2G/commit/c3c8bd1bc19806e551e45071228600772ccf33b5

Comment 9

[Eric Rahm [:erahm]]

:pyang, update your checkout of mozilla-b2g/B2G and this should be fixed. Feel free to reopen if you see this issue again.