Open Bug 912928 Opened 11 years ago Updated 2 years ago

The number of crash and assertion OOM bugs is too damn high [meta]

Categories

(Core :: JavaScript Engine, defect)

Product:
Core
Component:
JavaScript Engine
Platform:
All
Linux
Type:
defect

Tracking

()

People

(Reporter: decoder, Unassigned)

References

(Depends on 2 open bugs, Blocks 1 open bug)

Details

(Keywords: meta, sec-want) 

The number of crashes and assertions related to OOM conditions keeps growing (every larger new piece of code introduces additional OOM failures). Today, I count at least a dozen signatures related to OOM (probably more), that I see daily in the fuzzer. While I understand that fixing these bugs is not priority, the reality is that they don't get fixed at all. This is dangerous because some of these bugs have somewhat generic signatures and other bugs with the same signatures might go unseen because of that.

With the testing function added in bug 872823, the fuzzer can easily create short test cases for some of these OOM bugs. However, these tests are usually short-lived and a developer needs to start working on it right after it was filed, not a month or two later (when it's guaranteed to not reproduce and also impossible to tell if it's fixed or not).

I'd like to discuss ways to achieve this. Even a commitment to fixing only *one* OOM bug per week would quickly drive down the number of bugs. Even one per month would improve the situation.
Like you said, it's not a high priority for us and most of these bugs are not dangerous, but I agree there are probably some serious issues.

It's a bit like the differential testing bugs Gary is filing. We have to fix many old bugs first; that will take a while and many of them are edge cases, but after that we will hopefully be able to catch new and more interesting bugs much sooner.

It would be nice if somebody could step up and triage/fix a few of these OOM bugs every week. I'd be happy to do it, but getting the differential testing bugs fixed already takes a lot of time. Anybody else maybe?
Naveed, Hannes suggests that I file 1-2 OOM bugs (starting with the most common), and get you to assign an owner for them. Once they are solved, I can file a new one.

Due to the nature of OOM bugs, it is often not possible to bisect them at all. The initial owner could be any JS dev who then either fixes the problem directly, or assigns it to the person working in the area where the OOM is likely happening and not being handled properly.
Flags: needinfo?(nihsanullah)
I'm opposed to asking decoder to throttle these bugs on principle. We're talking about a quality problem, possibly a security problem.

Decoder offered this search as a starting point for OOM bugs:
  http://tinyurl.com/k8uscwm

I see 15 open bugs, including some old ones that might be gone.
(In reply to Jason Orendorff [:jorendorff] from comment #3)
> I'm opposed to asking decoder to throttle these bugs on principle. We're
> talking about a quality problem, possibly a security problem.

I used to throttle my Differential Testing bugs too until jandem picked them up recently and consistently kept fixing them (or assigned people to fix them), since he could do the initial triage and/or analysis and assign the right person.

Otherwise it's just too much to handle for a non-JS tech person, we need a point person for OOM bugs. Not really to fix everything, but to handle the initial triage/analysis in a role similar to jandem.
If Christian and Gary can order these by importance (e.g. likelyhood of masking more important signatures), then I think this should actually be quite a useful activity. I would be happy do triage, although I may have to rate-limit it to a handful a week.
Depends on: 871862
Depends on: 877437
Judging by the ones that *have* been fixed, about half seem to be due to failure to check for NULL return values.

You may be able to identify these easily:

  * the symptom is either "Assertion failure: foo != NULL" or "Assertion failure: foo"
    or some similar assertion, or a crash near NULL

  * they reproduce reliably -- might even bisect, if you can do that for OOMAfter bugs.

And they're easy to fix. So these should be filed as regressions, and if you find one that's a recent regression, I think we should back out the original patch if the committer isn't around or doesn't have time to fix it at the moment.

The other half don't seem to have a single common source.
Depends on: 914598
Depends on: 914601
Depends on: 914614
Depends on: 915497
Depends on: 917759
Depends on: 925146
Depends on: 929065
Depends on: 915336
Depends on: 932530
Depends on: 930526
Jason and Terrence thanks for taking on this triage
Flags: needinfo?(nihsanullah)
Depends on: 945568
Depends on: 945754
Depends on: 947233
Depends on: 947958
Depends on: 947963
Depends on: 948023
Depends on: 948187
Depends on: 948188
Depends on: 948233
Depends on: 937083
Depends on: 948647
Depends on: 950474
Depends on: 950658
Depends on: 929221
Keywords: meta
Summary: The number of OOM bugs is too damn high → The number of crash and assertion OOM bugs is too damn high [meta]
Depends on: 958598
Depends on: 959167
Depends on: 959208
Depends on: 964803
Depends on: 978802
Depends on: 987910
Depends on: 987933
Depends on: 987935
Depends on: 987947
Depends on: 988097
Depends on: 990071
Depends on: 990096
Depends on: 990787
Depends on: 990806
Depends on: 991027
Depends on: 991036
Depends on: 991249
Depends on: 992968
Depends on: 992274
Depends on: 994159
Depends on: 1000182
Depends on: 1000145
Depends on: 1026465
Depends on: 1026476
Assignee: general → nobody
Depends on: 1164532
Depends on: 1171909
Depends on: 1175755
Depends on: 1177122
Depends on: 1188296
Depends on: 1111201
Depends on: 1188301
Depends on: 1188347
Depends on: 1133630
Depends on: 1130672
Depends on: 1180064
Depends on: 1186982
Depends on: 1188390
Depends on: 1188878
Depends on: 1191756
Depends on: 1191758
Depends on: 1193039
Depends on: 1193043
Depends on: 1193102
Depends on: 1195452
Depends on: 1196027
Depends on: 1199175
Depends on: 1204721
Depends on: 1204725
Depends on: 1204847
Depends on: 1204849
Depends on: 1204866
Depends on: 1205603
Depends on: 1205639
Depends on: 1205708
Depends on: 1206539
Depends on: 1206677
Depends on: 1207413
Depends on: 1207569
Depends on: 1207574
Depends on: 1207863
Depends on: 1208994
Depends on: 1209001
Depends on: 1209026
Depends on: 1209497
Depends on: 1209585
Depends on: 1209943
Depends on: 1209945
Depends on: 1211009
Depends on: 1211913
Depends on: 1211939
Depends on: 1211949
Depends on: 1211956
Depends on: 1211962
Depends on: 1211964
Depends on: 1211977
Depends on: 1212094
Depends on: 1212258
Depends on: 1212278
Depends on: 1212279
Depends on: 1212296
Depends on: 1212298
Depends on: 1212343
Depends on: 1212389
Depends on: 1212390
Depends on: 1189343
Depends on: 1212927
Depends on: 1155618
Depends on: 1212469
Depends on: 1214175
Depends on: 1215058
Depends on: 1215363
Depends on: 1215600
Depends on: 1215678
Depends on: 1216157
Depends on: 1216261
Depends on: 1216599
Depends on: 1216607
Depends on: 1223021
Depends on: 1223023
Depends on: 1225078
Depends on: 1232676
Depends on: 1233115
Depends on: 1234280
Depends on: 1234387
Depends on: 1234402
Depends on: 1234410
Depends on: 1234411
Depends on: 1234414
Depends on: 1236473
Depends on: 1236476
Depends on: 1236525
Depends on: 1238555
Depends on: 1238575
Depends on: 1238577
Depends on: 1238582
Depends on: 1238610
Depends on: 1240502
Depends on: 1240503
Depends on: 1240521
Depends on: 1240527
Depends on: 1240546
Depends on: 1240736
Depends on: 1240803
Depends on: 1241731
Depends on: 1242279
Depends on: 1242812
Depends on: 1242835
Depends on: 1242840
Depends on: 1243374
Depends on: 1243397
Depends on: 1243410
Depends on: 1243787
Depends on: 1245520
Depends on: 1245862
Depends on: 1246607
Depends on: 1248101
Depends on: 1252329
Depends on: 1252707
Depends on: 1252903
Depends on: 1253124
Depends on: 1254122
Depends on: 1254123
Depends on: 1254172
Depends on: 1254190
Depends on: 1254203
Depends on: 1254578
Depends on: 1255954
Depends on: 1255956
Depends on: 1257194
Depends on: 1258999
Depends on: 1260259
Depends on: 1260725
Depends on: 1261308
Depends on: 1261329
Depends on: 1261342
Depends on: 1262936
Depends on: 1263862
Depends on: 1263865
Depends on: 1263868
Depends on: 1263870
Depends on: 1263871
Depends on: 1263874
Depends on: 1263879
Depends on: 1263884
Depends on: 1263886
Depends on: 1263895
Depends on: 1263902
Depends on: 1264612
Depends on: 1264823
Depends on: 1264948
Depends on: 1264954
Depends on: 1264961
Depends on: 1264998
Depends on: 1265690
Depends on: 1265693
Depends on: 1268309
Depends on: 1269705
Depends on: 1269710
Depends on: 1269714
Depends on: 1269718
Depends on: 1269722
Depends on: 1269755
Depends on: 1269756
Depends on: 1269759
Depends on: 1278193
Depends on: 1278839
Depends on: 1282743
Depends on: 1282986
Depends on: 1284485
Depends on: 1284491
Depends on: 1285217
Depends on: 1285927
Depends on: 1285934
Depends on: 1287411
Depends on: 1287412
Depends on: 1292564
Depends on: 1293311
Depends on: 1296661
Depends on: 1296667
Depends on: 1296669
Depends on: 1297142
Depends on: 1298355
Depends on: 1298776
Depends on: 1298804
Depends on: 1299103
Depends on: 1299106
Depends on: 1299115
Depends on: 1302411
Depends on: 1302417
Depends on: 1303015
Depends on: 1305739
Depends on: 1305791
Depends on: 1315946
Depends on: 1317329
Depends on: 1328151
Depends on: 1344265
Depends on: 1344334
Depends on: 1344673
Depends on: 1345453
Depends on: 1346810
Depends on: 1347120
Depends on: 1348903
Depends on: 1348904
Depends on: 1367526
Depends on: 1368570
Depends on: 1368577
Depends on: 1368582
Depends on: 1368735
Depends on: 1369773
Depends on: 1369994
Depends on: 1370905
Depends on: 1371283
Depends on: 1373663
Depends on: 1375446
Depends on: 1382434
Depends on: 1385428
Depends on: 1385842
Depends on: 1390476
Depends on: 1390856
Depends on: 1390861
Depends on: 1394505
Depends on: 1394523
Depends on: 1397411
Depends on: 1402534
Depends on: 1402649
Depends on: 1402876
Depends on: 1404306
Depends on: 1406447
Depends on: 1406455
Depends on: 1411294
Depends on: 1412285
Depends on: 1412653
Depends on: 1415948
Depends on: 1418971
Depends on: 1421565
Depends on: 1425691
Depends on: 1433502
Depends on: 1434391
Depends on: 1435295
Depends on: 1435327
Depends on: 1448691
Depends on: 1449589
Depends on: 1455702
Depends on: 1455703
Depends on: 1456524
Depends on: 1456536
Depends on: 1459244
Depends on: 1459258
Depends on: 1461324
Depends on: 1462337
Depends on: 1462341
Depends on: 1466626
Depends on: 1471371
Depends on: 1471949
Depends on: 1472638
Depends on: 1472734
Depends on: 1473956
Depends on: 1480390
Depends on: 1483016
Depends on: 1483182
Depends on: 1488584
Depends on: 1491326
Depends on: 1491337
Depends on: 1491350
Depends on: 1491353
Depends on: 1493627
Depends on: 1494752
Depends on: 1497612
Depends on: 1497906
Depends on: 1500744
Depends on: 1502012
Depends on: 1502886
Depends on: 1503722
Depends on: 1530643
Depends on: 1547655
Depends on: 1556155
Depends on: 1574877
Depends on: 1576865
Depends on: 1597206
Depends on: 1732601
Depends on: 1737495
Depends on: 1757191

In the process of migrating remaining bugs to the new severity system, the severity for this bug cannot be automatically determined. Please retriage this bug using the new severity system.

Severity: major → --
You need to log in before you can comment on or make changes to this bug.