1702411 - [meta] Monitor and fix QM_TRY failures

Status: REOPENED

(Core :: Storage: Quota Manager, task)

Description

[Jens Stutte [:jstutte]]

This bugs keeps track of the semi-automatic monitoring of QM_TRY failures.
Specific bugs will be filed underneath.

Comment 1

[Jens Stutte [:jstutte]]

Attachment: until_1617178176970

Comment 2

[Jens Stutte [:jstutte]]

Attachment: until_1617194371121

Comment 3

[Jens Stutte [:jstutte]]

Attachment: until_1634520128111

Comment 4

[Jens Stutte [:jstutte]]

Attachment: until_1617667188640.txt

Comment 5

[Jens Stutte [:jstutte]]

:janv, I am a bit puzzled to see QM_TRY_INSPECT as a function name here?

Clients	Sessions	Hits	Anchor	Stack
5	5	5	dom/quota/ActorsParent.cpp:QM_TRY_INSPECT	dom/quota/ActorsParent.cpp#667:None <- dom/quota/ActorsParent.cpp#5875:NS_ERROR_FILE_CORRUPTED
3	3	3	dom/quota/ActorsParent.cpp:QM_TRY_INSPECT	dom/quota/ActorsParent.cpp#667:None <- dom/quota/ActorsParent.cpp#5875:NS_ERROR_FILE_CORRUPTED
1	1	1	dom/quota/ActorsParent.cpp:QM_TRY_INSPECT	dom/quota/ActorsParent.cpp#2399:WIN32(0x570) <- dom/quota/ActorsParent.cpp#4647:WIN32(0x570) <- dom/quota/ActorsParent.cpp#4230:WIN32(0x570) <- dom/quota/ActorsParent.cpp#4233:WIN32(0x570) <- dom/quota/ActorsParent.cpp#6234:WIN32(0x570)
1	1	1	dom/quota/ActorsParent.cpp:QM_TRY_INSPECT	dom/quota/ActorsParent.cpp#2399:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4647:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4230:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4233:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#6234:NS_ERROR_FILE_ACCESS_DENIED

Comment 6

[Jan Varga [:janv]]

My first guess is that rust-code-analysis-cli is confused by the macro in some places. I would need to see the metrics file to tell more. Should I try myself ?

Comment 7

[Jan Varga [:janv]]

Or maybe the script needs a list of functions that should be ignored (if QM_TRY is considered as a function) and use other function name which is not the closest one.

Comment 8

[Jens Stutte [:jstutte]]

(In reply to Jan Varga [:janv] from comment #6)

My first guess is that rust-code-analysis-cli is confused by the macro in some places. I would need to see the metrics file to tell more. Should I try myself ?

This seems to be the case. Fetching ActorsParent.cpp manually and applying rust-code-analysis-cli -m -O json -o /tmp -p /tmp/ActorsParent.cpp manually yields:

...
             {
                    "name": "LoadLocalStorageArchiveVersion",
                    "start_line": 658,
                    "end_line": 755,
                    "kind": "function",
                    "spaces": [
                        {
                            "name": "QM_TRY_INSPECT",
                            "start_line": 662,
                            "end_line": 745,
                            "kind": "function",
                            "spaces": [],
                            "metrics": {
                                "nargs": {
                                    "sum": 7.0,
                                    "average": 7.0
                                },
                                "nexits": {
                                    "sum": 0.0,
                                    "average": 0.0
                                },
                                "cognitive": {
                                    "sum": 0.0,
                                    "average": 0.0
                                },
                                "cyclomatic": {
                                    "sum": 7.0,
                                    "average": 7.0
                                },
                                "halstead": {
                                    "n1": 19.0,
                                    "N1": 157.0,
                                    "n2": 65.0,
                                    "N2": 119.0,
                                    "length": 276.0,
                                    "estimated_program_length": 472.16453060227764,
                                    "purity_ratio": 1.710741052906803,
                                    "vocabulary": 84.0,
                                    "volume": 1764.279608686938,
                                    "difficulty": 17.392307692307693,
                                    "level": 0.05749668288367979,
                                    "effort": 30684.893809547437,
                                    "time": 1704.7163227526353,
                                    "bugs": 0.32670954919749584
                                },
                                "loc": {
                                    "sloc": 84.0,
                                    "ploc": 60.0,
                                    "lloc": 1.0,
                                    "cloc": 5.0,
                                    "blank": 19.0
                                },
                                "nom": {
                                    "functions": 1.0,
                                    "closures": 0.0,
                                    "total": 1.0
                                },
                                "mi": {
                                    "mi_original": 58.738179650614256,
                                    "mi_sei": 28.20463653446185,
                                    "mi_visual_studio": 34.34981266117793
                                }
                            }
                        },
                        {
                            "name": "~Observer",
                            "start_line": 747,
                            "end_line": 747,
                            "kind": "function",
                            "spaces": [],
                            "metrics": {
                                "nargs": {
                                    "sum": 0.0,
                                    "average": 0.0
                                },
                                "nexits": {
                                    "sum": 0.0,
                                    "average": 0.0
                                },
                                "cognitive": {
                                    "sum": 0.0,
                                    "average": 0.0
                                },
                                "cyclomatic": {
                                    "sum": 1.0,
                                    "average": 1.0
                                },
                                "halstead": {
                                    "n1": 4.0,
                                    "N1": 6.0,
                                    "n2": 3.0,
                                    "N2": 3.0,
                                    "length": 9.0,
                                    "estimated_program_length": 12.754887502163468,
                                    "purity_ratio": 1.4172097224626077,
                                    "vocabulary": 7.0,
                                    "volume": 25.26619429851844,
                                    "difficulty": 2.0,
                                    "level": 0.5,
                                    "effort": 50.53238859703688,
                                    "time": 2.807354922057604,
                                    "bugs": 0.004556086527391188
                                },
                                "loc": {
                                    "sloc": 1.0,
                                    "ploc": 1.0,
                                    "lloc": 1.0,
                                    "cloc": 0.0,
                                    "blank": 0.0
                                },
                                "nom": {
                                    "functions": 1.0,
                                    "closures": 0.0,
                                    "total": 1.0
                                },
                                "mi": {
                                    "mi_original": 153.97676999581998,
                                    "mi_sei": 146.54249035246173,
                                    "mi_visual_studio": 90.04489473439764
                                }
                            }
                        }
                    ],
                    "metrics": {
                        "nargs": {
                            "sum": 8.0,
                            "average": 2.6666666666666665
                        },
                        "nexits": {
                            "sum": 0.0,
                            "average": 0.0
                        },
                        "cognitive": {
                            "sum": 0.0,
                            "average": 0.0
                        },
                        "cyclomatic": {
                            "sum": 9.0,
                            "average": 3.0
                        },
                        "halstead": {
                            "n1": 20.0,
                            "N1": 176.0,
                            "n2": 68.0,
                            "N2": 132.0,
                            "length": 308.0,
                            "estimated_program_length": 500.38603510277034,
                            "purity_ratio": 1.6246299840999037,
                            "vocabulary": 88.0,
                            "volume": 1989.5049385402876,
                            "difficulty": 19.41176470588235,
                            "level": 0.05151515151515152,
                            "effort": 38619.80174813499,
                            "time": 2145.5445415630547,
                            "bugs": 0.3808483736064195
                        },
                        "loc": {
                            "sloc": 98.0,
                            "ploc": 69.0,
                            "lloc": 3.0,
                            "cloc": 5.0,
                            "blank": 24.0
                        },
                        "nom": {
                            "functions": 3.0,
                            "closures": 0.0,
                            "total": 3.0
                        },
                        "mi": {
                            "mi_original": 55.15619306227286,
                            "mi_sei": 21.9305595486369,
                            "mi_visual_studio": 32.25508366214787
                        }
                    }
                },
...

The line numbers are clearly wrong here. And similar problems are happening also in other places around QM_TRY_INSPECT. As the line numbers also of LoadLocalStorageArchiveVersion itself are wrong (end line), I fear that ignoring QM_* function names might not always help.

:marco, is this something you ever noticed in rust-code-analysis-cli?

Comment 9

[Andrew Sutherland [:asuth] (he/him)]

Is there a way to get rust-code-analysis to dump the tree-sitter AST/parse? I presume there's an error node in the given method that causes it to miss the end of the function and eat further.

Comment 10

[Calixte Denizet (:calixte)]

:asuth, yes it's possible:

cargo run -p rust-code-analysis-cli -- -p /tmp/ActorsParent.cpp -d --ls 658 --le 671

ls and le are here to limit the ast dump to nodes which are between these two line numbers (included).
and the problem is because of

QM_TRY_INSPECT(const auto& stmt, ...

because it isn't really grammatically correct.
In rca we've a grammar for mozilla-central code:
https://github.com/mozilla/rust-code-analysis/blob/master/tree-sitter-mozcpp/grammar.js
so we can amend it to handle special cases (it's the goal of this grammar), feel free to file a bug.

Comment 11

[Calixte Denizet (:calixte)]

I wrote a patch for rca to fix the problem:
https://github.com/mozilla/rust-code-analysis/pull/611

Comment 12

[Jens Stutte [:jstutte]]

(In reply to Calixte Denizet (:calixte) from comment #11)

I wrote a patch for rca to fix the problem:
https://github.com/mozilla/rust-code-analysis/pull/611

Thanks a lot!

Comment 13

[Jens Stutte [:jstutte]]

It works:

< 5 | 5 | 5 | dom/quota/ActorsParent.cpp:QM_TRY_INSPECT | [dom/quota/ActorsParent.cpp#667:None](https://hg.mozilla.org/releases/mozilla-beta/annotate/276ecc030f1c341ecf57287999ff4bec10adfb4f/dom/quota/ActorsParent.cpp#l667) <- [dom/quota/ActorsParent.cpp#5875:NS_ERROR_FILE_CORRUPTED](https://hg.mozilla.org/releases/mozilla-beta/annotate/276ecc030f1c341ecf57287999ff4bec10adfb4f/dom/quota/ActorsParent.cpp#l5875)
---
> 5 | 5 | 5 | dom/quota/ActorsParent.cpp:LoadLocalStorageArchiveVersion | [dom/quota/ActorsParent.cpp#667:None](https://hg.mozilla.org/releases/mozilla-beta/annotate/276ecc030f1c341ecf57287999ff4bec10adfb4f/dom/quota/ActorsParent.cpp#l667) <- [dom/quota/ActorsParent.cpp#5875:NS_ERROR_FILE_CORRUPTED](https://hg.mozilla.org/releases/mozilla-beta/annotate/276ecc030f1c341ecf57287999ff4bec10adfb4f/dom/quota/ActorsParent.cpp#l5875)

Thanks!

Comment 14

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1618012793220.txt

Comment 15

[Jens Stutte [:jstutte]]

Needs new bug:

Clients	Sessions	Hits	Anchor	Stack
2	2	2	dom/quota/ActorsParent.cpp:LoadLocalStorageArchiveVersion	dom/quota/ActorsParent.cpp#667:None <- dom/quota/ActorsParent.cpp#5875:NS_ERROR_FILE_CORRUPTED

Comment 16

[Jens Stutte [:jstutte]]

Needs new bug:

Clients	Sessions	Hits	Anchor	Stack
2	2	2	dom/quota/ActorsParent.cpp:QuotaManager::EnsureStorageIsInitialized	dom/quota/ActorsParent.cpp#6011:NS_ERROR_FILE_ACCESS_DENIED
1	1	1	dom/quota/ActorsParent.cpp:QuotaManager::EnsureStorageIsInitialized	dom/quota/ActorsParent.cpp#6011:NS_ERROR_STORAGE_BUSY

Comment 17

[Jens Stutte [:jstutte]]

Needs new bug:

Clients	Sessions	Hits	Anchor	Stack
1	1	1	dom/quota/ActorsParent.cpp:CreateDirectoryMetadata2	dom/quota/ActorsParent.cpp#2568:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#11027:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#9841:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#11015:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4534:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4619:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4622:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4678:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4758:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4222:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#6221:NS_ERROR_FILE_ACCESS_DENIED
1	1	1	dom/quota/ActorsParent.cpp:CreateDirectoryMetadata2	dom/quota/ActorsParent.cpp#2542:NS_ERROR_FAILURE <- dom/quota/ActorsParent.cpp#11044:NS_ERROR_FAILURE <- dom/quota/ActorsParent.cpp#9856:NS_ERROR_FAILURE <- dom/quota/ActorsParent.cpp#11032:NS_ERROR_FAILURE <- dom/quota/ActorsParent.cpp#4542:NS_ERROR_FAILURE <- dom/quota/ActorsParent.cpp#4632:NS_ERROR_FAILURE <- dom/quota/ActorsParent.cpp#4693:NS_ERROR_FAILURE <- dom/quota/ActorsParent.cpp#4773:NS_ERROR_FAILURE <- dom/quota/ActorsParent.cpp#4777:NS_ERROR_FAILURE <- dom/quota/ActorsParent.cpp#4780:NS_ERROR_FAILURE <- dom/quota/ActorsParent.cpp#4230:NS_ERROR_FAILURE <- dom/quota/ActorsParent.cpp#4233:NS_ERROR_FAILURE <- dom/quota/ActorsParent.cpp#6234:NS_ERROR_FAILURE

Comment 18

[Jens Stutte [:jstutte]]

Needs new bug:

Clients	Sessions	Hits	Anchor	Stack
1	1	1	dom/quota/ActorsParent.cpp:MaybeUpdateGroupForOrigin	dom/quota/ActorsParent.cpp#1829:NS_ERROR_MALFORMED_URI <- dom/quota/ActorsParent.cpp#4596:NS_ERROR_MALFORMED_URI <- dom/quota/ActorsParent.cpp#4628:NS_ERROR_MALFORMED_URI

Comment 19

[Jens Stutte [:jstutte]]

Needs new bug:

Clients	Sessions	Hits	Anchor	Stack
1	1	1	dom/quota/ActorsParent.cpp:QuotaManager::InitializeRepository	dom/quota/ActorsParent.cpp#4735:NS_ERROR_FILE_FS_CORRUPTED <- dom/quota/ActorsParent.cpp#4758:NS_ERROR_FILE_FS_CORRUPTED <- dom/quota/ActorsParent.cpp#4222:NS_ERROR_FILE_FS_CORRUPTED <- dom/quota/ActorsParent.cpp#6221:NS_ERROR_FILE_FS_CORRUPTED
1	1	1	dom/quota/ActorsParent.cpp:QuotaManager::InitializeRepository	dom/quota/ActorsParent.cpp#4735:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4758:NS_ERROR_FILE_ACCESS_DENIED
1	1	1	dom/quota/ActorsParent.cpp:QuotaManager::InitializeRepository	dom/quota/ActorsParent.cpp#4735:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4758:NS_ERROR_FILE_ACCESS_DENIED
1	1	183	dom/quota/ActorsParent.cpp:QuotaManager::InitializeRepository	dom/quota/ActorsParent.cpp#4735:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4758:NS_ERROR_FILE_ACCESS_DENIED
1	1	2	dom/quota/ActorsParent.cpp:QuotaManager::InitializeRepository	dom/quota/ActorsParent.cpp#4735:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4758:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4222:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#6221:NS_ERROR_FILE_ACCESS_DENIED
1	1	1	dom/quota/ActorsParent.cpp:QuotaManager::InitializeRepository	dom/quota/ActorsParent.cpp#4735:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4758:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4222:NS_ERROR_FILE_ACCESS_DENIED
1	1	62	dom/quota/ActorsParent.cpp:QuotaManager::InitializeRepository	dom/quota/ActorsParent.cpp#4735:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4758:NS_ERROR_FILE_ACCESS_DENIED
1	1	1	dom/quota/ActorsParent.cpp:QuotaManager::InitializeRepository	dom/quota/ActorsParent.cpp#4735:NS_ERROR_FILE_IS_LOCKED <- dom/quota/ActorsParent.cpp#4758:NS_ERROR_FILE_IS_LOCKED
1	1	2	dom/quota/ActorsParent.cpp:QuotaManager::InitializeRepository	dom/quota/ActorsParent.cpp#4735:NS_ERROR_FILE_IS_LOCKED <- dom/quota/ActorsParent.cpp#4758:NS_ERROR_FILE_IS_LOCKED
1	1	2	dom/quota/ActorsParent.cpp:QuotaManager::InitializeRepository	dom/quota/ActorsParent.cpp#4735:NS_ERROR_FILE_READ_ONLY <- dom/quota/ActorsParent.cpp#4758:NS_ERROR_FILE_READ_ONLY

Comment 20

[Jens Stutte [:jstutte]]

Needs new bug:

Clients	Sessions	Hits	Anchor	Stack
1	1	1	dom/quota/QuotaCommon.cpp:GetDirEntryKind	dom/quota/QuotaCommon.cpp#186:NS_ERROR_FILE_ACCESS_DENIED <- dom/indexedDB/ActorsParent.cpp#13125:NS_ERROR_FILE_ACCESS_DENIED <- dom/indexedDB/ActorsParent.cpp#13184:NS_ERROR_FILE_ACCESS_DENIED <- dom/indexedDB/ActorsParent.cpp#12772:NS_ERROR_FILE_ACCESS_DENIED <- dom/indexedDB/ActorsParent.cpp#12731:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4867:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4939:NS_ERROR_FILE_ACCESS_DENIED
1	1	1	dom/quota/QuotaCommon.cpp:GetDirEntryKind	dom/quota/QuotaCommon.cpp#186:NS_ERROR_FILE_ACCESS_DENIED <- dom/indexedDB/ActorsParent.cpp#13125:NS_ERROR_FILE_ACCESS_DENIED <- dom/indexedDB/ActorsParent.cpp#13184:NS_ERROR_FILE_ACCESS_DENIED <- dom/indexedDB/ActorsParent.cpp#12772:NS_ERROR_FILE_ACCESS_DENIED <- dom/indexedDB/ActorsParent.cpp#12731:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4867:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4939:NS_ERROR_FILE_ACCESS_DENIED
1	1	237	dom/quota/QuotaCommon.cpp:GetDirEntryKind	dom/quota/QuotaCommon.cpp#186:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4848:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4939:NS_ERROR_FILE_ACCESS_DENIED
1	1	62	dom/quota/QuotaCommon.cpp:GetDirEntryKind	dom/quota/QuotaCommon.cpp#186:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2327:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2335:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2338:NS_ERROR_FILE_ACCESS_DENIED
1	1	62	dom/quota/QuotaCommon.cpp:GetDirEntryKind	dom/quota/QuotaCommon.cpp#186:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2327:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2335:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2338:NS_ERROR_FILE_ACCESS_DENIED
1	1	79	dom/quota/QuotaCommon.cpp:GetDirEntryKind	dom/quota/QuotaCommon.cpp#186:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4848:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#4939:NS_ERROR_FILE_ACCESS_DENIED
1	2	3	dom/quota/QuotaCommon.cpp:GetDirEntryKind	dom/quota/QuotaCommon.cpp#187:WIN32(0x45D) <- dom/cache/FileUtilsImpl.h#42:WIN32(0x45D) <- dom/cache/FileUtilsImpl.h#93:WIN32(0x45D) <- dom/cache/QuotaClient.cpp#122:WIN32(0x45D) <- dom/cache/QuotaClient.cpp#49:WIN32(0x45D) <- dom/cache/QuotaClient.cpp#52:WIN32(0x45D) <- dom/cache/QuotaClient.cpp#124:WIN32(0x45D) <- dom/cache/QuotaClient.cpp#437:WIN32(0x45D) <- dom/cache/QuotaClient.cpp#49:WIN32(0x45D) <- dom/cache/QuotaClient.cpp#52:WIN32(0x45D) <- dom/cache/QuotaClient.cpp#478:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4882:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4954:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4958:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4961:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4750:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4773:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4777:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4780:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4230:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4233:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#6234:WIN32(0x45D)
1	2	2	dom/quota/QuotaCommon.cpp:GetDirEntryKind	dom/quota/QuotaCommon.cpp#187:WIN32(0x45D) <- dom/indexedDB/ActorsParent.cpp#13127:WIN32(0x45D) <- dom/indexedDB/ActorsParent.cpp#13186:WIN32(0x45D) <- dom/indexedDB/ActorsParent.cpp#12778:WIN32(0x45D) <- dom/indexedDB/ActorsParent.cpp#12737:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4882:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4954:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4958:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4961:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4750:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4773:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4777:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4780:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4230:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#4233:WIN32(0x45D) <- dom/quota/ActorsParent.cpp#6234:WIN32(0x45D)

Comment 21

[Jens Stutte [:jstutte]]

Needs new bug:

Clients	Sessions	Hits	Anchor	Stack
1	2	2	dom/quota/ActorsParent.cpp:QuotaManager::CopyLocalStorageArchiveFromWebAppsStore	dom/quota/ActorsParent.cpp#5514:WIN32(0x570) <- dom/quota/ActorsParent.cpp#5859:WIN32(0x570) <- dom/quota/ActorsParent.cpp#5863:WIN32(0x570) <- dom/quota/ActorsParent.cpp#6041:WIN32(0x570)

Comment 22

[Jens Stutte [:jstutte]]

Needs new bug:

Clients	Sessions	Hits	Anchor	Stack
1	1	1	dom/quota/QuotaCommon.cpp:ExecuteSingleStep	dom/quota/QuotaCommon.cpp#201:NS_ERROR_OUT_OF_MEMORY <- dom/quota/ActorsParent.cpp#665:NS_ERROR_OUT_OF_MEMORY <- dom/quota/ActorsParent.cpp#5875:NS_ERROR_OUT_OF_MEMORY <- dom/quota/ActorsParent.cpp#6041:NS_ERROR_OUT_OF_MEMORY

Comment 23

[Jens Stutte [:jstutte]]

Needs new bug:

Clients	Sessions	Hits	Anchor	Stack
1	1	1	dom/localstorage/ActorsParent.cpp:UpdateUsageFile	dom/localstorage/ActorsParent.cpp#1000:NS_ERROR_FILE_IS_LOCKED <- dom/localstorage/ActorsParent.cpp#8123:NS_ERROR_FILE_IS_LOCKED <- dom/localstorage/ActorsParent.cpp#8129:NS_ERROR_FILE_IS_LOCKED <- dom/localstorage/ActorsParent.cpp#8137:NS_ERROR_FILE_IS_LOCKED <- dom/quota/ActorsParent.cpp#4867:NS_ERROR_FILE_IS_LOCKED <- dom/quota/ActorsParent.cpp#4939:NS_ERROR_FILE_IS_LOCKED
1	1	2	dom/localstorage/ActorsParent.cpp:UpdateUsageFile	dom/localstorage/ActorsParent.cpp#1000:NS_ERROR_FILE_IS_LOCKED <- dom/localstorage/ActorsParent.cpp#8123:NS_ERROR_FILE_IS_LOCKED <- dom/localstorage/ActorsParent.cpp#8129:NS_ERROR_FILE_IS_LOCKED <- dom/localstorage/ActorsParent.cpp#8137:NS_ERROR_FILE_IS_LOCKED <- dom/quota/ActorsParent.cpp#4867:NS_ERROR_FILE_IS_LOCKED <- dom/quota/ActorsParent.cpp#4939:NS_ERROR_FILE_IS_LOCKED

Comment 24

[Jens Stutte [:jstutte]]

Needs new bug:

Clients	Sessions	Hits	Anchor	Stack
1	1	1	dom/quota/ActorsParent.cpp:QuotaManager::MaybeCreateOrUpgradeStorage	dom/quota/ActorsParent.cpp#5811:NS_ERROR_FILE_NO_DEVICE_SPACE <- dom/quota/ActorsParent.cpp#6013:NS_ERROR_FILE_NO_DEVICE_SPACE
1	1	1	dom/quota/ActorsParent.cpp:QuotaManager::MaybeCreateOrUpgradeStorage	dom/quota/ActorsParent.cpp#5811:NS_ERROR_FILE_NO_DEVICE_SPACE <- dom/quota/ActorsParent.cpp#6013:NS_ERROR_FILE_NO_DEVICE_SPACE

Comment 25

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1618358255198.txt

Comment 26

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1618790286229.txt

Comment 27

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1619105913113.txt

Comment 28

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1619135971912.txt

Comment 29

[Jens Stutte [:jstutte]]

The last two files are almost a week old, I will not file the single errors.

Comment 30

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1619343297418.txt

Comment 31

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1619481593759.txt

Comment 32

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1619343297418.txt

Hopefully better split with less false errors.

Comment 33

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1619481593759.txt

Also here, hopefully better split with less false alarms.

Comment 34

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1619763202744.txt

Comment 35

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1619941123084.txt

Comment 36

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1619999990354.txt

Comment 37

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1620194412675.txt

Comment 38

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1620341086678.txt

Comment 39

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1620345595329.txt

Comment 40

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1620691195434.txt

Comment 41

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1621176646038.txt

Comment 42

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1621209594183.txt

Comment 43

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1621814376473.txt

Comment 44

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1622505580053.txt

Comment 45

[Jens Stutte [:jstutte]]

Looking at the frequency of different stacks with nsResult:

Errors:
> cat qmerrors_until_1622505580053.json | grep result | sort | uniq -c
    166                 "result": "NS_ERROR_FILE_ACCESS_DENIED",
     25                 "result": "NS_ERROR_FILE_DIR_NOT_EMPTY",
     18                 "result": "NS_ERROR_FILE_FS_CORRUPTED",
     10                 "result": "NS_ERROR_FILE_IS_LOCKED",
      3                 "result": "NS_ERROR_FILE_NO_DEVICE_SPACE",
     50                 "result": "NS_ERROR_FILE_NOT_FOUND",
      3                 "result": "NS_ERROR_FILE_READ_ONLY",
      4                 "result": null,

Warnings:
> cat qmwarnings_until_1622505580053.json | grep result | sort | uniq -c
     65                 "result": "NS_ERROR_FAILURE",
     27                 "result": "NS_ERROR_FILE_ACCESS_DENIED",
     15                 "result": "NS_ERROR_FILE_CORRUPTED",
      3                 "result": "NS_ERROR_FILE_DEVICE_FAILURE",
      1                 "result": "NS_ERROR_FILE_DIR_NOT_EMPTY",
      9                 "result": "NS_ERROR_FILE_IS_LOCKED",
      1                 "result": "NS_ERROR_FILE_NO_DEVICE_SPACE",
    108                 "result": "NS_ERROR_FILE_NOT_FOUND",
      2                 "result": "NS_ERROR_FILE_READ_ONLY",
     45                 "result": "NS_ERROR_FILE_TARGET_DOES_NOT_EXIST",
      6                 "result": "NS_ERROR_FILE_TOO_BIG",
     44                 "result": "NS_ERROR_ILLEGAL_VALUE",
      6                 "result": "NS_ERROR_MALFORMED_URI",
      7                 "result": "NS_ERROR_STORAGE_IOERR",
      6                 "result": "NS_ERROR_UNEXPECTED",
     55                 "result": null,

Please note, that this does not count, how often those stacks occur, it is just the number of unique stack frames in this script run (we could do better but not with a 5min bash line).

Things worth an investigation:

Error	Comment
NS_ERROR_FILE_DEVICE_FAILURE	Should never be a Warning?
NS_ERROR_FILE_NO_DEVICE_SPACE	Should never be a Warning?
NS_ERROR_FAILURE	See bug 1712582
NS_ERROR_UNEXPECTED	Unspecific?

Comment 46

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1622505580053.txt

Comment 47

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1622505580053.txt

Comment 48

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1622843032554.txt

Comment 49

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1623023985622.txt

Comment 50

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1624233573514_3d.txt

Comment 51

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1624492765564.txt

Comment 52

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1626047912797.txt

Comment 53

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1626307093569.txt

Comment 54

[Jens Stutte [:jstutte]]

Clients	Sessions	Hits	Anchor	Stack
1	1	1	dom/quota/ActorsParent.cpp:GetLastModifiedTime	dom/quota/ActorsParent.cpp#2353:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2334:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2337:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2334:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2337:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2334:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2337:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2334:NS_ERROR_FILE_ACCESS_DENIED <- dom/quota/ActorsParent.cpp#2337:NS_ERROR_FILE_ACCESS_DENIED

Comment 55

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1626652771473.txt

Comment 56

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1626911774132.txt

Comment 57

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1627257571485.txt

Comment 58

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1628553536260.txt

Comment 59

[Jens Stutte [:jstutte]]

New entries (bugs to file):

Clients	Sessions	Hits	Anchor	Stack
1	1	1	dom/localstorage/ActorsParent.cpp:PrepareDatastoreOp::VerifyDatabaseInformation	dom/localstorage/ActorsParent.cpp#7129:None <- dom/localstorage/ActorsParent.cpp#6940:NS_ERROR_FILE_CORRUPTED

Clients	Sessions	Hits	Anchor	Stack
1	1	1	dom/quota/ActorsParent.cpp:GetLastModifiedTime	dom/quota/ActorsParent.cpp#2356:NS_ERROR_FILE_FS_CORRUPTED <- dom/quota/ActorsParent.cpp#2337:NS_ERROR_FILE_FS_CORRUPTED <- dom/quota/ActorsParent.cpp#2340:NS_ERROR_FILE_FS_CORRUPTED <- dom/quota/ActorsParent.cpp#2337:NS_ERROR_FILE_FS_CORRUPTED <- dom/quota/ActorsParent.cpp#2340:NS_ERROR_FILE_FS_CORRUPTED

Comment 60

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1629071977941.txt

Comment 61

[Jens Stutte [:jstutte]]

Attachment: qmstacks_until_1629331180685.txt

Comment 62

[Jan Rio Krause]

Attachment: qmstacks_until_1629676796018.txt

Comment 63

[Jan Rio Krause]

Attachment: qmstacks__until_1629935894699.txt

Comment 64

[Jan Rio Krause]

Attachment: qmstacks_until_1630281585379.txt

Comment 65

[Jan Rio Krause]

Attachment: qmstacks_until_1630627166007.txt

Comment 66

[Jan Rio Krause]

Attachment: qmstacks_until_1630885802663.txt

Comment 67

[Jan Rio Krause]

Attachment: qmstacks_until_1631145538534.txt

Comment 68

[Jan Rio Krause]

Attachment: qmstacks_until_1631491186185.txt

Comment 69

[Jan Rio Krause]

Attachment: qmstacks_until_1631750187982.txt

Comment 70

[Jan Rio Krause]

Attachment: qmstacks_until_1632095869196.txt

Comment 71

[Jan Rio Krause]

Attachment: qmstacks_until_1632355158667.txt

Comment 72

[Jan Rio Krause]

Attachment: qmstacks_until_1632700787544.txt

Comment 73

[Jan Rio Krause]

Attachment: qmstacks_until_1632959843839.txt

Comment 74

[Jan Rio Krause]

Attachment: qmstacks_until_1633305487383.txt

Comment 75

[Jan Rio Krause]

Attachment: qmstacks_until_1633564768234.txt

Comment 76

[Jan Rio Krause]

Attachment: qmstacks_until_1633910361301.txt

Comment 77

[Jan Rio Krause]

Attachment: qmstacks_until_1634169503723.txt

Comment 78

[Jan Rio Krause]

Attachment: qmstacks_until_1634515097306.txt

Comment 79

[Jan Rio Krause]

Attachment: qmstacks_until_1634774361093.txt

Comment 80

[Jan Rio Krause]

Attachment: qmstacks_until_1635119978505.txt

Comment 81

[Jan Rio Krause]

Attachment: qmstacks_until_1635379016948.txt

Comment 82

[Jan Rio Krause]

Attachment: qmstacks_until_1635724728893.txt

Comment 83

[Jan Rio Krause]

Attachment: qmstacks_until_1635983991407.txt

Comment 84

[Jan Rio Krause]

Attachment: qmstacks_until_1636329466803.txt

Comment 85

[Jan Rio Krause]

Attachment: qmstacks_until_1636588768920.txt

Comment 86

[Jan Rio Krause]

Attachment: qmstacks_until_1636934392308.txt

Comment 87

[Jan Rio Krause]

Attachment: qmstacks_until_1637193513530.txt

Comment 88

[Jan Rio Krause]

Attachment: qmstacks_until_1637539049308.txt

Comment 89

[Jan Rio Krause]

Attachment: qmstacks_until_1637798080391.txt

Comment 90

[Jan Rio Krause]

Attachment: qmstacks_until_1638143989574.txt

Comment 91

[Jan Rio Krause]

Attachment: qmstacks_until_1638403192259.txt

Comment 92

[Jan Rio Krause]

Attachment: qmstacks_until_1638748794500.txt

Comment 93

[Jan Rio Krause]

Attachment: qmstacks_until_1639007988749.txt

Comment 94

[Jan Rio Krause]

Attachment: qmstacks_until_1639353588307.txt

Comment 95

[Jan Rio Krause]

Attachment: qmstacks_until_1639612771991.txt

Comment 96

[Jan Rio Krause]

Attachment: qmstacks_until_1639958375072.txt

Comment 97

[Jan Rio Krause]

Attachment: qmstacks_until_1641167959223.txt

Comment 98

[Jan Rio Krause]

Attachment: qmstacks_until_1641427138540.txt

Comment 99

[Jan Rio Krause]

Attachment: qmstacks_until_1641772769720.txt

Comment 100

[Jan Rio Krause]

Attachment: qmstacks_until_1642031976459.txt

Comment 101

[Jan Rio Krause]

Attachment: qmstacks_until_1642377470839.txt

Comment 102

[Jan Rio Krause]

Attachment: qmstacks_until_1642636782199.txt

Comment 103

[Jan Rio Krause]

Attachment: qmstacks_until_1642982395552.txt

Comment 104

[Jan Rio Krause]

Attachment: qmstacks_until_1643241507821.txt

Comment 105

[Jan Rio Krause]

Attachment: qmstacks_until_1643587193129.txt

Comment 106

[Jan Rio Krause]

Attachment: qmstacks_until_1643846356179.txt

Comment 107

[Jan Rio Krause]

Attachment: qmstacks_until_1644191965024.txt

Comment 108

[Jan Rio Krause]

Attachment: qmstacks_until_1644451126557.txt

Comment 109

[Jan Rio Krause]

Attachment: qmstacks_until_1644796780585.txt

Comment 110

[Jan Rio Krause]

Attachment: qmstacks_until_1645055858822.txt

Comment 111

[Jan Rio Krause]

Attachment: qmstacks_until_1645401502554.txt

Comment 112

[Jan Rio Krause]

Attachment: qmstacks_until_1645660724072.txt

Comment 113

[Jan Rio Krause]

Attachment: qmstacks_until_1646006386179.txt

Comment 114

[Jan Rio Krause]

Attachment: qmstacks_until_1646265500329.txt

Comment 115

[Jan Rio Krause]

Attachment: qmstacks_until_1646870386480.txt

Comment 116

[Jan Rio Krause]

Attachment: qmstacks_until_1647215984390.txt

Comment 117

[Jan Rio Krause]

Attachment: qmstacks_until_1647475063843.txt

Comment 118

[Jan Rio Krause]

Attachment: qmstacks_until_1647820601008.txt

Comment 119

[Jan Rio Krause]

Attachment: qmstacks_until_1647820601008.txt

The rust-code-analysis-cli had to be run again because previously mostly unknown anchors were returned.

Comment 120

[Jan Rio Krause]

Attachment: qmstacks_until_1648079966861.txt

Comment 121

[Jan Rio Krause]

Attachment: qmstacks_until_1648425599484.txt

Comment 122

[Jan Rio Krause]

Attachment: qmstacks_until_1649030384727.txt

Comment 123

[Jan Rio Krause]

Attachment: qmstacks_until_1650499152064.txt

Comment 124

[Jan Rio Krause]

Attachment: qmstacks_until_1650844790704.txt

Comment 125

[Jan Rio Krause]

Attachment: qmstacks_until_1651103894656.txt

Comment 126

[Jan Rio Krause]

Attachment: qmstacks_until_1651449571141.txt

Comment 127

[Jan Rio Krause]

Attachment: qmstacks_until_1651708720415.txt

Comment 128

[Jan Rio Krause]

Attachment: qmstacks_until_1653263791369.txt

Comment 129

[Jan Rio Krause]

Attachment: Bug 1702411 - Add `context` to Bugzilla comment markup. r=#dom-storage-reviewers,jstutte

Comment 130

[Jan Rio Krause]

Attachment: qmstacks_until_1653868780843.txt

Comment 131

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/13c0b769cc03
Add `context` to Bugzilla comment markup. r=jstutte,dom-storage-reviewers

Comment 132

[Sandor Molnar]

https://hg.mozilla.org/mozilla-central/rev/13c0b769cc03

Comment 133

[Jens Stutte [:jstutte]]

The closure was not intended here, this is a perennial thing.

Comment 134

[Jan Rio Krause]

Attachment: qmstacks_until_1654127928500.txt

Comment 135

[Jan Rio Krause]

Attachment: qmstacks_until_1654732729648.txt

Comment 136

[Jan Rio Krause]

Attachment: qmstacks_until_1655078380400.txt

Comment 137

[Jan Rio Krause]

Attachment: qmstacks_until_1655337542322.txt

Comment 138

[Jan Rio Krause]

Attachment: qmstacks_until_1655683115580.txt

Comment 139

[Jan Rio Krause]

Attachment: qmstacks_until_1655941964590.txt

Comment 140

[Jan Rio Krause]

Attachment: qmstacks_until_1656287950511.txt

Comment 141

[Jan Rio Krause]

Attachment: qmstacks_until_1656547186468.txt

Comment 142

[Jan Rio Krause]

Attachment: qmstacks_until_1656892599966.txt

Comment 143

[Jan Rio Krause]

Attachment: qmstacks_until_1657151891987.txt

Comment 144

[Jan Rio Krause]

Attachment: qmstacks_until_1657497539103.txt

Comment 145

[Jan Rio Krause]

Attachment: qmstacks_until_1657756654544.txt

Comment 146

[Jan Rio Krause]

Attachment: qmstacks_until_1658102343883.txt

Comment 147

[Jan Rio Krause]

Attachment: qmstacks_until_1658361434047.txt

Comment 148

[Jan Rio Krause]

Attachment: qmstacks_until_1658706908571.txt

Comment 149

[Jan Rio Krause]

Attachment: qmstacks_until_1658966348866.txt

Comment 150

[Jan Rio Krause]

Attachment: qmstacks_until_1659311955391.txt

Comment 151

[Jan Rio Krause]

Attachment: qmstacks_until_1659916530335.txt

Comment 152

[Jan Rio Krause]

Attachment: qmstacks_until_1660175951576.txt

Comment 153

[Jan Rio Krause]

Attachment: qmstacks_until_1660521147237.txt

Comment 154

[Jan Rio Krause]

Attachment: qmstacks_until_1660780738604.txt

Comment 155

[Jan Rio Krause]

Attachment: qmstacks_until_1661126370199.txt

Comment 156

[Jan Rio Krause]

Attachment: qmstacks_until_1663545580609.txt

Comment 157

[Jan Rio Krause]

Attachment: qmstacks_until_1663804747415.txt

Comment 158

[Jan Rio Krause]

Attachment: qmstacks_until_1665014349433.txt

Comment 159

[Jan Rio Krause]

Attachment: qmstacks_until_1665359616902.txt

Comment 160

[Jan Rio Krause]

Attachment: qmstacks_until_1665619116067.txt

Comment 161

[Jan Rio Krause]

Attachment: qmstacks_until_1665964722235.txt

Comment 162

[Jan Rio Krause]

Attachment: qmstacks_until_1666223991830.txt

Comment 163

[Jan Rio Krause]

Attachment: qmstacks_until_1666569586979.txt

Comment 164

[Jan Rio Krause]

Attachment: qmstacks_until_1666828718420.txt

Comment 165

[Jan Rio Krause]

Attachment: qmstacks_until_1667779155893.txt

Comment 166

[Jan Rio Krause]

Attachment: qmstacks_until_1668038343368.txt

Comment 167

[Jan Rio Krause]

Attachment: qmstacks_until_1669593538878.txt

Comment 168

[Jan Rio Krause]

Attachment: qmstacks_until_1669852793579.txt

Comment 169

[Jan Rio Krause]

Attachment: qmstacks_until_1670198336144.txt

Comment 170

[Jan Rio Krause]

Attachment: qmstacks_until_1670457559163.txt

Comment 171

[Jan Rio Krause]

Attachment: qmstacks_until_1670802910353.txt

Comment 172

[Jan Rio Krause]

Attachment: qmstacks_until_1671062387359.txt

Comment 173

[Jan Rio Krause]

Attachment: qmstacks_until_1671407991247.txt

Comment 174

[Jan Rio Krause]

Attachment: qmstacks_until_1672617586254.txt

Comment 175

[Jan Rio Krause]

Attachment: qmstacks_until_1672876797874.txt

Comment 176

[Jan Rio Krause]

Attachment: qmstacks_until_1673222174256.txt

Comment 177

[Jan Rio Krause]

Attachment: qmstacks_until_1673481145790.txt

Comment 178

[Jan Rio Krause]

Attachment: qmstacks_until_1673827167265.txt

Comment 179

[Jan Rio Krause]

Attachment: qmstacks_until_1674086094216.txt

Comment 180

[Jan Rio Krause]

Attachment: qmstacks_until_1674431970403.txt

Comment 181

[Jan Rio Krause]

Attachment: qmstacks_until_1674690976748.txt

Comment 182

[Jan Rio Krause]

Attachment: qmstacks_until_1675036736451.txt

Comment 183

[Jan Rio Krause]

Attachment: qmstacks_until_1675295894792.txt

Comment 184

[Jan Rio Krause]

Attachment: qmstacks_until_1675641537355.txt

Comment 185

[Jan Rio Krause]

Attachment: qmstacks_until_1675900244446.txt

Comment 186

[Jan Rio Krause]

Attachment: qmstacks_until_1676246395291.txt

Comment 187

[Jan Rio Krause]

Attachment: qmstacks_until_1676505475234.txt

Comment 188

[Jan Rio Krause]

Attachment: qmstacks_until_1676851088948.txt

Comment 189

[Jan Rio Krause]

Attachment: qmstacks_until_1677110378988.txt

Comment 190

[Jan Rio Krause]

Attachment: qmstacks_until_1677455990182.txt

Comment 191

[Jan Rio Krause]

Attachment: qmstacks_until_1677715030956.txt

Comment 192

[Jan Rio Krause]

Attachment: qmstacks_until_1678060717439.txt

Comment 193

[Jan Rio Krause]

Attachment: qmstacks_until_1678319945583.txt

Comment 194

[Jan Rio Krause]

Attachment: qmstacks_until_1678665576478.txt

Comment 195

[Jan Rio Krause]

Attachment: qmstacks_until_1678924611670.txt

Comment 196

[Jan Rio Krause]

Attachment: qmstacks_until_1679270347248.txt