1639545 - Also run IndexedDB mochitests in private browsing mode

Status: RESOLVED FIXED

(Core :: Storage: IndexedDB, task, P3)

Comment 1

[Simon Giesecke [:sg] [he/him]]

Attachment: Bug 1639545 - Pull out the implementation of IndexedDBCipherKeyManager members from indexedDB/ActorsParent.cpp into it's own cpp file.r=#dom-storage-reviewers

Comment 2

[Simon Giesecke [:sg] [he/him]]

I tried to use the [include:file] special section to achieve running the tests with and without the pref browser.privatebrowsing.autostart=true. I then added dupe-manifest properties to avoid errors complaining about duplicate test files. But it still doesn't work, running ./mach mochitest --headless dom/indexedDB/test/mochitest.private.ini yields:

The mochitest command could not find any mochitests under the following
test path(s):

dom/indexedDB/test/mochitest.private.ini

Please check spelling and make sure there are mochitests living there.

Andrew, can you give me a hint on what I am doing wrong?

Comment 3

[Andrew Halberstadt [:ahal]]

I think it could be working, but due to a quirk of the test resolver code, the test's direct manifest is now mochitest.common.ini rather than mochitest.private.ini. What happens if you pass in the directory instead? Does it run both?

The TestResolver class should definitely be able to find tests via an ancestor-manifest though. I'll file a bug.

Comment 4

[Simon Giesecke [:sg] [he/him]]

I think they are both run when I pass the directory, but it's hard to tell from the output. What is particularly confusing that way is that on a test failure, it is not mentioned which manifest was used when the test failed, e.g :

dom/indexedDB/test/test_blob_file_backed.html
  FAIL indexedDB error, 'InvalidStateError'
    SimpleTest.ok@SimpleTest/SimpleTest.js:299:16
    errorHandler@dom/indexedDB/test/helpers.js:198:5
    EventHandlerNonNull*testSteps@dom/indexedDB/test/unit/test_blob_file_backed.js:42:3
    grabEventAndContinueHandler@dom/indexedDB/test/helpers.js:184:17
    EventHandlerNonNull*testSteps@dom/indexedDB/test/unit/test_blob_file_backed.js:34:3
    testSteps/<@dom/indexedDB/test/unit/test_blob_file_backed.js:23:21
    Async*testSteps@dom/indexedDB/test/unit/test_blob_file_backed.js:20:17
    testHarnessSteps@dom/indexedDB/test/helpers.js:152:19
    nextTestHarnessStep@dom/indexedDB/test/helpers.js:48:26

I can also run a single test file, and then both instances seem to be run, but it says:

 0:01.94 INFO Running manifest: dom/indexedDB/test/mochitest.private.ini:dom/indexedDB/test/mochitest.common.ini

both times, but the preferences listed below that are different.

Is it possible to output a test summary (XML or so) that gives an overview of which tests were run with which result, in a way that uniquely identifies each test instance?

Comment 5

[Andrew Halberstadt [:ahal]]

Yeah, we need test ids to contain the entire manifest chain for them to be truly unique. Solving this properly is going to be a major undertaking spanning all our test harnesses, mozbase and the build system. There's probably a hacky way to solve this in the short term, but I'm not sure what that is at the moment. I filed bug 1641029.

The second issue looks like a bug in mochitest. Would you mind filing a bug with STR?

I also filed bug 1641004 for the ./mach mochitest path/to/ancestor-manifest.ini issue.

Comment 6

[Simon Giesecke [:sg] [he/him]]

(In reply to Andrew Halberstadt [:ahal] from comment #5)

Yeah, we need test ids to contain the entire manifest chain for them to be truly unique. Solving this properly is going to be a major undertaking spanning all our test harnesses, mozbase and the build system. There's probably a hacky way to solve this in the short term, but I'm not sure what that is at the moment. I filed bug 1641029.

Thanks. I can imagine this is a complex issue, but it would really help a lot. It would also be one puzzle piece in getting able to get a better overview of intermittent failures. In the end, it would be great to have some table view to have a large table view which shows all test cases and which pass/fail/timeout on which platform, either aggregated or for a single treeherder execution. Extracting this information from individually filed intermittent bugs takes a lot of time and is error-prone.

The second issue looks like a bug in mochitest. Would you mind filing a bug with STR?

I am sorry, I was mistaken here. I checked it again and actually the two runs show the different manifests.

I also filed bug 1641004 for the ./mach mochitest path/to/ancestor-manifest.ini issue.

Thanks!

Comment 7

[Jan Varga [:janv]]

Attachment: Bug 1639545 - Also run IndexedDB mochitests in private browsing mode.r=#dom-storage-reviewers

Depends on D86704

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6814e78b09e7
Also run IndexedDB mochitests in private browsing mode.r=dom-storage-reviewers,janv
https://hg.mozilla.org/integration/autoland/rev/26d9a018ab8c
Pull out the implementation of IndexedDBCipherKeyManager members from indexedDB/ActorsParent.cpp into it's own cpp file.r=janv

Comment 9

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a10392d3ad2d
Backed out 5 changesets (bug 1639545, bug 1639548, bug 1818214) for multiple mochitest/marionette dom/indexedDB/test/... related test failures. CLOSED TREE

Comment 10

[Marian-Vasile Laza]

Backed out 5 changesets (Bug 1639545, Bug 1639548, Bug 1818214) for multiple mochitest dom/indexedDB/test/... related test failures.
Backout link
Push with failures <--> 2
Failure Log
Fixed by commit filter

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/91c76cf4f745
Pull out the implementation of IndexedDBCipherKeyManager members from indexedDB/ActorsParent.cpp into it's own cpp file.r=janv

Comment 12

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/91c76cf4f745

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a021d15b86fe
Also run IndexedDB mochitests in private browsing mode.r=dom-storage-reviewers,janv

Comment 14

[Narcis Beleuzu [:NarcisB]]

Backed out for multiple mochitest failures on dom/indexedDB/test/*

Backout link: https://hg.mozilla.org/integration/autoland/rev/df74dc8c210293aa63c5fa4d5c516c939205242b
Log dom/indexedDB/test/test_storage_manager_estimate.html
Log dom/indexedDB/test/test_file_os_delete.html
Log dom/indexedDB/test/test_blob_worker_xhr_post.html

Comment 15

[Harveer Singh]

Looking into the test failures.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1d1b57f7a165
Also run IndexedDB mochitests in private browsing mode.r=dom-storage-reviewers,janv

Comment 17

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/1d1b57f7a165