1367539 - Seeing IndexedDB DOMError on DevEdition viewing http://devdocs.io/offline

Status: NEW

(Core :: Storage: IndexedDB, defect, P5)

Description

[Bryan Clark (DevTools PM) [:clarkbw]]

STR:

1. Load http://devdocs.io/offline
2. See error in console

This issue doesn't seem to exist in other browsers and I can't reproduce this error in Nightly, only in DE so far.

https://github.com/Thibaut/devdocs/issues/584



Is there a change to IDB that we need to uplift?



Here's the trace, though its not particularly useful.


IDB error DOMError { name: "UnknownError", message: "The operation failed for reasons un…" }  application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:1337:20
	n/t[e] http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:1337:20
	app.DB</e.prototype.fail http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3138:76
	app.DB</e.prototype.onOpenError http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3134:1
	t/< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3085:8
	(Async: EventHandlerNonNull)
	app.DB</e.prototype.db http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3101:35
	app.DB</e.prototype.versions http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3256:50
	app.collections.Docs</o.prototype.getInstallStatuses http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3926:1
	app.views.OfflinePage</o.prototype.render http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4607:1
	app.views.OfflinePage</o.prototype.onRoute http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4630:1
	app.views.Content</o.prototype.afterRoute http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4454:1
	t/< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4337:8
	trigger http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2460:69
	app.Router</t.prototype.triggerRoute http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3406:32
	app.Router</t.prototype.offline http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3431:1
	e</t.prototype.middleware/</< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2561:53
	o http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2520:29
	e</t.prototype.middleware/</< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2561:63
	o http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2520:29
	e</t.prototype.middleware/</< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2561:63
	o http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2520:29
	app.Router</t.prototype.before http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3409:72
	e</t.prototype.middleware/</< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2561:53
	o http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2520:29
	page.dispatch http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2518:16
	page.replace http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2514:60
	page.start http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2504:1
	app.Router</t.prototype.start http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3402:1
	start http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2882:24
	app.collections.Docs</o.prototype.load/c</< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3907:83
	app.models.Doc</n.prototype._loadFromCache/e</< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4062:13
	o http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:1520:8
	(Async: setTimeout handler)
	t/< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:1737:46
	app.models.Doc</n.prototype._loadFromCache http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4064:10
	app.models.Doc</n.prototype.load http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4047:45
	app.collections.Docs</o.prototype.load/c</< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3907:23
	app.models.Doc</n.prototype._loadFromCache/e</< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4062:13
	o http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:1520:8
	(Async: setTimeout handler)
	t/< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:1737:46
	app.models.Doc</n.prototype._loadFromCache http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4064:10
	app.models.Doc</n.prototype.load http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4047:45
	app.collections.Docs</o.prototype.load/c</< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3907:23
	app.models.Doc</n.prototype._loadFromCache/e</< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4062:13
	o http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:1520:8
	(Async: setTimeout handler)
	t/< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:1737:46
	app.models.Doc</n.prototype._loadFromCache http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4064:10
	app.models.Doc</n.prototype.load http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4047:45
	app.collections.Docs</o.prototype.load/c</< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3907:23
	app.models.Doc</n.prototype._loadFromCache/e</< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4062:13
	o http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:1520:8
	(Async: setTimeout handler)
	t/< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:1737:46
	app.models.Doc</n.prototype._loadFromCache http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4064:10
	app.models.Doc</n.prototype.load http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:4047:45
	app.collections.Docs</o.prototype.load/c</< http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3907:23
	app.collections.Docs</o.prototype.load http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:3914:60
	bootAll http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2872:21
	init http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:2825:30
	(Async: EventListener.handleEvent)
	<anonymous> http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:6720:4
	<anonymous> http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:6716:2

Comment 1

[Hsin-Yi Tsai (she/her) [:hsinyi]]

I was trying to find which bug fixes the issue by mozregression --find-fix but in vain. I didn't encounter this issue on Nightly version from 55 back to 52. However, I did see this in released 53.

Is there difference in nightly and release versions? Is there a change to IDB that we need to uplift?

(I tried to find a possible cause from https://public.etherpad-mozilla.org/p/quota-manager-schema-change-log, but didn't get a confident idea...)

Comment 2

[Andrew Sutherland [:asuth] (he/him)]

My gut reaction at this point is "profile from the future" (most recently witnessed in IDB as bug 1357428), but dev edition gets its own profile, so that shouldn't happen.  So my guess then is "some file under storage/ that shouldn't be there and freaks Quota Manager or IDB out", for example bug 1358571 where if you copy a profile from OS X to Linux or Windows, Quota Manager can get angry.

Unfortunately, the UNKNOWN_FILE_WARNING reporting enhancement only happen in Fx55 as part of bug 1339081.  There's only an NS_WARNING and generic error returned right now.  So :clarkbw, unless you have a debug build, the answer is in your profile, specifically the output of running something like "find PROFILE/storage -maxdepth 4"[1] and seeing what's in there that shouldn't be there.  The contents of that directory of course contain private information (visited origins, names of their databases), so if a skim of it yourself doesn't reveal a smoking gun, then you'd probably want to send the list privately to someone like me if analysis is desired.  I think it would be helpful to have more data points about what's breaking people's QM/IDB, because it's obviously the kind of massively frustrating failure with no obvious solution that can drive someone to a different browser.

If you just want things to work again, go to about:support and refresh your profile.  This does not copy over the contents of anything persisted by Quota Manager, so the broken-ness won't follow you.

1: The maxdepth avoids listing the contents of the DOM cache morgue directory where all the bodies are stored, or the IDB files directories.

Comment 3

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Andrew Sutherland [:asuth] from comment #2)
> There's only an NS_WARNING and generic error returned right now.

By "right now" I mean, in Fx 54 and Fx 53.

Comment 4

[Bryan Clark (DevTools PM) [:clarkbw]]

For the devdocs case I'm seeing this:

./storage/default/http+++devdocs.io
./storage/default/http+++devdocs.io/.metadata
./storage/default/http+++devdocs.io/.metadata-v2
./storage/default/http+++devdocs.io/idb
./storage/default/http+++devdocs.io/idb/3164636493dsoc.files
./storage/default/http+++devdocs.io/idb/3164636493dsoc.sqlite

Which doesn't say much to me.  The rest of the list doesn't appear to have a smoking gun either so I'll send it to you for an extra set of eyes.  Let me know if there's anything I can do within the sqlite files to find more.  Thanks!

Comment 5

[Andrew Sutherland [:asuth] (he/him)]

Thank you for providing the directory listing.  I didn't see anything approaching a red flag, which means we'll want to depend on things getting logged to the browser console.  I think the console service still has a circular buffer, so you'd want to do something like:

- Restart / or freshly start the browser.
- Bring up the browser console and look for suspicious errors.  If all of QuotaManager is broken, we'll expect to see an error here when QuotaManager initializes and gets upset.  (Unfortunately there are some things that are even now still an NS_WARNING that doesn't go to the browser console and only logged in DEBUG builds, so this isn't perfect.  I'll provide a patch for the storage.sqlite schema version one later today.)
- Bring up the devdocs site and look for suspicious errors.  We would expect errors here if the problem is specific to the contents of the devdocs IDB database.
- Bring up some other IDB using site, maybe one like https://bl.ocks.org/nolanlawson/8a2ead46a184c9fae231 that should ideally obviously fail or not, and see if it thinks IDB is working okay or not and if any other browser console errors show up.  This is really just an additional test of whether all of QM/IDB is broken.

If it seems like the problem is localized to the one site, perhaps zip up the contents of the ./storage/default/http+++devdocs.io directory and send it over.

Comment 6

[Bryan Clark (DevTools PM) [:clarkbw]]

Here are the log errors from loading the page:

The Application Cache API (AppCache) is deprecated and will be removed at a future date.  Please consider using ServiceWorker for offline support.  devdocs.io
Error opening input stream (invalid filename?): moz-extension://843f7200-f3d9-0d4a-b881-afe41f356a25/js/content.bundle.js  ExtensionContent.jsm:276
Error opening input stream (invalid filename?): moz-extension://843f7200-f3d9-0d4a-b881-afe41f356a25/js/pagewrap.bundle.js  ExtensionContent.jsm:276
Offline cache doesn't need to update, URL=http://devdocs.io/manifest.appcache
Error opening input stream (invalid filename?): moz-extension://843f7200-f3d9-0d4a-b881-afe41f356a25/js/content.bundle.js  ExtensionContent.jsm:276
Error opening input stream (invalid filename?): moz-extension://843f7200-f3d9-0d4a-b881-afe41f356a25/js/pagewrap.bundle.js  ExtensionContent.jsm:276
IndexedDB UnknownErr: ActorsParent.cpp:596  (unknown)
IDB error <unavailable>  application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js:1337
The Application Cache API (AppCache) is deprecated and will be removed at a future date.  Please consider using ServiceWorker for offline support.  blank
Offline cache doesn't need to update, URL=http://devdocs.io/manifest.appcache


This error looks interesting: IndexedDB UnknownErr: ActorsParent.cpp:596  (unknown) 

Right below that is linked to this page: view-source:http://devdocs.io/assets/application-bd7e7c1225fda8972a4f1861e84e4ad6574c168ab34852221dc26ba17b39a754.js which isn't intersting.  It highlights a section of code like this: "n && n(e, r), o && Function.prototype.apply.call(o, i, t);"

Comment 7

[Bryan Clark (DevTools PM) [:clarkbw]]

And then the blocks page provides these errors:

An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can remove its sandboxing.  8a2ead46a184c9fae231
IndexedDB UnknownErr: ActorsParent.cpp:596  (unknown)
UnknownError  index.js:21:14

Comment 8

[Andrew Sutherland [:asuth] (he/him)]

Unfortunately, the IDB error is just the one it gives when normalizing error codes.  Any word on browser console messages when the browser is first started and/or whether other origins' IDB's work?  (If you can use devtools' Storage tab to inspect IDB, that also counts as operating.)  Assuming IDB seems to work, zipping up the origin and sending it to me seems like the thing to do.  Please also include PROFILE/storage.sqlite (which does not include any private information, just the QM schema version).

Comment 9

[Bryan Clark (DevTools PM) [:clarkbw]]

On a fresh restart with the --jsconsole I see this as the browser starts up:

IndexedDB UnknownErr: ActorsParent.cpp:596  (unknown)
undefined  Promise-backend.js:925
1496352089557	addons.xpi-utils	WARN	Synchronous load of XPI database due to getAddonsByType([theme, webextension-theme])
OpenGL compositor Initialized Succesfully.
Version: 2.1 INTEL-10.25.13
Vendor: Intel Inc.
Renderer: Intel(R) HD Graphics 530
FBO Texture Target: TEXTURE_2D
IndexedDB UnknownErr: ActorsParent.cpp:596  (unknown)
UnknownError  indexed-db.js:58:9
undefined  Promise-backend.js:925
UnknownError  indexed-db.js:58:9
OpenGL compositor Initialized Succesfully.
Version: 2.1 INTEL-10.25.13
Vendor: Intel Inc.
Renderer: Intel(R) HD Graphics 530
FBO Texture Target: TEXTURE_2DIndexedDB UnknownErr: ActorsParent.cpp:596  (unknown)
PushService:stateChangeProcessEnqueue: Error transitioning state UnknownError  PushService.jsm:160
UnknownError  IndexedDBHelper.jsm:71:12
IndexedDB UnknownErr: ActorsParent.cpp:596  (unknown)
UnknownError  aboutHome.js:131:20

Comment 10

[Andrew Sutherland [:asuth] (he/him)]

That suggests that QuotaManager is unhappy and this is a "profile from the future" case.

You can confirm by running: sqlite3 PROFILE/storage.sqlite "PRAGMA user_version;"

If the result is 131072 (== 2 << 16) then it means the profile has been used in Firefox 55.  Or more specifically, a build with bug 1339081 landed in it.)  The general problem is captured by bug 1246615, with bug 1357428 (and others) covering the IDB-specific manifestation of the problem.

If the result is 65536 (== 1 << 16) then something more interesting is happening.


The big concern here is if dev edition's separate profile logic has somehow regressed with the removal of Aurora.  The happiest answer about what happened here is that the profile was used under Fx55/nightly intentionally/explicitly, rather than the profile affinity stuff having broken.

Comment 11

[Bryan Clark (DevTools PM) [:clarkbw]]

$ sqlite3 storage.sqlite "PRAGMA user_version;"
131072

I don't think I've ever run Nightly against my DevEdition profile; this is even a new laptop.  However I can't be certain of that.  I guess I'll need to reproduce this some how.

Comment 12

[Jens Stutte [:jstutte]]

Bulk-downgrade of unassigned, untouched DOM/Storage bug's priority.

If you have reason to believe, this is wrong, please write a comment and ni :jstutte.