Closed Bug 1224356 Opened 10 years ago Closed 10 years ago

[Compat Data][Importer] Committing data can lead to timeout, 503

Categories

(developer.mozilla.org Graveyard :: General, defect)

All
Other
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jwhitlock, Assigned: jwhitlock)

References

Details

(Keywords: in-triage, Whiteboard: [specification][type:bug])

What did you do? ================ 1. Log in as alpha user, with a cold Heroku cache 2. Go to a page ready for import 3. Click "Commit" What happened? ============== After 30 seconds, Heroku restarts the web process, and a 500 is returned. What should have happened? ========================== The new data should have been committed without error Is there anything else we should know? ====================================== A bug was preventing cache updating for new instances, which was fixed, but is now causing cascading cache invalidation in Heroku. This is designed to be done asynchronously, but celery is running in "eager" mode, which means that the cache is being populated synchronously, which takes about 5 minutes on a cold cache, and less than 5 seconds on a warmed cache. One solution would be to configure asynchronous Celery workers in Heroku, which is the desired production configuration.
Blocks: 996570
Commits pushed to master at https://github.com/mdn/browsercompat https://github.com/mdn/browsercompat/commit/56d372629a251b4042ff6bd0ea1c7b91bdf6c03e bug 1224356 - Add Redis cache and celery backends * Allow setting Redis from the environment as the cache backend. * Allow setting the celery broker and results database * Optionally run integration tests with Redis as the cache, broker, and results database. https://github.com/mdn/browsercompat/commit/3289b847bbf7f009d317c77a7257c00153d62419 bug 1224356 - Poll importer for end of work If the import status indicates that background parsing is in progress, then poll the importer every 1 second to see if parsing is complete, and refresh the page when complete. https://github.com/mdn/browsercompat/commit/b662493f893050e8ef62e080cb36519c6b9fad59 bug 1224356 - Let user know commit has started Change the Commit button and label so that the user knows the commit has started, instead of wondering if they hit the button or if the system is frozen.
This should avoid the web request timeout now, but the interface still needs work: - load_json should not be setting the "on submit" function for the commit button every round - When the commit is done, the text should change from "Committing data to the API" to "Loading new IDs from API" - Polling should start, reloading the page when the new ID response has returned.
Assignee: nobody → jwhitlock
Fixed with wrong bug number. Here's what got pasted to the other bug: Commits pushed to master at https://github.com/mdn/browsercompat https://github.com/mdn/browsercompat/commit/646b9541206eab5fc1420106942c7d15b079e5b6 fix bug 1124356 - Improve UI for async commit When the server is in async mode, change text after a successful sync, and start polling the server to reload with new data IDs. Also, only set the handler for the commit button during page setup. https://github.com/mdn/browsercompat/commit/16b0dc570590146867f1c5ea35298b81102e5cf6 Merge pull request #87 from mdn/async_commit_1224356 fix bug 1124356 - Improve UI for async commit
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Product: developer.mozilla.org → developer.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.