Form submission extremely slow on large forms (with form history turned on)

RESOLVED FIXED in mozilla1.9.1b2

Status

()

Toolkit
Form Manager
RESOLVED FIXED
9 years ago
8 years ago

People

(Reporter: Karel Kolman, Assigned: Dolske)

Tracking

({perf, verified1.9.0.9})

Trunk
mozilla1.9.1b2
perf, verified1.9.0.9
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 1 obsolete attachment)

(Reporter)

Description

9 years ago
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5

When submitting a form with many inputs, it takes a long time before the entered values are crunched by form history manager and form submission actually happens.
I have a 120+ inputs form, it takes 10 second before request is sent to the form action target url. I have a not-so-fast Sempron 2400+ system with an old hard disk. I hear heavy disk activity after clicking the Submit button.

Resubmitting the same form is fast. Clearing the form history (through Clear private data) and resubmitting the form takes a long time again.

Reproducible: Always

Steps to Reproduce:
1. Open a page with a form with many inputs in the browser.
2. Clear private data history.
3. Submit the form.
4. Submit the form again.
Actual Results:  
Step 3. takes way too long to finish.
Step 4. finishes quickly.

Expected Results:  
A previously not submitted form with many inputs will take only slightly more time to submit than subsequent submissions of the same form.
(Reporter)

Comment 1

9 years ago
Created attachment 313567 [details]
A form with 100+ inputs.

A form with 100+ inputs (post action to the same file).
It takes 10+ seconds on my system to submit.
The formhistory.sqlite db has 9216 bytes when submitting in a clean profile.
(Reporter)

Comment 2

9 years ago
Created a profile on the same system but located on a much faster sata disk, the initial form submission delay is a bit noticeable but generally much shorter (around 1 sec). 

Comment 3

9 years ago
I met the same problem. 
web form has about 1000 input fields

environment                           resullt
firefox 2.0.0.14 + win xp             ok(1 sec)
firefox 3.0 + winxp                   slow(above 20 secs)
firefox(v2 or v3)+ win2000            very slow(above 30secs). 
Product: Firefox → Toolkit
(Assignee)

Comment 4

9 years ago
Yeah, yikes, something crazy is going on here. Submitting the testcase from comment 1 results in 1748 disk operations for formhistory. :-O A small segment from a typical rwsnoop run on my OS X box looks like:

# rwsnoop -vp 44159
TIMESTR                UID    PID CMD          D   BYTES FILE
...
Sep  6 14:51:45   501  44159 firefox-bin  W      24 formhistory.sqlite-journal
Sep  6 14:51:45   501  44159 firefox-bin  W       1 formhistory.sqlite-journal
Sep  6 14:51:45   501  44159 firefox-bin  W       4 formhistory.sqlite-journal
Sep  6 14:51:45   501  44159 firefox-bin  W    1024 formhistory.sqlite-journal
Sep  6 14:51:45   501  44159 firefox-bin  W       4 formhistory.sqlite-journal
Sep  6 14:51:45   501  44159 firefox-bin  W       4 formhistory.sqlite-journal
Sep  6 14:51:45   501  44159 firefox-bin  W    1024 formhistory.sqlite-journal
Sep  6 14:51:45   501  44159 firefox-bin  W       4 formhistory.sqlite-journal
Sep  6 14:51:45   501  44159 firefox-bin  W       4 formhistory.sqlite-journal
Sep  6 14:51:45   501  44159 firefox-bin  W    1024 formhistory.sqlite
Sep  6 14:51:45   501  44159 firefox-bin  W    1024 formhistory.sqlite
Sep  6 14:51:45   501  44159 firefox-bin  W       4 formhistory.sqlite-journal
Sep  6 14:51:45   501  44159 firefox-bin  W    1024 formhistory.sqlite-journal
Sep  6 14:51:45   501  44159 firefox-bin  W       4 formhistory.sqlite-journal
Sep  6 14:51:45   501  44159 firefox-bin  W       4 formhistory.sqlite-journal
...
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: blocking1.9.1?
OS: Windows XP → All
Hardware: PC → All
Version: unspecified → Trunk
(Assignee)

Comment 5

9 years ago
Looks like we create a transaction and execute a statement for *each* form field that's saved. I'm going to go out on a limb and guess that's not the most efficient way to be doing things. :-)
Keywords: perf
(In reply to comment #5)
> Looks like we create a transaction and execute a statement for *each* form
> field that's saved. I'm going to go out on a limb and guess that's not the most
> efficient way to be doing things. :-)
No, and that'd be why you hit the perf issue!

I'm not sure where this code lives, but this sounds like a prime candidate to use the async storage API too!
(Assignee)

Updated

9 years ago
Assignee: nobody → dolske
Target Milestone: --- → mozilla1.9.1b2
(Assignee)

Comment 7

9 years ago
Created attachment 342003 [details] [diff] [review]
Patch v.1

Fixes the problem by removing the per-entry transaction, and instead using a per-form transaction.

I benchmarked this by measuring the time it takes to execute the for-loop in Notify() [which this patch wraps in a transaction]. Repeated 5 times, deleting formhistory.sqlite in between runs (we short-circuit when the saved value already exists).

In the current code, submitting the attached testcase was measured at 224ms. With this patch, it now takes 8ms. 28x faster, woo! :-)

Looking at the file IO with DTrace, I see a handful of tiny writes to formhistory-sqlite-journal, followed by 8 1K writes to formhistory.sqlite, followed by a few more writes to formhistory-sqlite-journal. Much better than the flood I saw in comment 4.
Attachment #342003 - Flags: review?(sdwilsh)
(Assignee)

Comment 8

9 years ago
Oh, meant to also note: I also measured the perf with removing the per-entry transaction (but not adding the per-form transaction), and didn't see a significant change in the numbers.
(In reply to comment #8)
> Oh, meant to also note: I also measured the perf with removing the per-entry
> transaction (but not adding the per-form transaction), and didn't see a
> significant change in the numbers.
What is more interesting in both cases is the number of fsyncs.
Comment on attachment 342003 [details] [diff] [review]
Patch v.1

>+  transaction.Commit();
> 
>   return NS_OK;
please return the value from transaction.Commit() - it's possible it could fail.

r=sdwilsh with that fixed.
Attachment #342003 - Flags: review?(sdwilsh) → review+
(Assignee)

Comment 11

9 years ago
Created attachment 342161 [details] [diff] [review]
Patch v.2
Attachment #342003 - Attachment is obsolete: true
Attachment #342161 - Flags: review?(gavin.sharp)
Attachment #342161 - Flags: review?(gavin.sharp) → review+
(Assignee)

Updated

9 years ago
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Flags: blocking1.9.1?
Resolution: --- → FIXED
(Assignee)

Updated

9 years ago
Summary: Form submission extremely slow on large forms (with form Form history turned on) → Form submission extremely slow on large forms (with form history turned on)
(Assignee)

Comment 12

8 years ago
This fix was included as part of bug 483096.
Keywords: fixed1.9.0.8
(Assignee)

Comment 13

8 years ago
(On 1.9.0.8, that is.)
Verified through bug 483096 for 1.9.0.8.
Keywords: fixed1.9.0.8 → verified1.9.0.8
You need to log in before you can comment on or make changes to this bug.