58428 - Sherlock files aren't auto-updating

Status: VERIFIED FIXED

(SeaMonkey :: Search, defect, P3)

Description

[Claudius Gayle]

***Overview Description:
  Damn! i'm hoping that there is something wrong with my testcase but
i can't see what, so I'm forced to conclude that this feature may not
be working correctly.

I created a page at the above url(currently netscape internal while I
work out the kinks in the testcase) that will add a sherlock file for
the search engine 'test' to the 'Web' category of your search sidebar.

The engine is set to update every 1 days. The only thing kinky about
the testcase is that the location of the updated file is different
than that of the original source, it is at ../extradir/test.src. The
name attribute of the sherlock file (not the file itself) is also
different - it is set to 'modified' so you can see the change.
These concessions were made so that someone could test the feature
w/o needing ocntrol over a webserver. I don't believe I've broken any
'rules'.

***Steps to Reproduce:
1) Go to the above url and click the link and OK in the dialog.
2) Perform a search using the new 'test' engine.
3) Quit the browser (is this necessary?). Change your system time to
be two days into the future.
4) Launch the browser. (run another search?)

***Actual Results:

Nothing, i'm still stuck with the old, non-updated sherlock file.
When should the change occur? How long should it take?

***Expected Results:

Like a thief in the night test.src should be overwritten with the
newly available version from the specified directory.

***Build Date & Platform Bug Found:

I've been testing with a 2000102509 Mac Branch build. I have no
reason to believe anything would have changed since then. Or that
this wouldn't be cross-platform (but I haven't checked).

***Additional Information:

The involved files (relative to the url above) are:
../search.html
../test.src
../test.gif
../extradir/test.src <--modified version of installed test.src

Comment 1

[Daniel Veditz [:dveditz]]

I have to assume it's far too late to fix this, but I have no idea how bad the
fall-out is from this bug. If there's a small safe fix and this is really
important please talk to the PDT directly Monday morning as that might be the
last chance of getting something into the branch.

Comment 2

[johng]

It is probably too late to fix, but this is a *very serious* problem.  It means
that any time a search engine (including Netscape Search) makes a server side
change, the search funtionality will break.  It is kind of a time bomb that will
cause Search to break one day in the near future.

Changing rtm- to rtm need info to get it on the radar for Robert Churchill and
others.  Also cc'ing folks on the Search team because this directly effects
their plans.

Claudius, work directly with Robert on this.  This requires immediate and urgent
response. What platforms did you test this on?  This is listed as Mac - what is
the deal on Windows?

Comment 3

[Robert John Churchill]

Claudius, instead of your point-to-a-different-file approach, please try this:

o create a valid Sherlock file (which will search some engine) with an update
URL that points to where you'll have it stored remotely on your web server and
an update day count of let's say "2"
o install the same file into your Search Plugins folder
o run the browser, do a search with that engine
o wait 60 seconds (i.e. let one full minute pass)
o quit browser, set your computers date ahead "3" days
o modify the Sherlock file up on the server (changing the title? sure!) and save
o run browser, wait 60 seconds (yep, let one full minute pass again)
o check file, the new one should have been installed

Works for me.

(Since the auto-update process is rather "invisible" by design, its trickier to
determine what's going on.)

Comment 4

[Robert John Churchill]

Attachment: Claudius, use this valid Sherlock file for your testing

Comment 5

[Robert John Churchill]

Attachment: Diff for edge-case of first-time validation marking

Comment 6

[Robert John Churchill]

OK, a comedy of errors:

o The file Claudius was using was slightly invalid; enough so to prevent updating 
(Claudius, the "</search>" section had to be moved up, before the <browser> 
section)

o Claudius's file just pretends to be a "Google" search file (I told him that 
should be OK), but as it turns out, it can cause the browser to get confused if a 
Google file is already there. That was happening, and Google's search file would 
get updated instead of Claudius's!

o The search file up on the server doesn't change its Last-Mod date, it just uses 
a level of indirection to point to another file to be used; the *first* time the 
browser would see a new search engine being used, it would get the Last-Mod date 
from the HTTP server and, having nothing to compare it against (since it had 
never done the stat before), would just mark the engine as valid.  Since 
Claudius's test case never changes its Last-Mod date, the file would never be 
detected as having changed.

Claudius, here's a revised list of steps for testing (after the patch I've 
attached to this bug makes its way into the tree)

o install the valid search file I've attached onto this bug into your "Search 
Plugins" folder INSTEAD of the search file you've been using
o delete the "localstore.rdf" file from the profile directory you've been using 
(as it'll have stale info in it regarding search stats)
o run Mozilla
o select the "Test" search engine and search for something, say "LDAP"
o set your computer's clock ahead a few days/years/millenia [i.e. prevent that 
actual "real user time" has passed]
o search for something else, say "RDF" [yes, search for something else, otherwise 
the previous search will just come from the cache and no network operations will 
occur]
o wait a minute or so (i.e. give the search update timer code a chance to fire)
o check the "Test" search file, it will have been updated to the search file you 
point to thru re-direction.

Anyway, after all that... the patch (attached in a diff) is a fix for the "first 
time" blessing issue. It would be good to take for the situation where a browser 
user ends up NEVER having using an engine, Netcenter changes a file, and only 
AFTER the change does a user try and use the engine, that the Last-Mod difference 
will be detected.

Now, the "PDT" might ask, is this function called from various places? The answer 
is "yes", however, the variable "dirtyFlag" is always "nsnull" EXCEPT when its 
called from the code which is trying to determine if an update needs to occur... 
so no other code-paths are affected.

Also, if one were to look at the function, you'd see that the exact same code is 
executing in the case if we already had the "Last-Mod" date...  the code change 
is simply to do the same thing if its the very first time we stat the file.

Comment 7

[Robert John Churchill]

<Rats, forgot to mention this as an additional QA testing step> Claudius, as an 
additional testing step, when you add in the Sherlock file from this bug, also 
remove the "google.src" file to prevent the conflict mentioned above.

Comment 8

[Chris Waterson]

gagan and I just talked with rjc about this bug. sr=waterson

Comment 9

[Gagan]

r=gagan

Comment 10

[johng]

PDT, please get in if possible.  Once again, if we don't fix this, search will
eventually break as the server side makes changes occassionally
(as all search engines do, typically once or twice a year, sometimes more).

Good job gang for the fast turn around on this urgent bug.

Comment 11

[Robert John Churchill]

Fixed on trunk.

Comment 12

[Jim Roskind]

Per discussion at PDT, and in consideration of low risk, moving to candidate limbo.
Since this is already in the trunk, please report any/all regressions to this
bug asap.
Anticipate a double plus RSN if the branch gods shine favor upon us...

Comment 13

[selmer (gone)]

rtm++, please checkin ASAP so we can build today.

Comment 14

[Robert John Churchill]

Fixed on branch now also.

Comment 15

[Claudius Gayle]

VERIFIED Fixed on the Branch with 2000110609 builds. I've added the
vtrunk keyword, although this bug will be the first one i look at
when i start verifying trunk fixes.

Comment 16

[Claudius Gayle]

search update repro info

Comment 17

[sairuh (rarely reading bugmail)]

mass-verifying claudius' Fixed bugs which haven't changed since 2001.12.31.

if you think this particular bug is not fixed, please make sure of the following
before reopening:

a. retest with a *recent* trunk build.
b. query bugzilla to see if there's an existing, open bug (new, reopened,
assigned) that covers your issue.
c. if this does need to be reopened, make sure there are specific steps to
reproduce (unless already provided and up-to-date).

thanks!

[set your search string in mail to "AmbassadorKoshNaranek" to filter out these
messages.]