575629 - need research in the matter of retiring the 'frames' table

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[K Lars Lohn [:lars] [:klohn]]

The frames table appears to be deprecated, but it hangs around using up storage and processing resources.  According to research done by Ryan, there is mechanism is the UI to access this table, but it is not actually hooked up the UI.  It is my recollection that this was disabled for performance reasons about the same time that I introduced overall crash signatures nearly two years ago.

While the UI doesn't support it, it is possible that some devs still may use saved queries that do exercise the frames table.  That was the case up to about three months after the feature was withdrawn.

Does anyone still use the feature through save queries?  How do we tell?  Ryan suggested that we should grep the apache log.  I'd like some one to do that.  If we can eliminate the frames table, not only can we save some resources now, but we can speed along both creation and eventual performance of the 1.8 HBase to PostgreSQL daemon.

My original conversation with Ryan on Monday, June 28:


(04:53:34 PM) lars: open question:  does any thing in the Socorro UI code do anything at all with the "frames" table?  Does anyone know of any where at all that the 'frames' table is actually used?
(04:56:33 PM) ryansnyder: lars: It's being used in 1 place
(04:56:47 PM) ryansnyder: In the process of building an advanced search query
(04:57:27 PM) ryansnyder: web-app/application/models/common.php:402
(04:58:29 PM) lars: but does the UI actually have any controls that use it?  I'm looking at the UI from the user's perspective and I see nothing that allows me to do anything with data from that table.
(04:58:45 PM) ryansnyder: looking...
(04:59:33 PM) ryansnyder: Trying to figure out if that is used via the stack signature field in advanced search...
(05:01:52 PM) ryansnyder: In the query form, that input field for stack signature is commented out
(05:02:24 PM) chowse|mtg left the room (quit: Quit: chowse|mtg).
(05:03:04 PM) lars: so there is mechanism for using that table, but it isn't hooked up to anything, ja?
(05:03:46 PM) ryansnyder: That's what it seems...  but it doesn't really look fully baked even if  
(05:03:54 PM) ryansnyder: we uncommented that field in the query form
(05:04:21 PM) lars: it's a feature that was removed for performance reasons long ago and never restored nor removed entirely
(05:05:25 PM) lars: would it be possible that someone could manually enter a query URL (say something saved from 2 years ago) and actually manage to access that table?
(05:06:04 PM) lars: I seem to recall that when the feature was removed, some dev were actually doing that
(05:06:45 PM) ryansnyder: perhaps - using &query_search=stack
(05:06:50 PM) lars: I wondering out loud if they've finally quit doing it - guess we could tell if we retired the table and their queries quit
(05:07:03 PM) ryansnyder: lol, that's always the easiest solution
(05:07:42 PM) lars: ok, that's something to think about for the next meeting.
(05:08:54 PM) ryansnyder: Can we grep the apache logs to see if that search query is being used? 
(05:09:52 PM) lars: I'm casting about looking for efficiencies that we may or may not need in the next couple weeks.

Comment 1

[(not currently active) Ted Mielczarek]

bug 480503 is on file requesting that we add back "search in the top 10 frames". I think given the state of things it's probably fine to make that a 1.8 or later feature as a map/reduce job or however things will work in the glorious future.

Comment 2

[K Lars Lohn [:lars] [:klohn]]

long done