454620 - Create branch administration page

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[(not currently active) Ted Mielczarek]

The Pylons app has a branch administration page to administer the branches table (duh):
http://code.google.com/p/socorro/source/browse/trunk/webapp/socorro/templates/branch_maintenance.html
http://code.google.com/p/socorro/source/browse/trunk/webapp/socorro/controllers/admin.py

Not a huge deal, but the contents of this table dictate what shows up on the topcrashes page, so it'd be nice to get that working again so new versions will show up there.

Comment 3

[Les Orchard [:lorchard]]

Attachment: patch to port the branch admin page from legacy python web app

Still need to find the apache config to enable LDAP auth for the admin pages, but posting this patch to add branches for now.

Comment 4

[Les Orchard [:lorchard]]

Attachment: quick patch revision to restrain report table search time period

Oops - quick revision to the patch!  The original webapp searched 1 week worth of reports for missing branches, but I was playing with an interval of 4 years since my laptop has barely any data.

Comment 5

[Austin King [:ozten]]

This is great. 

Please remove use of NOW() - 1 week interval and replace with a literal date. See 
https://bugzilla.mozilla.org/show_bug.cgi?id=465632#c0 for requirements.

Comment 6

[Les Orchard [:lorchard]]

Attachment: tweak to revise date math on reports

Hmm, that seems vaguely familiar to what Lars was talking about at the on-site.  How's this patch look?  Seems to work, though my laptop probably doesn't have the partitioning

Comment 7

[Austin King [:ozten]]

} if (isset($_POST[$name])) {
should be 
}elseif (isset($_POST[$name])) {

in MY_Controller.php
public function getRequestParameters($params_and_defaults)

Comment 8

[Les Orchard [:lorchard]]

Attachment: another correction, this time to getching POST vars

Doh.  So it should be indeed!

Comment 9

[Austin King [:ozten]]

Comment on attachment 349468 [details] [diff] [review]
another correction, this time to getching POST vars

Looks good.

Comment 10

[Austin King [:ozten]]

Once more note per a change in https://bugzilla.mozilla.org/show_bug.cgi?id=465632

reports.date BETWEEN 
should be
reports.date_processed BETWEEN

Comment 11

[Les Orchard [:lorchard]]

trying to circle back to this bug, since I don't think I've checked in the patch but want to close the bug at some point soon.  Have there been any other major DB changes since 12/10?  I'm totally out of synch with things at this point, but it seems like I heard there were some big things moving around.

Comment 12

[Austin King [:ozten]]

Nothing to change the branches table.

The reports.date -> reports.date_processed will happen, but isn't in prod yet.

There are some other chanegs, but I don't think they affect the admin screen.

Comment 13

[Les Orchard [:lorchard]]

Been sitting on this bug awhile now, thinking I probably won't be the one to close it.  Cleaning up my queue, so bowing out of it for now.

Comment 14

[Austin King [:ozten]]

The latest patch attached to this bug has bitrot... the branches table is now a view on the productdims table. This Bug is a good start on the overall Admin page project.

Comment 15

[Frank Griswold [:griswolf] [:fgriswold]]

There are two related tables that may need to be simultaneously maintained:

productdims: Holds product,version,branch info

product_visibility: Holds start and stop dates for materialized views with a foreign key into productdims

description of the two tables is here:
http://code.google.com/p/socorro/wiki/SocorroDatabaseSchema#Aggregate_Reporting

Per discussion I've seen elsewhere, it may be nice to allow authorized users to add to the productdims table without updating product_visibility, or allow start dates in product_visibility that are in the future. product_visibility dates will almost surely need to be individually editable in the face of changing requirements

Comment 16

[Frank Griswold [:griswolf] [:fgriswold]]

You may want to look at http://code.google.com/p/socorro/source/browse/#svn/branches/fg-admin for some code that doesn't all hook together and work. models has some simple sql stuff.

Comment 17

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Attachment: 1st Patch for Bug #454620

Attached is the patch for bug #454620.

Included are fixes for LDAP, as well as the admin screen.

The LDAP authentication link has been moved to the footer, and an Admin link has been placed in the header for logged-in users.

An admin index page and controller have been added. The page for reading/updating/deleting product versions is:
http://rsnyder.khan.mozilla.org/reporter/admin/branch_data_sources

Comment 18

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Attachment: 2nd Patch for Bug #454620

2nd patch for bug #454620.

Per Austin, updating select statement to read `productdims` instead of `branches`, in order to pickup the `release` field.

Comment 19

[Austin King [:ozten]]

(In reply to comment #18)
I think "2nd Patch for Bug #454620" is actually a patch from 456402 Improve user experience

Can you re-upload your patch?

Comment 20

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Attachment: 3rd Patch for Bug #454620

Thanks Austin...

Comment 21

[Austin King [:ozten]]

Comment on attachment 409354 [details] [diff] [review]
3rd Patch for Bug #454620

Nice work on CSS for message box and moving around the login link.

"admin" link from /reporter/admin/branch_data_sources is 404. Should be relative

Page is s l o w. We should analyze the queries and A note below about dates in the report table may help.

branch.php line 24 - we should setup the start/end date in product_visibility to be 90 days
Also this should be in the form as you initiall add the product/version. We'll just pre-fill it with 90 days, but they can tweak the date.

line 82 - due to the way reports table is partitioned... we aren't allowed to use date math in the SQL layer. Pre-calcuate the 1 week offset in PHP
timestamp with time zone '$now' - CAST('1 week' AS INTERVAL) AND                                                                                                               

admin.php line 24 - make call to parent::__construct() the first statement.

Related Work:
We should update http://code.google.com/p/socorro/wiki/TrackNewProductVersion

Comment 22

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Attachment: Patch from Socorro Week

Attached is the patch from working with Austin, Frank and Lars during Socorro week.

Comment 23

[Austin King [:ozten]]

Comment on attachment 409976 [details] [diff] [review]
Patch from Socorro Week

Excellent work! We did a face to face and you have a couple minor notes, but good to go overall. Thanks.

Comment 24

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Committing final changes.

==

Sending        webapp-php/README.txt
Adding         webapp-php/application/controllers/admin.php
Sending        webapp-php/application/libraries/MY_Controller.php
Sending        webapp-php/application/models/branch.php
Adding         webapp-php/application/views/admin
Adding         webapp-php/application/views/admin/branch_data_sources.php
Adding         webapp-php/application/views/admin/index.php
Sending        webapp-php/application/views/common/query_form.php
Sending        webapp-php/application/views/layout.php
Sending        webapp-php/css/screen.css
Adding         webapp-php/js/socorro/admin.js
Sending        webapp-php/modules/auth/libraries/Auth.php
Transmitting file data ...........
Committed revision r1429.