684256 - need a simple tool to import JSON files into AUS database

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P5)

Description

[[e-mail] (:bhearsum)]

We already have generate-json.py, which knows how to create JSON files based on snippet stores or candidates directories. In order to fix bug 684243, we need a tool that can pull the name, version, and product out of these, and stuff them into the database. It needs to support importing into the dated version, as well as a -Nightly or -Latest version.

We might also need to enhance it to flush out some of the release-wide details, such as extv, appv, and fileUrls.

Comment 1

[[e-mail] (:bhearsum)]

Attachment: enhance generate-json.py to support nightly snippet dirs & db insertion

This is a bit of a quick hack, but seems to work. The nightly snippet parser generates fully complete JSON files for a nightly build. I also added the option to insert parsed data into a db, which works with all types of directories. I'm not sure the 'Linux_x86-64-gcc4' build target is valid, but we have it in mozilla-central's snippet directory. We can remove it later if need be.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

'hg grep --all -d -f Linux_x86-64-gcc4' says
mozilla2/config.py:109:-:Wed Jun 18 15:30:43 2008 -0400:BRANCHES['mozilla-central']['platforms']['linux64']['update_platform'] = 'Linux_x86-64-gcc4'
mozilla2/config.py:106:+:Wed Jun 18 08:18:37 2008 -0400:BRANCHES['mozilla-central']['platforms']['linux64']['update_platform'] = 'Linux_x86-64-gcc4'

So I'd say it was a short lived mistake-o that need not live on.

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 559282 [details] [diff] [review]
enhance generate-json.py to support nightly snippet dirs & db insertion

>diff --git a/generate-json.py b/generate-json.py
>-    ('Linux_x86_64-gcc3',): {
>+    ('Linux_x86_64-gcc3', 'Linux_x86-64-gcc4'): {

We did 3 or 4 linux64 nightlies while the bogus target was set, but I can't find any pings from them within the last week, so I went ahead and removed the symlink on AUS. We can skip this change.

>+    relData = {"name": options.name, "data_version": DATA_VERSION, "platforms": {},
>+               "hash_func": options.hash_func}

Should be "hashFunction" instead of "hash_func". Dunno why I used data_version instead of dataVersion, but we can fixed that some other time.

Otherwise, lgtm.

Comment 4

[[e-mail] (:bhearsum)]

Attachment: patch as landed

Comment 5

[[e-mail] (:bhearsum)]

Attachment: delete releases before adding them

Tiny follow-up to make this work with -nightly releases, which get overwritten every night.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 559448 [details] [diff] [review]
delete releases before adding them

Ok for now.