677690 - Need FTP access for one folder at getfirebug.com

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Sebastian Zartner [:sebo]]

I am trying to read XPI files for our Swarms page. Because reading the data from the XPIs takes some time, I want to cache the data somewhere as files.
Normally we're just having access to getfirebug.com via a SVN repository, which is synchronized with the online page. Though because we need to be able to change the data created by the script manually, but it bypasses the repository, we're not having access to it.

So what is needed?
An FTP account, which has access to one specific folder, which lies outside of the with Subversion synchronized root.
Alternatively the data could be stored in a database, but we also don't have success to that.

Comment 1

[Dave Camp (:dcamp)]

Can we get this done for the firebug guys please?

Comment 2

[Corey Shields [:cshields]]

I'm not quite sure what is the end result needed here but we don't do ftp read access to something like this.

Is there a way to speed the XPI up?

Comment 3

[Sebastian Zartner [:sebo]]

Thanks Dave for pushing this.

> I'm not quite sure what is the end result needed here but we don't do ftp read 
> access to something like this.
Ok, little more explanation: The script reads the XPIs, extracts the install.rdf, reads the data from it and then stores that data as separate XML files in a specific folder. The next time you're accessing the script it checks for the XML file and, if it exists, reads the data from there instead of having to extract it from the XPI again AND - and that is the more important part here - we want to add Firebug version compatibility information, because most of the extensions don't have that info in the XPI.

> Is there a way to speed the XPI up?
No, that's why I am storing the data separately. The second reason for saving the data separately is to avoid influencing download statistics. I bet you're also not reading the XPIs everytime someone is at an AMO site, do you?

As I said I also wouldn't have a problem to store that data in a database. Then of course we would need an account for that database.

Comment 4

[Corey Shields [:cshields]]

(In reply to Sebastian Zartner from comment #3)
> As I said I also wouldn't have a problem to store that data in a database.
> Then of course we would need an account for that database.

Let's do a database then..  Are you ready for a DB now or do you need some time to work it into the app?

Comment 5

[Sebastian Zartner [:sebo]]

My "app" is just consisting of one file, so I will need a bit to replace the XML access by a DB access. Though you can already set up the DB. What do you need from me?

Comment 6

[Sebastian Zartner [:sebo]]

Would it be possible to set up a MySQL database for this purpose?

Comment 7

[Corey Shields [:cshields]]

Hi Sebastian,

Brian is almost done prepping getfirebug.com in the new cluster, I'll have him setup an extra db for this and send you the credentials.  This will be available once the site turns over per bug 684704.

Comment 8

[Corey Shields [:cshields]]

Also, is there processing that will need to be done on a regular basis to populate this DB (a script we need to cron somewhere), or will your app take care of the xpi processing?

thanks

Comment 9

[Sebastian Zartner [:sebo]]

That are great news.
The table inside the DB will be populated automatically by my script.
Currently you just have to add the URL to an XPI to that script and the rest is done automatically.
As soon as the DB is set up, the URL will be put into the DB instead and the script will save the data from the XPIs in there. I guess that's similar to what you're doing on AMO. Just instead of reading the data from a URL you let the people upload their extensions.

One question related to the compatibility checking. I just saw lately, that the requires property was removed from the install manifests (https://developer.mozilla.org/en/install.rdf#requires). What was the reason for that? Is there another way to achieve that now?

Comment 10

[Brian Hourigan [:digi]]

Sebastian,

We have migrated blog.getfirebug/getfirebug to our phx1 staging and development environment. Please test the site at http://blog.getfirebug.allizom.org and http://getfirebug.allizom.org and let us know when we can change the DNS of the production site.

After the site has been made live from phx1 we will set you up with a second database and configure that for you. Please advise where the database details can be configured.

Comment 11

[Sebastian Zartner [:sebo]]

http://blog.getfirebug.allizom.org is not reachable and http://getfirebug.allizom.org gives me a 403 error.

> Please advise where the database details can be configured.
What exactly do you need from me?

Comment 12

[Brian Hourigan [:digi]]

Sebastian,

Rather than providing the database login credentials via e-mail please advise what configuration file I can add the database information to.

Comment 13

[Brian Hourigan [:digi]]

Sebastian,

The existence of config/config.php has caused problems with firstrun.php. While this is a blank file and it's in SVN, I would prefer for you to upload the script and specify where exactly I can input the credentials.

Comment 14

[Sebastian Zartner [:sebo]]

Funny thing is, that I didn't see any problems related to the first run page yesterday.
If you mean "commit to SVN" by "upload the script", then it's currently lying at http://getfirebug.com/releases/swarms/swarms.php.
Note, that I don't have an idea, of the actual server structure, so I could just give a relative path inside the Subversion repository.

Comment 15

[Brian Hourigan [:digi]]

Sebastian,

Please find the svn status here:
http://digi.pastebin.mozilla.org/1338629

I'll email you the URL to a tarball shortly.

I checked the swarms.php but I didn't see any area to configure the MySQL credentials.

Comment 16

[Brian Hourigan [:digi]]

Database info sent via email