As a security precaution, we have turned on the setting "Require API key authentication for API requests" for everyone. If this has broken something, please contact
Last Comment Bug 677690 - Need FTP access for one folder at
: Need FTP access for one folder at
Product: Infrastructure & Operations
Classification: Other
Component: WebOps: Other (show other bugs)
: other
: x86 Windows XP
: -- normal (vote)
: ---
Assigned To: Brian Hourigan [:digi]
: Corey Shields [:cshields]
Depends on:
  Show dependency treegraph
Reported: 2011-08-09 14:16 PDT by Sebastian Zartner
Modified: 2013-10-09 10:29 PDT (History)
6 users (show)
See Also:
Due Date:
QA Whiteboard:
Iteration: ---
Points: ---
Cab Review: ServiceNow Change Request (use flag)


Description User image Sebastian Zartner 2011-08-09 14:16:38 PDT
I am trying to read XPI files for our Swarms page. Because reading the data from the XPIs takes some time, I want to cache the data somewhere as files.
Normally we're just having access to via a SVN repository, which is synchronized with the online page. Though because we need to be able to change the data created by the script manually, but it bypasses the repository, we're not having access to it.

So what is needed?
An FTP account, which has access to one specific folder, which lies outside of the with Subversion synchronized root.
Alternatively the data could be stored in a database, but we also don't have success to that.
Comment 1 User image Dave Camp (:dcamp) 2011-09-08 11:28:18 PDT
Can we get this done for the firebug guys please?
Comment 2 User image Corey Shields [:cshields] 2011-09-08 11:32:29 PDT
I'm not quite sure what is the end result needed here but we don't do ftp read access to something like this.

Is there a way to speed the XPI up?
Comment 3 User image Sebastian Zartner 2011-09-08 14:18:35 PDT
Thanks Dave for pushing this.

> I'm not quite sure what is the end result needed here but we don't do ftp read 
> access to something like this.
Ok, little more explanation: The script reads the XPIs, extracts the install.rdf, reads the data from it and then stores that data as separate XML files in a specific folder. The next time you're accessing the script it checks for the XML file and, if it exists, reads the data from there instead of having to extract it from the XPI again AND - and that is the more important part here - we want to add Firebug version compatibility information, because most of the extensions don't have that info in the XPI.

> Is there a way to speed the XPI up?
No, that's why I am storing the data separately. The second reason for saving the data separately is to avoid influencing download statistics. I bet you're also not reading the XPIs everytime someone is at an AMO site, do you?

As I said I also wouldn't have a problem to store that data in a database. Then of course we would need an account for that database.
Comment 4 User image Corey Shields [:cshields] 2011-09-09 11:56:46 PDT
(In reply to Sebastian Zartner from comment #3)
> As I said I also wouldn't have a problem to store that data in a database.
> Then of course we would need an account for that database.

Let's do a database then..  Are you ready for a DB now or do you need some time to work it into the app?
Comment 5 User image Sebastian Zartner 2011-09-09 12:16:09 PDT
My "app" is just consisting of one file, so I will need a bit to replace the XML access by a DB access. Though you can already set up the DB. What do you need from me?
Comment 6 User image Sebastian Zartner 2011-09-20 12:13:42 PDT
Would it be possible to set up a MySQL database for this purpose?
Comment 7 User image Corey Shields [:cshields] 2011-09-22 11:24:58 PDT
Hi Sebastian,

Brian is almost done prepping in the new cluster, I'll have him setup an extra db for this and send you the credentials.  This will be available once the site turns over per bug 684704.
Comment 8 User image Corey Shields [:cshields] 2011-09-22 11:27:06 PDT
Also, is there processing that will need to be done on a regular basis to populate this DB (a script we need to cron somewhere), or will your app take care of the xpi processing?

Comment 9 User image Sebastian Zartner 2011-09-22 15:07:40 PDT
That are great news.
The table inside the DB will be populated automatically by my script.
Currently you just have to add the URL to an XPI to that script and the rest is done automatically.
As soon as the DB is set up, the URL will be put into the DB instead and the script will save the data from the XPIs in there. I guess that's similar to what you're doing on AMO. Just instead of reading the data from a URL you let the people upload their extensions.

One question related to the compatibility checking. I just saw lately, that the requires property was removed from the install manifests ( What was the reason for that? Is there another way to achieve that now?
Comment 10 User image Brian Hourigan [:digi] 2011-09-22 18:02:31 PDT

We have migrated blog.getfirebug/getfirebug to our phx1 staging and development environment. Please test the site at and and let us know when we can change the DNS of the production site.

After the site has been made live from phx1 we will set you up with a second database and configure that for you. Please advise where the database details can be configured.
Comment 11 User image Sebastian Zartner 2011-09-23 04:21:26 PDT is not reachable and gives me a 403 error.

> Please advise where the database details can be configured.
What exactly do you need from me?
Comment 12 User image Brian Hourigan [:digi] 2011-09-23 07:43:17 PDT

Rather than providing the database login credentials via e-mail please advise what configuration file I can add the database information to.
Comment 13 User image Brian Hourigan [:digi] 2011-09-27 16:06:40 PDT

The existence of config/config.php has caused problems with firstrun.php. While this is a blank file and it's in SVN, I would prefer for you to upload the script and specify where exactly I can input the credentials.
Comment 14 User image Sebastian Zartner 2011-09-27 22:42:23 PDT
Funny thing is, that I didn't see any problems related to the first run page yesterday.
If you mean "commit to SVN" by "upload the script", then it's currently lying at
Note, that I don't have an idea, of the actual server structure, so I could just give a relative path inside the Subversion repository.
Comment 15 User image Brian Hourigan [:digi] 2011-09-28 13:18:11 PDT

Please find the svn status here:

I'll email you the URL to a tarball shortly.

I checked the swarms.php but I didn't see any area to configure the MySQL credentials.
Comment 16 User image Brian Hourigan [:digi] 2011-09-28 13:29:35 PDT
Database info sent via email

Note You need to log in before you can comment on or make changes to this bug.