1021247 - Store Jenkins configs for B2G Flame Jenkins instance in source control

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Infrastructure, defect)

Description

[Jonathan Griffin (:jgriffin)]

Once the Jenkins instance for the flame devices is setup, we'll want to configure it (somehow) to store its config files in source control; this will enable us to know how jobs were invoked at arbitrary points in the past.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I'm not aware of a great solution for this, however both Mozmill and Eideticker store the Jenkins configuration in github (see links below). As Jenkins configuration is entirely file based, it's relatively easy to store in source control, however each deployment will likely have environment specific configurations. For Mozmill these are handled using patch files per environment, but for Eideticker these configurations are maintained manually. They include things like passwords, authentication, domain names, etc.

I have been considering writing a small setup script for the Eideticker CI which would prompt users for the configuration, providing suitable defaults where appropriate and then using something like Selenium and a headless browser to make the changes via the UI. In the meantime most updates allow us to simply stash the local changes on the server, pull in the updates, and reapply the stash.

There is an SCM Sync configuration plugin (https://wiki.jenkins-ci.org/display/JENKINS/SCM+Sync+configuration+plugin) which may be worth looking into, although it does have a large number of open issues.

Eideticker CI: https://github.com/mozilla/eideticker-ci
Mozmill CI: https://github.com/mozilla/mozmill-ci

Comment 2

[Malini Das [:mdas] - Away, not checking bugmail]

(In reply to Dave Hunt (:davehunt) [PTO until 9th July] from comment #1)
> I'm not aware of a great solution for this, however both Mozmill and
> Eideticker store the Jenkins configuration in github (see links below). As
> Jenkins configuration is entirely file based, it's relatively easy to store
> in source control, however each deployment will likely have environment
> specific configurations. For Mozmill these are handled using patch files per
> environment, but for Eideticker these configurations are maintained
> manually. They include things like passwords, authentication, domain names,
> etc.
> 
> I have been considering writing a small setup script for the Eideticker CI
> which would prompt users for the configuration, providing suitable defaults
> where appropriate and then using something like Selenium and a headless
> browser to make the changes via the UI. In the meantime most updates allow
> us to simply stash the local changes on the server, pull in the updates, and
> reapply the stash.

If we know the exact location of the config/managed scripts, then perhaps there's another way other than stashing/applying. The scripts can be stored in github, and set up a commit listener in jenkins. When a commit is made, jenkins pulls in the new config/managed scripts and we can run a find and replace script on these files to add any environment specific information, or use environment variables to fill that data in. Would that work?

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Malini Das [:mdas] from comment #2)
> (In reply to Dave Hunt (:davehunt) [PTO until 9th July] from comment #1)
> > I'm not aware of a great solution for this, however both Mozmill and
> > Eideticker store the Jenkins configuration in github (see links below). As
> > Jenkins configuration is entirely file based, it's relatively easy to store
> > in source control, however each deployment will likely have environment
> > specific configurations. For Mozmill these are handled using patch files per
> > environment, but for Eideticker these configurations are maintained
> > manually. They include things like passwords, authentication, domain names,
> > etc.
> > 
> > I have been considering writing a small setup script for the Eideticker CI
> > which would prompt users for the configuration, providing suitable defaults
> > where appropriate and then using something like Selenium and a headless
> > browser to make the changes via the UI. In the meantime most updates allow
> > us to simply stash the local changes on the server, pull in the updates, and
> > reapply the stash.
> 
> If we know the exact location of the config/managed scripts, then perhaps
> there's another way other than stashing/applying. The scripts can be stored
> in github, and set up a commit listener in jenkins. When a commit is made,
> jenkins pulls in the new config/managed scripts and we can run a find and
> replace script on these files to add any environment specific information,
> or use environment variables to fill that data in. Would that work?

Yeah, I think that would work. Sometimes it's not just entries in scripts though but entire config files or sections. For example, we're now using the role strategy plugin, but we might not want that or our user's details in source control.

Comment 4

[BMO Automation]

Firefox OS is not being worked on