736464 - Put rsyncd configs under version control

Status: RESOLVED WONTFIX

(Release Engineering :: Release Automation: Other, defect, P3)

Description

[Rail Aliiev [:rail]]

During Firefox 11.0 release due to a mistake made in rsyncd configs the external mirrors didn't get the files. To prevent such mistakes in the future it will be better to put rsyncd configs under version control and use reviews for changes.

In the future we can split the files by product, channel to simplify automation.

AFAIK, there is no private information in those files and we can use /build/rsyncd-configs hg repo for it.

Ideas, suggestions, objections?

Comment 1

[[e-mail] (:bhearsum)]

I think this is a great idea, and if we pull them via cron or some other automated process it removes another manual touchpoint from our release process. I'm not too fussy about how they get split up as long as they're still understandable by humans.

cc'ing Dave and Dumitru to get an IT perspective/opinion.

Comment 2

[Justin Wood (:Callek)]

If pulled via cron also means that making SeaMonkey additions/changes does not need to take up others time (though still should be coordinated)

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

I don't want to derail this in an access control discussion, so we'll keep this managed by RelEng for now. Unless anyone has objections I suggest a new repo in http://hg.mozilla.org/build/.

Comment 4

[Justin Wood (:Callek)]

(In reply to Nick Thomas [:nthomas] from comment #3)
> I don't want to derail this in an access control discussion

O sure, I did not mean to imply that I would/we should change the current policies with it. Was more of a "here is a potential process improvement we can gain from doing so as well"

Comment 5

[Dave Miller [:justdave]]

These configs are currently synced out to all the mirrors, in fact, so they're entirely public. :)

They currently go in the /pub/mozilla.org/zz/ directory

-rw-r--r-- 1 root   root       29 Mar 19 20:00 marker.txt
-rw-r--r-- 1 root   root      560 Mar 19 04:43 rsyncd-motd
-rw-r--r-- 1 cltbld mozilla   163 Mar 13 12:38 rsyncd-mozilla-current.exclude
-rw-r--r-- 1 cltbld mozilla 20158 Mar 15 20:10 rsyncd-mozilla-prereleases.exclude
-rw-r--r-- 1 cltbld mozilla 14203 Mar 13 23:06 rsyncd-mozilla-releases.exclude

The two files owned by root are both generated by cron jobs.  marker.txt every 5 minutes, rsync-motd daily.  We can put the entire directory under version control if you add those two files to whatever the equivalent of svnignore is.

Comment 6

[[e-mail] (:bhearsum)]

I think this is actually an Automation bug.

Comment 7

[Aki Sasaki (not active)]

Would stage cleanup crontab entries fit in this new repo?

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

That sounds more like puppet to me.

Comment 9

[[e-mail] (:bhearsum)]

Bulk move of bugs to Release Automation component.

Comment 10

[Rail Aliiev [:rail]]

Attachment: initial checkin

Comments incoming

Comment 11

[Rail Aliiev [:rail]]

Comment on attachment 619639 [details] [diff] [review]
initial checkin

Review of attachment 619639 [details] [diff] [review]:
-----------------------------------------------------------------

You can skip *.exclude files, they are copy/paste from stage. I'll update them before landing the patch.

Maybe we could just copy the files always (without make timestamp magic) and get rid of force-install target and dummy touches.

::: stage.mozilla.org/Makefile
@@ +3,5 @@
> +
> +diff:
> +	@for f in $(wildcard *.$(SUFFIX)); do \
> +	    diff -u $$f $(TARGET)/$$f; \
> +	done

default target, diffs configs

@@ +7,5 @@
> +	done
> +
> +install: $(foreach f,$(wildcard *.$(SUFFIX)),$(TARGET)/$(f))
> +
> +force-install: touch install

overwrites target files (to prevent local modifications)

Comment 12

[Rail Aliiev [:rail]]

Comment on attachment 619639 [details] [diff] [review]
initial checkin

New stage has these files under SVN. No more needed.