633953 - separate pre push checks and pre-release virus scan

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[[e-mail] (:bhearsum)]

Currently, doing the pre-release virus scan is blocked on all of the pre push permission checks succeeding. This is unfortunate in cases where one of the pre push checks fail, because the virus scan ends up being delayed until we can make it pass. This sometimes delays the release. There's no reason to block the virus scan on these checks succeeding, so we should do them in parallel.

Comment 1

[Rail Aliiev [:rail]]

Attachment: scripts

* push-to-mirrors.py accepts "permissions", "antivirus", and "push" parameters
* dry run rsync is run after both "permissions" and "antivirus"

Comment 2

[Rail Aliiev [:rail]]

Attachment: buildbotcustom

* not tested yet
* revision and release_config properties are required by push-to-mirrors scripts. revision is set to a "floating" revision (FIREFOX_*_RELEASE), release_config is passed from buildbot-configs (see the next patch), so the builders are still reconfig safe.
* push_to_mirrors still requires revision and release_config to be set manually (it's safer)
* The patch should also fix bug 633957, you'll just need to rebuild check_permissions and antivirus builders manually, w/o setting any properties.

Comment 3

[Rail Aliiev [:rail]]

Attachment: configs

Comment 4

[[e-mail] (:bhearsum)]

Comment on attachment 512178 [details] [diff] [review]
buildbotcustom

This is good overall, but I explicitly don't want revision to be set, because it changes per-release. I know it's overridable, but in new code I want to avoid *anything* being set at reconfig that changes per-release. Definitely like the releaseConfigFile stuff.

Comment 5

[[e-mail] (:bhearsum)]

Comment on attachment 512178 [details] [diff] [review]
buildbotcustom

Rail reminded me that these aren't being force build'ed. It sucks that we require a reconfig for this, but it's unavoidable for now. Post signing on demand these can be Dependent fired, and we can drop the revision/script_repo_revision Builder properties at that point.

Comment 6

[Rail Aliiev [:rail]]

Comment on attachment 512179 [details] [diff] [review]
configs

Nothing changed here after a staging run.

Comment 7

[Rail Aliiev [:rail]]

Attachment: scripts

This version should also fix bug 634237. The documentation should be updated as well.

Comment 8

[Rail Aliiev [:rail]]

Attachment: buildbotcustom

* -        builderNames=builderPrefix(...)
  +        builderNames=[builderPrefix(...)]
* Fixes bug 634237 as well

Comment 9

[Rail Aliiev [:rail]]

Attachment: scripts

Updated diff

Comment 10

[Rail Aliiev [:rail]]

Attachment: buildbotcustom

Updated against bug 635150

Comment 11

[Rail Aliiev [:rail]]

Comment on attachment 513480 [details] [diff] [review]
scripts

http://hg.mozilla.org/build/tools/rev/dd11634aac33

Comment 12

[Rail Aliiev [:rail]]

Comment on attachment 513481 [details] [diff] [review]
buildbotcustom

http://hg.mozilla.org/build/buildbotcustom/rev/723c8aaff82e
merged to production in http://hg.mozilla.org/build/buildbotcustom/rev/1046bc8c7e00

Comment 13

[Rail Aliiev [:rail]]

Comment on attachment 512179 [details] [diff] [review]
configs

http://hg.mozilla.org/build/buildbot-configs/rev/5d164675d55b
merged to default in http://hg.mozilla.org/build/buildbot-configs/rev/ff0e345be8f2

Comment 14

[Rail Aliiev [:rail]]

Yay! I see new builders on the waterfall pages. :)