1148930 - Support bitbucket repos

Status: RESOLVED FIXED

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Wander Lairson Costa]

github is very unstable due to DDoS attacks, lets allow mozharness support bitbucket.

Comment 1

[Wander Lairson Costa]

Attachment: Allow mozharness build bitbucket repos.

Comment 2

[Justin Wood (:Callek)]

Comment on attachment 8585190 [details] [diff] [review]
Allow mozharness build bitbucket repos.

Review of attachment 8585190 [details] [diff] [review]:
-----------------------------------------------------------------

there are security implications in allowing bitbucket from automation. I'd love a sec review before we allow that. But that said, I'm not interested enough or aware enough of related blockers with the github DDoS to follow closely.

Comment 3

[Wander Lairson Costa]

(In reply to Justin Wood (:Callek) from comment #2)
> Comment on attachment 8585190 [details] [diff] [review]
> Allow mozharness build bitbucket repos.
> 
> Review of attachment 8585190 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> there are security implications in allowing bitbucket from automation. I'd
> love a sec review before we allow that. But that said, I'm not interested
> enough or aware enough of related blockers with the github DDoS to follow
> closely.

This is a taskcluster thing. Taskcluster client code lives in the gecko tree, so when we change this code and have to test it, we push our tree to personal github repo and tell taskcluster to use it. With the DDoS attacks, often taskcluster has problems to clone the repo.

Comment 4

[Pete Moore [:pmoore][:pete]]

Comment on attachment 8585190 [details] [diff] [review]
Allow mozharness build bitbucket repos.

Review of attachment 8585190 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good to me! I've tested and can confirm that bitbucket follows the same url path convention as github ({repo}/raw/{rev}/{filename}), e.g.:

https://bitbucket.org/hpk42/tox/raw/45733e7d58f2617938fd2e9dd0c64860785b3974/CHANGELOG
https://github.com/mozilla/build-mozharness/raw/1a3c6f032d48effbe254756725b278f7be3f44a6/configs/android/android_panda_releng.py

Thanks for looking into this Wander!

Pete

Comment 5

[Pete Moore [:pmoore][:pete]]

(In reply to Justin Wood (:Callek) from comment #2)
> Comment on attachment 8585190 [details] [diff] [review]
> Allow mozharness build bitbucket repos.
> 
> Review of attachment 8585190 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> there are security implications in allowing bitbucket from automation. I'd
> love a sec review before we allow that. But that said, I'm not interested
> enough or aware enough of related blockers with the github DDoS to follow
> closely.

I think a sec review isn't necessary since the code does not switch anything to use bitbucket, it simply fixes the previously broken attempt to grab a raw file from your git repo, if you happen to use bitbucket. If bitbucket had followed a different naming convention, it would have always worked. We have no whitelisting of supported git providers, so it has always been possible to use arbitrary git repo urls in the config files, and this fix doesn't change anything there.

Given that the DDoS attacks on github at the moment are impacting our ability to deliver work, this should hopefully improve the situation on the ground in the short term.

Comment 6

[Pete Moore [:pmoore][:pete]]

Comment on attachment 8585190 [details] [diff] [review]
Allow mozharness build bitbucket repos.

Landed on default: https://hg.mozilla.org/build/mozharness/rev/386df53d1604

Currently checking with coop|buildduty whether I should also merge to production...

Comment 7

[Chris Cooper [:coop] (he/him)]

mozharness production tag moved to: https://hg.mozilla.org/build/mozharness/rev/production