1029851 - Tracking bug for 02-sep-2014 migration work

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Comment 1

[Aki Sasaki (not active)]

If there are Fennec 31.0b10 tag/relbranch conflicts during the Sept 1 mozilla-beta -> mozilla-release migration, see bug 1038421 for clues.

Comment 2

[Rail Aliiev [:rail]]

Attachment: mozilla-release-bump.diff

To be landed before we start the RC build.

Comment 3

[Hal Wine [:hwine] use NI!]

Comment on attachment 8473706 [details] [diff] [review]
mozilla-release-bump.diff

Review of attachment 8473706 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm! 6 weeks sure flies by

Comment 4

[Rail Aliiev [:rail]]

Attachment: [legacy vcs sync] start syncing aurora to v2.1

Will be landed and deployed on the merge day (Sep 2nd) according to https://wiki.mozilla.org/ReleaseEngineering/Merge_Duty/Steps#Bump_gecko.git_vcs-sync_.28odd_numbered_central_gecko_version.29

Comment 5

[Rail Aliiev [:rail]]

Attachment: [new vcs sync]: start syncing aurora to v2.1

Will be landed and deployed on the merge day (Sep 2nd) according to https://wiki.mozilla.org/ReleaseEngineering/Merge_Duty/Steps#Bump_gecko.git_vcs-sync_.28odd_numbered_central_gecko_version.29

Comment 6

[Rail Aliiev [:rail]]

Attachment: Pause Gecko l10n vcs-sync

https://wiki.mozilla.org/ReleaseEngineering/Merge_Duty/Steps#Pause_Gecko_l10n_vcs-sync

Comment 7

[Rail Aliiev [:rail]]

Attachment: add new Gaia and l10n to vcs-sync

https://wiki.mozilla.org/ReleaseEngineering/Merge_Duty/Steps#Add_new_Gaia_and_l10n_vcs-sync

Comment 8

[Hal Wine [:hwine] use NI!]

Comment on attachment 8474648 [details] [diff] [review]
[legacy vcs sync] start syncing aurora to v2.1

Review of attachment 8474648 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm

Comment 9

[Hal Wine [:hwine] use NI!]

Comment on attachment 8474652 [details] [diff] [review]
[new vcs sync]: start syncing aurora to v2.1

Review of attachment 8474652 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm

Comment 10

[Hal Wine [:hwine] use NI!]

Comment on attachment 8474664 [details] [diff] [review]
Pause Gecko l10n vcs-sync

Review of attachment 8474664 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm

Comment 11

[Hal Wine [:hwine] use NI!]

Comment on attachment 8474668 [details] [diff] [review]
add new Gaia and l10n to vcs-sync

Review of attachment 8474668 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm

Comment 12

[Hal Wine [:hwine] use NI!]

Rail: you also need the legacy vcs-sync patches for the l10n changes. That's the current "live" system.

Comment 13

[Rail Aliiev [:rail]]

(In reply to Hal Wine [:hwine] (use needinfo) from comment #12)
> Rail: you also need the legacy vcs-sync patches for the l10n changes. That's
> the current "live" system.

Thanks for the reminder. That part has always been a bit unclear for me. I'll prep the patches today/tomorrow.

Comment 14

[Rail Aliiev [:rail]]

Comment on attachment 8473706 [details] [diff] [review]
mozilla-release-bump.diff

https://hg.mozilla.org/build/buildbot-configs/rev/6f940be40e19

Comment 15

[Rail Aliiev [:rail]]

Attachment: merge-day-buildbot-configs.diff

https://wiki.mozilla.org/ReleaseEngineering/Merge_Duty/Steps#Merge_the_B2G_version_from_Central_to_Aurora

Comment 16

[Hal Wine [:hwine] use NI!]

Comment on attachment 8478286 [details] [diff] [review]
merge-day-buildbot-configs.diff

Review of attachment 8478286 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm - nice catch on nexus eng build

Comment 17

[Simone Bruno [:simone]]

In production with reconfig on 2014-08-25 08:49 PT

Comment 18

[Rail Aliiev [:rail]]

Attachment: merge-mozharness-1.diff

Mozharness patches from https://wiki.mozilla.org/ReleaseEngineering/Merge_Duty/Steps#Merge_the_B2G_version_from_Central_to_Aurora

Comment 19

[Rail Aliiev [:rail]]

Attachment: merge-puppet.diff

Puppet from https://wiki.mozilla.org/ReleaseEngineering/Merge_Duty/Steps#Merge_the_B2G_version_from_Central_to_Aurora

Comment 20

[Rail Aliiev [:rail]]

Attachment: aurora-in-tree-configs.diff

https://wiki.mozilla.org/ReleaseEngineering/Merge_Duty/Steps#Merge_the_B2G_version_from_Central_to_Aurora

Comment 21

[Rail Aliiev [:rail]]

Attachment: aurora-in-tree-configs.diff

Also bump B2G_UPDATE_CHANNEL

Comment 22

[Rail Aliiev [:rail]]

Attachment: central-bump-b2g-version.diff

Comment 23

[Rail Aliiev [:rail]]

Attachment: b2g_branch_repos.py.diff

flame.xml and flame-kk.xml refer kernel_lm  using "firefox-one" and firefox-one-v1.4" branches what confuses/brakes branching. Preferring "firefox-one" since we don't use flame-kk in our configs.

Comment 24

[Rail Aliiev [:rail]]

Attachment: Tag B2G Gecko+Gaia for MERGEDAY

https://wiki.mozilla.org/ReleaseEngineering/Merge_Duty/Steps#Tag_B2G_Gecko.2BGaia_for_MERGEDAY

Comment 25

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to Rail Aliiev [:rail] from comment #23)
> flame.xml and flame-kk.xml refer kernel_lm  using "firefox-one" and
> firefox-one-v1.4" branches what confuses/brakes branching. Preferring
> "firefox-one" since we don't use flame-kk in our configs.

Fairly soon we will have flame-kk, bug 1055305.

Comment 26

[Rail Aliiev [:rail]]

Attachment: Add new Gaia

https://wiki.mozilla.org/ReleaseEngineering/Merge_Duty/Steps#Add_new_Gaia_and_l10n_vcs-sync

Comment 27

[Hal Wine [:hwine] use NI!]

Comment on attachment 8478431 [details] [diff] [review]
merge-mozharness-1.diff

Review of attachment 8478431 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm

Comment 28

[Hal Wine [:hwine] use NI!]

Comment on attachment 8478529 [details] [diff] [review]
aurora-in-tree-configs.diff

Review of attachment 8478529 [details] [diff] [review]:
-----------------------------------------------------------------

all present and accounted for

Comment 29

[Hal Wine [:hwine] use NI!]

Comment on attachment 8478716 [details] [diff] [review]
b2g_branch_repos.py.diff

Review of attachment 8478716 [details] [diff] [review]:
-----------------------------------------------------------------

good for this merge day

::: configs/merge_day/b2g_branch_repos.py
@@ +29,5 @@
>          "b2g-4.3_r2.1",   # prefer jellybean over kitkat for now, since
>          "b2g-jellybean",  # most of our builds use jb
>          "ics_chocolate_rb4.2",  # prefer wasabi's hardware_qcom_display over
>                                  # otoro's
> +        "foxfone-one",  # prefer flame's kernel_lm ove flame-kk's

do we need to open a bug to handle flame-kk per comment 25

Comment 30

[Hal Wine [:hwine] use NI!]

Comment on attachment 8479470 [details] [diff] [review]
Add new Gaia

Review of attachment 8479470 [details] [diff] [review]:
-----------------------------------------------------------------

yay! doc update must have helped!

Comment 31

[Rail Aliiev [:rail]]

(In reply to Hal Wine [:hwine] (use needinfo) from comment #30)
> yay! doc update must have helped!

Yeah, thanks for the update!

Comment 32

[Rail Aliiev [:rail]]

(In reply to Hal Wine [:hwine] (use needinfo) from comment #29)
> do we need to open a bug to handle flame-kk per comment 25

Let me ask here first.

Viral,

in https://github.com/mozilla-b2g/b2g-manifest/commit/333d215958d5d12bb0a69f8a2c2a830462786e7c#diff-ca3d85637592a0cbef22a5d3e895f1d1R14 flame-kk started tracking foxfone-one-v1.4, while flame tracks the same repo using the foxfone-one branch.

This means, when we branch v2.1 off of master we have to use either foxfone-one-v1.4 or foxfone-one as a point on which v2.1 will be based on.

In other words if I branched today, v2.1 would look like this:
  $ git show-ref v2.1  origin/foxfone-one-v1.4 origin/foxfone-one
  9eb619d2efdf4bd121587d8296f5c10481f750b8 refs/heads/v2.1
  9eb619d2efdf4bd121587d8296f5c10481f750b8 refs/remotes/origin/foxfone-one
  9e62af4da848d56841bdde326f9bba26c743c33a refs/remotes/origin/foxfone-one-v1.4

Additionally, the new manifests would look like this:

flame-kk.xml (no changes):
...
<project name="kernel_lk" path="bootable/bootloader/lk" remote="b2g" revision="foxfone-one-v1.4"/>
...

flame.xml (changed from foxfone-one to v2.1):
...
<project name="kernel_lk" path="bootable/bootloader/lk" remote="b2g" revision="v2.1"/>
...

Any concerns regarding the changes above?

Comment 33

[Rail Aliiev [:rail]]

Comment on attachment 8478774 [details] [diff] [review]
Tag B2G Gecko+Gaia for MERGEDAY

https://hg.mozilla.org/build/mozharness/rev/a5b35bb943ba

Comment 34

[viral [:viralwang]]

(In reply to Rail Aliiev [:rail] from comment #32)
> (In reply to Hal Wine [:hwine] (use needinfo) from comment #29)
> > do we need to open a bug to handle flame-kk per comment 25
> 
> Let me ask here first.
> 
> Viral,
> 
> in
> https://github.com/mozilla-b2g/b2g-manifest/commit/
> 333d215958d5d12bb0a69f8a2c2a830462786e7c#diff-
> ca3d85637592a0cbef22a5d3e895f1d1R14 flame-kk started tracking
> foxfone-one-v1.4, while flame tracks the same repo using the foxfone-one
> branch.
> 
> This means, when we branch v2.1 off of master we have to use either
> foxfone-one-v1.4 or foxfone-one as a point on which v2.1 will be based on.
> 
> In other words if I branched today, v2.1 would look like this:
>   $ git show-ref v2.1  origin/foxfone-one-v1.4 origin/foxfone-one
>   9eb619d2efdf4bd121587d8296f5c10481f750b8 refs/heads/v2.1
>   9eb619d2efdf4bd121587d8296f5c10481f750b8 refs/remotes/origin/foxfone-one
>   9e62af4da848d56841bdde326f9bba26c743c33a
> refs/remotes/origin/foxfone-one-v1.4
> 
> Additionally, the new manifests would look like this:
> 
> flame-kk.xml (no changes):
> ...
> <project name="kernel_lk" path="bootable/bootloader/lk" remote="b2g"
> revision="foxfone-one-v1.4"/>
> ...
> 
> flame.xml (changed from foxfone-one to v2.1):
> ...
> <project name="kernel_lk" path="bootable/bootloader/lk" remote="b2g"
> revision="v2.1"/>
> ...
> 
> Any concerns regarding the changes above?
I think we should use the branch name like this:
refs/remotes/origin/foxfone-one => flame-jb
refs/remotes/origin/foxfone-one-v1.4 => flame-kk

Since partner only release jb+v1.3 and kk+v1.4 to us, they use foxfone-one-v1.4 for kk build.

I think reference phone need to support different combinations like:
jb+v1.4, jb+v2.0, jb+v2.1, kk+v1.4, kk+v2.0, kk+v2.1
for those jb base images, we will use refs/remotes/origin/foxfone-one
for those kk base images, we will use refs/remotes/origin/foxfone-one-v1.4

Hi Michael,

Any feedback in your side?
Thank you!

Comment 35

[Rail Aliiev [:rail]]

Does this mean that we shouldn't even create v2.1 branch in kernel_lk and continue using existing branches for kernel_lk in flame.xml and flame-kk.xml?

Comment 36

[Michael Wu [:mwu]]

I don't think we need to care about kk+v1.4. Also, jb+v2.1 can probably be ignored if the base images for KK are publicly released soon.

Comment 37

[Rail Aliiev [:rail]]

Comment on attachment 8478716 [details] [diff] [review]
b2g_branch_repos.py.diff

https://hg.mozilla.org/build/mozharness/rev/7c3eb9e93baa

Comment 38

[Rail Aliiev [:rail]]

Attachment: [new vcssync] start syncing gaia l10n from central to v2.1

Comment 39

[Rail Aliiev [:rail]]

Attachment: [new vcssync] start syncing geckco l10n (m-a -> v2.1, m-c ->v2.2)

Comment 40

[Rail Aliiev [:rail]]

Attachment: Branch gaia too

* Gaia needs to be branched, removing it from no_branch_repos and extra_branch_manifest_repos
* no_lock_revision_repos is not used anywhere, removing

Comment 41

[Hal Wine [:hwine] use NI!]

Comment on attachment 8481313 [details] [diff] [review]
[new vcssync] start syncing geckco l10n (m-a -> v2.1, m-c ->v2.2)

Review of attachment 8481313 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm for gecko

Comment 42

[Hal Wine [:hwine] use NI!]

Comment on attachment 8481307 [details] [diff] [review]
[new vcssync] start syncing gaia l10n from central to v2.1

Review of attachment 8481307 [details] [diff] [review]:
-----------------------------------------------------------------

r- -- this isn't for now, but when l10n cuts over

Comment 43

[Hal Wine [:hwine] use NI!]

Comment on attachment 8481335 [details] [diff] [review]
Branch gaia too

Review of attachment 8481335 [details] [diff] [review]:
-----------------------------------------------------------------

<stamp/>

Comment 44

[Rail Aliiev [:rail]]

Attachment: branch_gaia_too.diff

It turns out I need to leave "gaia" in extra_branch_manifest_repos to avoid locking it to a particular revision in b2g-manifests. Works "on my laptop". :)

Comment 45

[Rail Aliiev [:rail]]

Comment on attachment 8481307 [details] [diff] [review]
[new vcssync] start syncing gaia l10n from central to v2.1

Obsoleting this

Comment 46

[Rail Aliiev [:rail]]

Comment on attachment 8479470 [details] [diff] [review]
Add new Gaia

http://hg.mozilla.org/users/hwine_mozilla.com/repo-sync-configs/rev/a582f7bf0925

Comment 47

[Hal Wine [:hwine] use NI!]

Comment on attachment 8481515 [details] [diff] [review]
branch_gaia_too.diff

Review of attachment 8481515 [details] [diff] [review]:
-----------------------------------------------------------------

<stamp/>

Comment 48

[Jordan Lund (:jlund)]

something landed into production

Comment 49

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: aurora-in-tree-configs2.diff

Adds flame-kk to your patch.

Comment 50

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: merge-mozharness-2.diff

Adds recent additions (dolphin and flame-kk).

Comment 51

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8478286 [details] [diff] [review]
merge-day-buildbot-configs.diff

This will conflict with bug 1055305, fun times. We want to make sure we don't enable nightlies for flame-kk yet.

Comment 52

[Rail Aliiev [:rail]]

Comment on attachment 8481515 [details] [diff] [review]
branch_gaia_too.diff

https://hg.mozilla.org/build/mozharness/rev/0006e28cddc5

Comment 53

[Rail Aliiev [:rail]]

Attachment: tb_merge_day.diff

Comment 54

[Mark Banner (:standard8)]

Comment on attachment 8482674 [details] [diff] [review]
tb_merge_day.diff

Great, thanks!

Comment 55

[Rail Aliiev [:rail]]

Comment on attachment 8474652 [details] [diff] [review]
[new vcs sync]: start syncing aurora to v2.1

remote:   https://hg.mozilla.org/build/mozharness/rev/387f944f12c2
remote:   https://hg.mozilla.org/build/mozharness/rev/2873e9a03cd5

Comment 56

[Rail Aliiev [:rail]]

Comment on attachment 8474648 [details] [diff] [review]
[legacy vcs sync] start syncing aurora to v2.1

http://hg.mozilla.org/users/hwine_mozilla.com/repo-sync-tools/rev/d7c5541bc1c4

Comment 57

[Rail Aliiev [:rail]]

Comment on attachment 8478286 [details] [diff] [review]
merge-day-buildbot-configs.diff

https://hg.mozilla.org/build/buildbot-configs/rev/2790b9917159

Comment 58

[Rail Aliiev [:rail]]

Comment on attachment 8482674 [details] [diff] [review]
tb_merge_day.diff

https://hg.mozilla.org/build/buildbot-configs/rev/9408a2be554c

Comment 59

[Rail Aliiev [:rail]]

Comment on attachment 8474664 [details] [diff] [review]
Pause Gecko l10n vcs-sync

remote:   https://hg.mozilla.org/build/mozharness/rev/3eee2285d995
remote:   https://hg.mozilla.org/build/mozharness/rev/53e741142b24

Comment 60

[Rail Aliiev [:rail]]

Comment on attachment 8482087 [details] [diff] [review]
merge-mozharness-2.diff

remote:   https://hg.mozilla.org/build/mozharness/rev/e27b4b2e88eb
remote:   https://hg.mozilla.org/build/mozharness/rev/23ba32e4ae89

Comment 61

[Rail Aliiev [:rail]]

Comment on attachment 8478434 [details] [diff] [review]
merge-puppet.diff

remote:   https://hg.mozilla.org/build/puppet/rev/9297871b08a9
remote:   https://hg.mozilla.org/build/puppet/rev/7d0c85b4e7f0

Comment 62

[Rail Aliiev [:rail]]

Comment on attachment 8482085 [details] [diff] [review]
aurora-in-tree-configs2.diff

https://hg.mozilla.org/releases/mozilla-aurora/rev/5b8210dcf52a

Comment 63

[Rail Aliiev [:rail]]

Comment on attachment 8478532 [details] [diff] [review]
central-bump-b2g-version.diff

https://hg.mozilla.org/mozilla-central/rev/065b9c02859c

Comment 64

[Rail Aliiev [:rail]]

Comment on attachment 8481313 [details] [diff] [review]
[new vcssync] start syncing geckco l10n (m-a -> v2.1, m-c ->v2.2)

remote:   https://hg.mozilla.org/build/mozharness/rev/5675ff8803dd
remote:   https://hg.mozilla.org/build/mozharness/rev/a7ce19755ef4

Comment 65

[Ryan VanderMeulen [:RyanVM]]

Apparently it takes a regular Bumper Bot push to point Gaia at the correct repo post-uplift:
https://hg.mozilla.org/releases/mozilla-aurora/rev/9698450944cd

This may explain the rash of random bustage we had in the pushes prior to that. Is there any reason we couldn't manually make the needed gaia.json changes along with theother post-uplift changes already being made like in the push below?
https://hg.mozilla.org/releases/mozilla-aurora/rev/5b8210dcf52a

Comment 66

[Rail Aliiev [:rail]]

I doubt that it would help if we changed the URL manually and left the changeset the same (because we branched at that point). Or there is something that I'm missing?

Comment 67

[Ryan VanderMeulen [:RyanVM]]

Update the changeset too?

Comment 68

[Rail Aliiev [:rail]]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #65)
> Apparently it takes a regular Bumper Bot push to point Gaia at the correct
> repo post-uplift:
> https://hg.mozilla.org/releases/mozilla-aurora/rev/9698450944cd
> 
> This may explain the rash of random bustage we had in the pushes prior to
> that. Is there any reason we couldn't manually make the needed gaia.json
> changes along with theother post-uplift changes already being made like in
> the push below?
> https://hg.mozilla.org/releases/mozilla-aurora/rev/5b8210dcf52a

One of the theories I have:

1) branching of the b2g repos happens
2) gaia v2.1 starts the conversion to intergration/gaia-2_1
3) central to aurora merge happens, gaia.json uses 14400ee07e836d74039e2317feebc0a13fcb60c8
4) intergration/gaia-2_1 is finally ready (it took multiple hours after step 2)
5) the bumper detect the bump way behind (there are ~70 changesets between 14400ee07e836d74039e2317feebc0a13fcb60c8 and 1e5e149934e3eb86064575e4bdf0bdef8d39e67b)

Hal may have some ideas too.

Comment 69

[Chris Cooper [:coop] (he/him)]

Merged to production, and deployed. (yesterday)

Comment 70

[Hal Wine [:hwine] use NI!]

I'm not familiar with bumper requirements and assumptions in detail. I do suspect that the several hour delay required to create the integration/gaia-2_1 hg repo would throw it off.

Rail & I have discussed ways to shorten the conversion time next time. That should help.

I do note that the phrase "random bustage we had in the pushes prior to [uplift]" bothers me a bit. I don't know if the design of the bumper is that "mid uplift" builds are valid. The conservative approach would be to ensure they don't occur.

Comment 71

[Pete Moore [:pmoore][:pete]]

Attachment: [new vcssync]: updating l10n staging config

Comment 72

[Pete Moore [:pmoore][:pete]]

Comment on attachment 8493711 [details] [diff] [review]
[new vcssync]: updating l10n staging config

On default branch:     https://hg.mozilla.org/build/mozharness/rev/f12caf1970aa
On production branch:  https://hg.mozilla.org/build/mozharness/rev/83effe9ef7fc