1000207 - upload flame gecko/gaia mars to public ftp

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

I was going to test this on try, but we don't have builders there. I think it's safe enough to not warrant spending 2-3h waiting for a staging build.

Comment 1

[[e-mail] (:bhearsum)]

Attachment: upload flame mar

Comment 2

[[e-mail] (:bhearsum)]

Comment on attachment 8411036 [details] [diff] [review]
upload flame mar

https://hg.mozilla.org/mozilla-central/rev/b5395baf5883

16:47 < bhearsum> flame-only, just adds an extra file to the upload
16:47 < RyanVM> tell you waht, just push it to m-c DONTBUILD
16:47 < RyanVM> i'll be merging m-c back around soon anyway
16:48 < bhearsum> fine with me, as long as you're ok with me not being around until its builds finish
16:48 < RyanVM> DONTBUILD
16:48 < RyanVM> :)
16:48 < bhearsum> :)
16:48 < RyanVM> you weren't going to get quick builds anyway
16:48 < RyanVM> Flame is periodic
16:48 < bhearsum> oh, right
16:48 < bhearsum> thanks, much appreciated
16:49 < RyanVM> np

Comment 3

[[e-mail] (:bhearsum)]

With this landed, we should get MARs uploaded for the next Flame nightlies \o/

Comment 5

[[e-mail] (:bhearsum)]

So this worked, but we end up with files like http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/2014-04-24-04-02-01-mozilla-central/b2g-gecko-update.mar, which is going to be a problem when we have multiple devices doing it. I think we probably need to adjust the file names for these mars...

Comment 6

[[e-mail] (:bhearsum)]

(In reply to Ben Hearsum [:bhearsum] from comment #5)
> So this worked, but we end up with files like
> http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/2014-04-24-04-02-01-
> mozilla-central/b2g-gecko-update.mar, which is going to be a problem when we
> have multiple devices doing it. I think we probably need to adjust the file
> names for these mars...

That stuff is defined over at https://github.com/mozilla-b2g/gonk-misc/blob/master/Android.mk#L220. Maybe we can put $(PRODUCT_MODEL) in UPDATE_PACKAGE_TARGET? Not sure if that's going to be accessible or not.

Comment 7

[[e-mail] (:bhearsum)]

Attachment: upload any mar in b2g-update

(In reply to Ben Hearsum [:bhearsum] from comment #6)
> (In reply to Ben Hearsum [:bhearsum] from comment #5)
> > So this worked, but we end up with files like
> > http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/2014-04-24-04-02-01-
> > mozilla-central/b2g-gecko-update.mar, which is going to be a problem when we
> > have multiple devices doing it. I think we probably need to adjust the file
> > names for these mars...
> 
> That stuff is defined over at
> https://github.com/mozilla-b2g/gonk-misc/blob/master/Android.mk#L220. Maybe
> we can put $(PRODUCT_MODEL) in UPDATE_PACKAGE_TARGET? Not sure if that's
> going to be accessible or not.

I tried this out by hand and it looks like $(PRODUCT_MODEL) works. That's probably good enough for now.

Here's a patch to be less fussy about the filename of the mar we upload. I'll need to make a PR to gonk-misc to change the filename, I guess.

Comment 8

[[e-mail] (:bhearsum)]

Attachment: include $(PRODUCT_MODEL) in mar name

I think comment #5 and the pull request have all of the relevant info - let me know if you have any questions.

Comment 9

[[e-mail] (:bhearsum)]

Comment on attachment 8412629 [details] [diff] [review]
upload any mar in b2g-update

https://hg.mozilla.org/mozilla-central/rev/3d8e8213e08e

Comment 10

[Michael Wu [:mwu]]

Comment on attachment 8412639 [details] [review]
include $(PRODUCT_MODEL) in mar name

I agree with dhylands WRT the variables - PRODUCT_MODEL isn't what we want.

Comment 11

[[e-mail] (:bhearsum)]

OK, so bug 1001542 is fixed but I don't think that affected anything here, as it turns out. Should we just use $(PRODUCT_DEVICE) in the mar name instead, since that what we'll use in the update URL?

Comment 12

[Michael Wu [:mwu]]

Yeah. I think $(TARGET_DEVICE) is actually a little bit more correct in this case, so use that if it works.

Comment 13

[[e-mail] (:bhearsum)]

Looks like $(TARGET_DEVICE) works. I ended up with b2g-flame-gecko-update.mar, which sounds right to me. I'll send a new pull request.

Comment 14

[[e-mail] (:bhearsum)]

Comment on attachment 8412639 [details] [review]
include $(PRODUCT_MODEL) in mar name

I updated this pull request to use $(TARGET_DEVICE).

Comment 15

[Michael Wu [:mwu]]

Comment on attachment 8412639 [details] [review]
include $(PRODUCT_MODEL) in mar name

Looks fine, though you should probably rebase your patch so the merge is a bit cleaner.

Comment 16

[[e-mail] (:bhearsum)]

(In reply to Michael Wu [:mwu] from comment #15)
> Comment on attachment 8412639 [details] [review]
> include $(PRODUCT_MODEL) in mar name
> 
> Looks fine, though you should probably rebase your patch so the merge is a
> bit cleaner.

Done

Comment 17

[[e-mail] (:bhearsum)]

Ack - please hold off on merging this for now. I just discovered that our build scripts hardcode "b2g-gecko-update.mar".

Comment 18

[Dave Hylands [:dhylands]]

Comment on attachment 8412639 [details] [review]
include $(PRODUCT_MODEL) in mar name

When you do a build, stuff shows up in out/target/product/$(TARGET_DEVICE)/... so this feels like the right variable to use.

Although this is probably going to break a bunch of other things.

So I think that before we land this, we need to find all of the places which have b2g-gecko-update.mar hard-coded in them.

I don't know which scripts releng and QA use for creating updates and whether any of those depend on this name.

Comment 19

[[e-mail] (:bhearsum)]

(In reply to Dave Hylands [:dhylands] (away - back May 16) from comment #18)
> So I think that before we land this, we need to find all of the places which
> have b2g-gecko-update.mar hard-coded in them.
> 
> I don't know which scripts releng and QA use for creating updates and
> whether any of those depend on this name.

I've got a patch waiting on review in bug 1000208 to fix the build script to be compatible with both styles.

Looks like there's at least one QA test that needs fixing: http://mxr.mozilla.org/mozilla-central/source/testing/marionette/update-smoketests/stage-update.py#32. I'll get in contact with them to make sure there's nothing else.

Comment 20

[[e-mail] (:bhearsum)]

(In reply to Ben Hearsum [:bhearsum] from comment #19)
> (In reply to Dave Hylands [:dhylands] (away - back May 16) from comment #18)
> > So I think that before we land this, we need to find all of the places which
> > have b2g-gecko-update.mar hard-coded in them.
> > 
> > I don't know which scripts releng and QA use for creating updates and
> > whether any of those depend on this name.
> 
> I've got a patch waiting on review in bug 1000208 to fix the build script to
> be compatible with both styles.
> 
> Looks like there's at least one QA test that needs fixing:
> http://mxr.mozilla.org/mozilla-central/source/testing/marionette/update-
> smoketests/stage-update.py#32. I'll get in contact with them to make sure
> there's nothing else.

Jeff, I'm told you're the person to ask about this script. Do you know if it's still in use and if there's anything else that relies on "b2g-gecko-update.mar" as a name (either in dist/ or on FTP/pvtbuilds).

Comment 21

[Jeff Griffiths (:canuckistani) (:⚡︎)]

I'm really sure you mean someone else ( I work as a PM on Devtools )

Comment 22

[Aki Sasaki (not active)]

Pretty sure it's Jonathan.

Comment 23

[Jonathan Griffin (:jgriffin)]

So, that particular test hasn't been run for 6+ months, and would likely need considerable TLC before it could run again.  I don't know of anything else that relies on that filename.

Comment 24

[[e-mail] (:bhearsum)]

(In reply to Jonathan Griffin (:jgriffin) from comment #23)
> So, that particular test hasn't been run for 6+ months, and would likely
> need considerable TLC before it could run again.  I don't know of anything
> else that relies on that filename.

OK, so once my patch from 1000208 lands in production (probably later today), we should be able to merge the pull request. I'll update the bug when we're clear.

Comment 25

[[e-mail] (:bhearsum)]

OK, this can be merged anytime now. Dave, can you do that?

Comment 26

[Dave Hylands [:dhylands]]

https://github.com/mozilla-b2g/gonk-misc/commit/abedd83e1ba61cb83e59c723ad6279ad2a7aa1d1