1011489 - hamachi device builds submitting bad urls to balrog

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

They use URLs like http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/latest-mozilla-central/fota-update.mar

These are wrong both because they use the latest dir (bug 1011486) and because fota-update.mar doesn't get uploaded to public FTP.

IIRC, Hamachi updates are served as a FOTA because of space limitations on the device. I'm pretty sure these are already in public via update.boot2gecko.org (eg http://update.boot2gecko.org/hamachi/2.0.0/nightly/b2g_update_20140516040203.mar?build_id=20140516040203&version=32.0a1).

I think we need to do a few things:
* adjust query_marfile to return the right thing
* adjust the public upload config to get them uploaded to FTP. 
* fix the name to be something with "hamachi" in it.

Aki, any thoughts?

Comment 1

[Aki Sasaki (not active)]

Thoughts:

* balrog can handle fota now?
* looks like hamachi, for space reasons, and tarako, for business? memory? reasons, are both fota
* other than the extra snippet flag, and probably no partials, I *think* we can otherwise handle these like normal update mars.
* did you have any specific questions? I think your above statements look correct.

Comment 2

[[e-mail] (:bhearsum)]

(In reply to Ben Hearsum [:bhearsum] from comment #0)
> I think we need to do a few things:
> * adjust query_marfile to return the right thing

This was done in bug 1011550.

> * adjust the public upload config to get them uploaded to FTP. 

Doing this soon.

> * fix the name to be something with "hamachi" in it.

Waiting for this to be merged into gonk-misc, tracked in bug 1011550.

(In reply to Aki Sasaki [:aki] from comment #1)
> Thoughts:
> 
> * balrog can handle fota now?

Nope, this bug is only about getting the right FTP bits. bug 1000221 will add that support, and then we'll probably need some real tweaks.

> * looks like hamachi, for space reasons, and tarako, for business? memory?
> reasons, are both fota

OK, good to know. It's unclear if we'll be backporting as far as 1.3, so only need to worry about Hamach for now...

Comment 3

[[e-mail] (:bhearsum)]

(In reply to Ben Hearsum [:bhearsum] from comment #2)
> (In reply to Ben Hearsum [:bhearsum] from comment #0)
> > I think we need to do a few things:
> > * adjust query_marfile to return the right thing
> 
> This was done in bug 1011550.
> 
> > * adjust the public upload config to get them uploaded to FTP. 
> 
> Doing this soon.

This was actually done in bug 1010313, but because the FOTA MAR isn't created until after prep-upload, it doesn't get uploaded =\. Aki, is there any reason we can't do prep-upload/upload after make-update-xml (which seems like a weird place to be creating the FOTA, come to think of it....). I can't see anything in that step that depends on prep-upload or upload.

Comment 4

[Aki Sasaki (not active)]

(In reply to Ben Hearsum [:bhearsum] from comment #3)
> (In reply to Ben Hearsum [:bhearsum] from comment #2)
> > (In reply to Ben Hearsum [:bhearsum] from comment #0)
> > > I think we need to do a few things:
> > > * adjust query_marfile to return the right thing
> > 
> > This was done in bug 1011550.
> > 
> > > * adjust the public upload config to get them uploaded to FTP. 
> > 
> > Doing this soon.
> 
> This was actually done in bug 1010313, but because the FOTA MAR isn't
> created until after prep-upload, it doesn't get uploaded =\. Aki, is there
> any reason we can't do prep-upload/upload after make-update-xml (which seems
> like a weird place to be creating the FOTA, come to think of it....). I
> can't see anything in that step that depends on prep-upload or upload.

I think it's created in make-updates, but not copied to the upload dir til make-update-xml.  This is probably so we don't upload it during the upload step, but instead during the upload-updates step.
http://hg.mozilla.org/build/mozharness/file/c160e7b19eee/scripts/b2g_build.py#l1493

I might be wrong, but:

* I think you're trying to keep current updates working until you're switching over
* the FOTA mar was created in make-updates, which happens before prep-upload
* with just a brief glance, I think it makes sense to copy the FOTA mar into the upload dir in prep-upload for balrog enabled devices and leave it in make-update-xml for non-balrog-enabled devices.  

After we switch everything to balrog, I think we can get rid of make-update-xml and upload-updates.

Comment 5

[[e-mail] (:bhearsum)]

Attachment: fix-hamachi-path.diff

Per IRC, turns out we're just globbing wrong, yay!

Assuming ${workdir} is "build" (which I think it is, based on sources.xml being there), this should work.

Comment 6

[[e-mail] (:bhearsum)]

Comment on attachment 8427986 [details] [diff] [review]
fix-hamachi-path.diff

https://hg.mozilla.org/integration/b2g-inbound/rev/77a75db90ff4

Comment 7

[[e-mail] (:bhearsum)]

Comment on attachment 8427986 [details] [diff] [review]
fix-hamachi-path.diff

Pushed directly to central, too: https://hg.mozilla.org/mozilla-central/rev/37cc88af7669

Comment 8

[[e-mail] (:bhearsum)]

Hamachi MARs are now being uploaded to places such as http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/2014-05-26-12-22-57-mozilla-central-hamachi/fota-update.mar. The name will change when bug 1011550 gets merged. Balrog is getting the right link, too: https://aus4-admin.mozilla.org/releases/B2G-mozilla-central-nightly-latest/data.

Comment 9

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/77a75db90ff4

Comment 10

[[e-mail] (:bhearsum)]

We're all good now. Balrog has accurate, dated URLs to Hamachi MARs now.