Closed Bug 922145 Opened 8 years ago Closed 8 years ago

publish nightly builds of GeckoView library and assets

Categories

(Firefox for Android Graveyard :: General, defect)

x86
macOS
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
Firefox 27

People

(Reporter: blassey, Assigned: blassey)

References

Details

Attachments

(2 files, 1 obsolete file)

No description provided.
Flags: needinfo?(armenzg)
Armen, what do we need to do to put the GeckoView library and assets
I think I need a little bit of more context in here.

Is the code already part of the Android builds?
Is this a change for Releng? or for mfinkle to land a change to enable the code on the builds?
Is this code in a given branch?

Sorry I could not be of more help at first.
Flags: needinfo?(armenzg) → needinfo?(blassey.bugs)
This code is on central. Essentially we want to make sure that the contents of dist/geckoview_library/ end up on ftp. currently that is geckoview_assets.zip and geckoview_library.zip.
Flags: needinfo?(blassey.bugs) → needinfo?(armenzg)
After an IRC conversation I understand this a bit better.
We are now creating a zip file in our build process for a zip file that developers can use to create apps based on gecko for mobile.

* There's no need to sign it
* We will be releasing this on every fennec release

I don't know much of the code, however, I believe these pieces of code are related:
http://mxr.mozilla.org/build/source/buildbotcustom/process/factory.py#203
http://mxr.mozilla.org/build/source/buildbotcustom/process/factory.py#2419
Flags: needinfo?(armenzg)
Assignee: nobody → blassey.bugs
Attachment #814463 - Flags: review?(ted)
Comment on attachment 814463 [details] [diff] [review]
upload_geckoview.patch

Should you add a check for MOZ_DISABLE_GECKOVIEW? We added that to stop l10n builds from attempting to make the archives.
After asking to the rest of releng, that a build patch is the only thing needed to add this to the nightly directories.

For the release side we might only need to modify push_fennec.sh:
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Push_Fennec_to_mirrors
Comment on attachment 814463 [details] [diff] [review]
upload_geckoview.patch

Review of attachment 814463 [details] [diff] [review]:
-----------------------------------------------------------------

We actually wildcard everything in UPLOAD_EXTRA_FILES, so having it missing is not harmful.
Attachment #814463 - Flags: review?(ted) → review+
It seems that the two files get uploaded as expected:
http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android/1381270935/geckoview_assets.zip
http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android/1381270935/geckoview_library.zip

It seems that the mozharness scripts get confused and tried to download the crashreporter-symbols.zip for them:
http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android/1381270935/geckoview_assets.crashreporter-symbols.zip

I think the problem is the way the sendchanges are constructed:
python /builds/slave/m-in-and-000000000000000000000/tools/buildfarm/utils/retry.py -s 5 -t 1800 -r 5 --stdout-regexp 'change sent successfully' buildbot sendchange --master buildbot-master81.build.mozilla.org:9301 --username sendchange-unittest --branch mozilla-inbound-android-opt-unittest --revision 8a4946c07c92 --comments 'bug 922145 - publish nightly builds of GeckoView library and assets r=ted' --property buildid:20131008152215 --property pgo_build:False --property builduid:86cf43a4ba21411a93bd6fc49336a3ef http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android/1381270935/geckoview_assets.zip http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android/1381270935/fennec-27.0a1.en-US.android-arm.tests.zip

It specifies this:
geckoview_assets.zip
when it should have specified this:
fennec-27.0a1.en-US.android-arm.apk
Attached patch [untested] ignore geckoview (obsolete) — Splinter Review
This is untested.

I think this is the fix.
Attachment #815039 - Flags: feedback?(aki)
aki, would you like me to test it?

(I so wish testing these type of changes were easier to test)
Assignee: blassey.bugs → armenzg
Comment on attachment 815039 [details] [diff] [review]
[untested] ignore geckoview

I'm pretty sure this will do the trick.
Attachment #815039 - Flags: feedback?(aki) → review+
Comment on attachment 815039 [details] [diff] [review]
[untested] ignore geckoview

https://hg.mozilla.org/build/buildbotcustom/rev/717dccdcb472

I will push this live tomorrow morning and give the bug back to blassey to land his patch.
I will also do a push to try before giving the bug back just to verify that it works.
Attachment #815039 - Flags: checkin+
The code is now live.
I tried pushing it to try, however, my pushes to try are not going through.
Instead I decided to re-trigger the Android build jobs for revision 8a4946c07c92.
We should see in an hour or so the builds complete and hopefully not see the test jobs fail in less than 10 minutes.
I don't understand. This should have worked.
I will have to go to staging.

[1] https://tbpl.mozilla.org/php/getParsedLog.php?id=28935824&tree=Mozilla-Inbound&full=1
My new fix did not make the cut.
I'm now testing this:

diff --git a/process/factory.py b/process/factory.py
--- a/process/factory.py
+++ b/process/factory.py
@@ -225,17 +225,17 @@ def parse_make_upload(rc, stdout, stderr
         elif m.endswith('apk') and 'robocop' in m:
             retval['robocopApkUrl'] = m
         elif 'jsshell-' in m and m.endswith('.zip'):
             retval['jsshellUrl'] = m
         elif m.endswith('.complete.mar'):
             retval['completeMarUrl'] = m
         elif m.endswith('.mar') and '.partial.' in m:
             retval['partialMarUrl'] = m
-        elif m.startswith('geckoview'):
+        elif m.find('geckoview') >= 0:
             pass
         else:
             retval['packageUrl'] = m
     return retval
 
 
 def short_hash(rc, stdout, stderr):
     ''' This function takes an hg changeset id and returns just the first 12 chars'''
Attached patch geckoview.diffSplinter Review
The right properties get set:
http://dev-master01.build.scl1.mozilla.com:8040/builders/Android%202.2%20mozilla-inbound%20build/builds/2

The right sendchange is created:
buildbot sendchange --master dev-master01.build.scl1.mozilla.com:9901 --username sendchange-unittest --branch mozilla-inbound-android-opt-unittest --revision 8a4946c07c92 --property buildid:20131010155817 --property pgo_build:False --property builduid:26f7446c38fd4cdb85d7a1e085d3f6ce http://dev-stage01.srv.releng.scl3.mozilla.com/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android/1381445897/fennec-27.0a1.en-US.android-arm.apk http://dev-stage01.srv.releng.scl3.mozilla.com/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android/1381445897/fennec-27.0a1.en-US.android-arm.tests.zip
Attachment #815039 - Attachment is obsolete: true
Attachment #815680 - Flags: review?(aki)
Attachment #815680 - Flags: review?(aki) → review+
(In reply to Brad Lassey [:blassey] (use needinfo?) from comment #3)
> This code is on central. Essentially we want to make sure that the contents
> of dist/geckoview_library/ end up on ftp. currently that is
> geckoview_assets.zip and geckoview_library.zip.

We should probably investigate the possibility of shipping a single aar (Android Archive) containing both assets and library instead.
Attachment #815680 - Flags: checkin+
This is live now.

I pushed a build to try:
https://tbpl.mozilla.org/?tree=Try&rev=13eed7b55142

blassey: this should be good to good.
You can wait on try if you want to be safe, however, I believe we're good.
Assignee: armenzg → blassey.bugs
Builds take a while.
The try build and test have completed successfully.
whooo! Thanks a ton!
pushed to inbound https://hg.mozilla.org/integration/mozilla-inbound/rev/c2e371d16e65

(In reply to Lucas Rocha (:lucasr) from comment #19)
> (In reply to Brad Lassey [:blassey] (use needinfo?) from comment #3)
> > This code is on central. Essentially we want to make sure that the contents
> > of dist/geckoview_library/ end up on ftp. currently that is
> > geckoview_assets.zip and geckoview_library.zip.
> 
> We should probably investigate the possibility of shipping a single aar
> (Android Archive) containing both assets and library instead.

Sounds good, file a bug for that?
https://hg.mozilla.org/mozilla-central/rev/c2e371d16e65
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 27
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.