Closed Bug 603826 Opened 10 years ago Closed 9 years ago

verify signature on Android apks

Categories

(Release Engineering :: General, defect, P3)

ARM
Android
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: aki, Assigned: mjessome)

References

Details

(Whiteboard: [signing][automation][android][releases])

Attachments

(6 files, 8 obsolete files)

3.97 KB, patch
lsblakk
: review+
lsblakk
: checked-in+
Details | Diff | Splinter Review
1.61 KB, patch
lsblakk
: review+
lsblakk
: checked-in+
Details | Diff | Splinter Review
1.29 KB, patch
lsblakk
: review+
lsblakk
: checked-in+
Details | Diff | Splinter Review
4.01 KB, patch
lsblakk
: review+
lsblakk
: checked-in+
Details | Diff | Splinter Review
2.23 KB, patch
lsblakk
: review+
lsblakk
: checked-in+
Details | Diff | Splinter Review
3.69 KB, patch
lsblakk
: review+
lsblakk
: checked-in+
Details | Diff | Splinter Review
Currently the only known way to verify an Android apk signature is

a) install a known previous apk, with a signature from the desired key, onto an Android device
b) attempt to install the apk to test onto the same device without uninstalling first.

It would be good to be able to verify the signature without actual installation (or multiple installation, as it were).

Then, automate it.  For release builds, definitely; for nightly builds quite possibly.
Blocks: 478420
Whiteboard: [signing][automation][android][releases]
Welcome to RelEng, here's your first real bug! :-)
Assignee: nobody → mjessome
Note: when this lands, a comment should be made in bug 557260 to remind John Ford that this step will want to be included in his work there.
Adds a bash script which will perform the verification step, as well as 3 .sig files which store the correct signature information for staging, nightly, and release.
This will add the step to the buidbotcustom factory to verify the android signature. Currently hard coded to use the nightly signature.
Attachment #531085 - Flags: review?(lsblakk)
Attachment #531085 - Flags: review?(lsblakk)
Add android signature verification steps to buildbotcustom factory.
This patch is to correct the incorrectly attached 531090.
Attachment #531090 - Attachment is obsolete: true
Attachment #531115 - Flags: review?(lsblakk)
Adds a bash script which will perform the verification step, as well as 3 .sig files which store the correct signature information for staging, nightly, and release.
Attachment #531085 - Attachment is obsolete: true
Attachment #531118 - Flags: review?(lsblakk)
Comment on attachment 531118 [details] [diff] [review]
Add android signature verification tool.


>+#Pars arguments

typo nit: should be "Parse" r+ with that change.

Otherwise, looks good.
Attachment #531118 - Flags: review?(lsblakk) → review+
Removed unused args variable, fixed "Parse" type.
Attachment #531118 - Attachment is obsolete: true
Attachment #531145 - Flags: review?(lsblakk)
Comment on attachment 531115 [details] [diff] [review]
Add android signature verification steps to factory, correction.


>+        workdir='%s/%s/%s' % (self.baseWorkDir, self.branchName, self.objdir)

Not using this, so don't need it

>+                 WithProperties('%(toolsdir)s/release/signing/verify-android-signature.sh --apk=dist/%(completeMarFilename)s --tools-dir=%(toolsdir)s --staging')],

set to --nightly instead of --staging

>+            haltOnFailure=True,
>+        )

for now, let's set this to False so that staging runs don't break on this and also the staging sig file can be removed from the tools patch.
Attachment #531115 - Flags: review?(lsblakk) → review-
Removed unused workingdir variable, haltOnFailure set to False, changed --staging to --nightly.
Attachment #531115 - Attachment is obsolete: true
Attachment #531172 - Flags: review?(lsblakk)
Removed staging from the android verification tool
Attachment #531145 - Attachment is obsolete: true
Attachment #531180 - Flags: review?(lsblakk)
Attachment #531145 - Flags: review?(lsblakk)
Set haltOnFailure back to True.
Attachment #531172 - Attachment is obsolete: true
Attachment #531181 - Flags: review?(lsblakk)
Attachment #531172 - Flags: review?(lsblakk)
Comment on attachment 531180 [details] [diff] [review]
Android signature verification tool no staging

looks great. i'll land this tonight and it will go into production tomorrow morning during the usual Tuesday reconfig.
Attachment #531180 - Flags: review?(lsblakk) → review+
Comment on attachment 531181 [details] [diff] [review]
Verify android signature factory step

looks good, i'll land this now and it will be in tomorrow's reconfig.
Attachment #531181 - Flags: review?(lsblakk) → review+
Comment on attachment 531180 [details] [diff] [review]
Android signature verification tool no staging

http://hg.mozilla.org/build/tools
Attachment #531180 - Flags: checked-in+
bash: /builds/slave/aurora-mob-andrd-r7-ntly/tools/release/signing/verify-android-signature.sh: Permission denied
Today's Android nightly had verify signature step run on it and all is well, half this bug is solved.

Now we just need a builder on the 0.7 release that will wait for the signed android builds to show up in the candidates dir and run the apk verification.


create a scheduler like this http://hg.mozilla.org/build/buildbot-configs/file/22920666a9b3/mozilla2/release_master.py#l38 using FtpPoller (http://mxr.mozilla.org/build/source/buildbotcustom/changes/ftppoller.py) that will keep an eye on a dir like http://stage.mozilla.org/pub/mozilla.org/mobile/candidates/4.0b2-candidates/build1/android-r7/ -- we create the android-r7 dir before doing the signing, once the en-US and multi dirs show up in there the builds are available (1 in each dir) for signature verification. We'll also need a signature verification factory in buildbotcustom/process/factory to run the script you wrote and send it the relevant config settings for the particular release that's being run in order to point to the right ftp dir.
Checks for url by "://" sub-string (for file://, http://, https://, ftp://, etc.)
Attachment #532000 - Flags: review?(lsblakk)
This adds a modified version of the 0.8 script factory to 0.7; Note that it should not be ported to 0.8.
Attachment #534023 - Flags: review?(lsblakk)
Attached patch Android Verification Scheduler (obsolete) — Splinter Review
To schedule android verification on releases.
Note: uses the modified script factory for 0.7 and should not be ported to 0.8, but will be easy to implement on 0.8 when necessary.
Attachment #534025 - Flags: review?(lsblakk)
Comment on attachment 532000 [details] [diff] [review]
Add url handling to verify-android-signature tool

looks good.
Attachment #532000 - Flags: review?(lsblakk) → review+
Comment on attachment 534023 [details] [diff] [review]
Add a modified script factory to production-0.7


>+class ScriptFactory(BuildFactory):
>+    def __init__(self, scriptRepo, scriptName, cwd=None, interpreter=None,
>+         extra_data=None, extra_args=None,

>+ 
>+         envJava = {}
>+         envJava['PATH'] = '/tools/jdk6/bin:%s' % envJava.get('PATH', '/opt/local/bin:/tools/python/bin:/tools/buildbot/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/cltbld/bin')
>+         self.addStep(ShellCommand(name="run_script",
>+             command=[interpreter, WithProperties(scriptName)],
>+             timeout=script_timeout, maxTime=script_maxtime,
>+             workdir=".",
>+             haltOnFailure=True,
>+             env=envJava,
>+             warnOnWarnings=True))

instead of setting envJava in here, pass it into the factory from your release_mobile_master as extra_args['env'] so that the ScriptFactory can use different envs depending on what's passed in, an empty env = {} if not.
Attachment #534023 - Flags: review?(lsblakk) → review-
Schedule android verification on releases.
Note: uses the modified script factory for 0.7 and should not be ported to 0.8, but will be easy to implement on 0.8 when necessary.

v2: moved envJava to builder creation, rather than in ScriptFactory.
Attachment #534025 - Attachment is obsolete: true
Attachment #534041 - Flags: review?(lsblakk)
Attachment #534025 - Flags: review?(lsblakk)
This adds a modified version of the 0.8 script factory to 0.7; Note that it should not be ported to 0.8.

v2: add env variable, removing the envJava setting in ScriptFactory.
Attachment #534023 - Attachment is obsolete: true
Comment on attachment 534042 [details] [diff] [review]
Add a modified script factory to production-0.7 v2

>diff --git a/process/factory.py b/process/factory.py
>--- a/process/factory.py
>+++ b/process/factory.py
>@@ -7851,8 +7851,56 @@ class AndroidReleaseBuildFactory(Android
>                                self.objdir),
>          extract_fn = parse_make_upload,
>          haltOnFailure=True,
>          description=['upload'],
>          timeout=60*60 # 60 minutes
>         )
>         if self.createSnippet and uploadSnippet:
>             self._uploadSnippet()
>+
>+class ScriptFactory(BuildFactory):
>+    def __init__(self, scriptRepo, scriptName, cwd=None, interpreter=None,
>+         env=None, extra_data=None, extra_args=None,
>+         script_timeout=1200, script_maxtime=None):
>+
>+         BuildFactory.__init__(self)
>+         self.addStep(SetBuildProperty(
>+             property_name='master',
>+             value=lambda b: b.builder.botmaster.parent.buildbotURL
>+         ))
>+         self.addStep(ShellCommand(
>+             name="clobber_scripts",
>+             command=['rm', '-rf', 'scripts'],
>+             workdir=".",
>+         ))
>+         self.addStep(ShellCommand(
>+             name="clone_scripts",
>+             command=['hg', 'clone', scriptRepo, 'scripts'],
>+             workdir=".",
>+             haltOnFailure=True))
>+         self.addStep(ShellCommand(
>+             name="update_scripts",
>+             command=['hg', 'update', '-C', '-r',
>+                      WithProperties('%(script_repo_revision:-default)s')],
>+             haltOnFailure=True,
>+             workdir='scripts'
>+         ))
>+         self.addStep(SetBuildProperty,
>+              name='set_who',
>+              property_name='who',
>+              value=lambda build:str(build.source.changes[0].who),
>+              haltOnFailure=True
>+         )
>+         self.addStep(SetBuildProperty,
>+              name='set_locale',
>+              property_name='locale',
>+              value=lambda build:str(build.source.changes[0].who.split('/')[-2]),
>+              haltOnFailure=True
>+         )
>+ 
>+         self.addStep(ShellCommand(name="run_script",
>+             command=[interpreter, WithProperties(scriptName)],
>+             timeout=script_timeout, maxTime=script_maxtime,
>+             workdir=".",
>+             haltOnFailure=True,
>+             env=env,
>+             warnOnWarnings=True))
Attachment #534042 - Flags: review?(lsblakk)
Attachment #534041 - Flags: review?(lsblakk) → review+
Comment on attachment 534042 [details] [diff] [review]
Add a modified script factory to production-0.7 v2

Awesome - great work. I'll land this today.
Attachment #534042 - Flags: review?(lsblakk) → review+
Comment on attachment 532000 [details] [diff] [review]
Add url handling to verify-android-signature tool

http://hg.mozilla.org/build/tools/rev/47ce531e5de1
Attachment #532000 - Flags: checked-in+
Comment on attachment 534041 [details] [diff] [review]
Android Verification Scheduler v2

Just need this patch but for mozilla2/release_mobile_master.py as the staging one is landed, then this bug can be closed.
The same scheduler from bug 534041 but for Mozilla2.
Attachment #534107 - Flags: review?(lsblakk)
Attachment #534107 - Flags: review?(lsblakk) → review+
Comment on attachment 534107 [details] [diff] [review]
Scheduler for mozilla2

http://hg.mozilla.org/build/buildbot-configs/rev/6fa2a8b0d677

landed on default - these will get picked up in production on the next reconfig, and will get used in the next 0.7 release (Fennec 5.0b3?)
Attachment #534107 - Flags: checked-in+
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.