verify signature on Android apks

RESOLVED FIXED

Status

Release Engineering
General
P3
normal
RESOLVED FIXED
8 years ago
5 years ago

People

(Reporter: aki, Assigned: mjessome)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [signing][automation][android][releases])

Attachments

(6 attachments, 8 obsolete attachments)

3.97 KB, patch
lsblakk
: review+
lsblakk
: checked-in+
Details | Diff | Splinter Review
1.61 KB, patch
lsblakk
: review+
lsblakk
: checked-in+
Details | Diff | Splinter Review
1.29 KB, patch
lsblakk
: review+
lsblakk
: checked-in+
Details | Diff | Splinter Review
4.01 KB, patch
lsblakk
: review+
lsblakk
: checked-in+
Details | Diff | Splinter Review
2.23 KB, patch
lsblakk
: review+
lsblakk
: checked-in+
Details | Diff | Splinter Review
3.69 KB, patch
lsblakk
: review+
lsblakk
: checked-in+
Details | Diff | Splinter Review
(Reporter)

Description

8 years ago
Currently the only known way to verify an Android apk signature is

a) install a known previous apk, with a signature from the desired key, onto an Android device
b) attempt to install the apk to test onto the same device without uninstalling first.

It would be good to be able to verify the signature without actual installation (or multiple installation, as it were).

Then, automate it.  For release builds, definitely; for nightly builds quite possibly.
(Reporter)

Updated

8 years ago
Blocks: 478420
(Reporter)

Updated

8 years ago
Whiteboard: [signing][automation][android][releases]
Welcome to RelEng, here's your first real bug! :-)
Assignee: nobody → mjessome
Note: when this lands, a comment should be made in bug 557260 to remind John Ford that this step will want to be included in his work there.
(Assignee)

Comment 3

7 years ago
Created attachment 531085 [details] [diff] [review]
Add android signature verification steps.

Adds a bash script which will perform the verification step, as well as 3 .sig files which store the correct signature information for staging, nightly, and release.
(Assignee)

Comment 4

7 years ago
Created attachment 531090 [details] [diff] [review]
Add android signature verification steps to factory.

This will add the step to the buidbotcustom factory to verify the android signature. Currently hard coded to use the nightly signature.
Attachment #531085 - Flags: review?(lsblakk)
Attachment #531085 - Flags: review?(lsblakk)
(Assignee)

Comment 5

7 years ago
Created attachment 531115 [details] [diff] [review]
Add android signature verification steps to factory, correction.

Add android signature verification steps to buildbotcustom factory.
This patch is to correct the incorrectly attached 531090.
Attachment #531090 - Attachment is obsolete: true
Attachment #531115 - Flags: review?(lsblakk)
(Assignee)

Comment 6

7 years ago
Created attachment 531118 [details] [diff] [review]
Add android signature verification tool.

Adds a bash script which will perform the verification step, as well as 3 .sig files which store the correct signature information for staging, nightly, and release.
Attachment #531085 - Attachment is obsolete: true
Attachment #531118 - Flags: review?(lsblakk)
Comment on attachment 531118 [details] [diff] [review]
Add android signature verification tool.


>+#Pars arguments

typo nit: should be "Parse" r+ with that change.

Otherwise, looks good.
Attachment #531118 - Flags: review?(lsblakk) → review+
(Assignee)

Comment 8

7 years ago
Created attachment 531145 [details] [diff] [review]
Android signature verification tool

Removed unused args variable, fixed "Parse" type.
Attachment #531118 - Attachment is obsolete: true
Attachment #531145 - Flags: review?(lsblakk)
Comment on attachment 531115 [details] [diff] [review]
Add android signature verification steps to factory, correction.


>+        workdir='%s/%s/%s' % (self.baseWorkDir, self.branchName, self.objdir)

Not using this, so don't need it

>+                 WithProperties('%(toolsdir)s/release/signing/verify-android-signature.sh --apk=dist/%(completeMarFilename)s --tools-dir=%(toolsdir)s --staging')],

set to --nightly instead of --staging

>+            haltOnFailure=True,
>+        )

for now, let's set this to False so that staging runs don't break on this and also the staging sig file can be removed from the tools patch.
Attachment #531115 - Flags: review?(lsblakk) → review-
(Assignee)

Comment 10

7 years ago
Created attachment 531172 [details] [diff] [review]
Verify android signature factory step

Removed unused workingdir variable, haltOnFailure set to False, changed --staging to --nightly.
Attachment #531115 - Attachment is obsolete: true
Attachment #531172 - Flags: review?(lsblakk)
(Assignee)

Comment 11

7 years ago
Created attachment 531180 [details] [diff] [review]
Android signature verification tool no staging

Removed staging from the android verification tool
Attachment #531145 - Attachment is obsolete: true
Attachment #531180 - Flags: review?(lsblakk)
Attachment #531145 - Flags: review?(lsblakk)
(Assignee)

Comment 12

7 years ago
Created attachment 531181 [details] [diff] [review]
Verify android signature factory step

Set haltOnFailure back to True.
Attachment #531172 - Attachment is obsolete: true
Attachment #531181 - Flags: review?(lsblakk)
Attachment #531172 - Flags: review?(lsblakk)
Comment on attachment 531180 [details] [diff] [review]
Android signature verification tool no staging

looks great. i'll land this tonight and it will go into production tomorrow morning during the usual Tuesday reconfig.
Attachment #531180 - Flags: review?(lsblakk) → review+
Comment on attachment 531181 [details] [diff] [review]
Verify android signature factory step

looks good, i'll land this now and it will be in tomorrow's reconfig.
Attachment #531181 - Flags: review?(lsblakk) → review+
Comment on attachment 531180 [details] [diff] [review]
Android signature verification tool no staging

http://hg.mozilla.org/build/tools
Attachment #531180 - Flags: checked-in+
(Reporter)

Comment 17

7 years ago
bash: /builds/slave/aurora-mob-andrd-r7-ntly/tools/release/signing/verify-android-signature.sh: Permission denied
Today's Android nightly had verify signature step run on it and all is well, half this bug is solved.

Now we just need a builder on the 0.7 release that will wait for the signed android builds to show up in the candidates dir and run the apk verification.


create a scheduler like this http://hg.mozilla.org/build/buildbot-configs/file/22920666a9b3/mozilla2/release_master.py#l38 using FtpPoller (http://mxr.mozilla.org/build/source/buildbotcustom/changes/ftppoller.py) that will keep an eye on a dir like http://stage.mozilla.org/pub/mozilla.org/mobile/candidates/4.0b2-candidates/build1/android-r7/ -- we create the android-r7 dir before doing the signing, once the en-US and multi dirs show up in there the builds are available (1 in each dir) for signature verification. We'll also need a signature verification factory in buildbotcustom/process/factory to run the script you wrote and send it the relevant config settings for the particular release that's being run in order to point to the right ftp dir.
(Assignee)

Comment 20

7 years ago
Created attachment 532000 [details] [diff] [review]
Add url handling to verify-android-signature tool

Checks for url by "://" sub-string (for file://, http://, https://, ftp://, etc.)
Attachment #532000 - Flags: review?(lsblakk)
(Assignee)

Comment 21

7 years ago
Created attachment 534023 [details] [diff] [review]
Add a modified script factory to production-0.7

This adds a modified version of the 0.8 script factory to 0.7; Note that it should not be ported to 0.8.
Attachment #534023 - Flags: review?(lsblakk)
(Assignee)

Comment 22

7 years ago
Created attachment 534025 [details] [diff] [review]
Android Verification Scheduler

To schedule android verification on releases.
Note: uses the modified script factory for 0.7 and should not be ported to 0.8, but will be easy to implement on 0.8 when necessary.
Attachment #534025 - Flags: review?(lsblakk)
Comment on attachment 532000 [details] [diff] [review]
Add url handling to verify-android-signature tool

looks good.
Attachment #532000 - Flags: review?(lsblakk) → review+
Comment on attachment 534023 [details] [diff] [review]
Add a modified script factory to production-0.7


>+class ScriptFactory(BuildFactory):
>+    def __init__(self, scriptRepo, scriptName, cwd=None, interpreter=None,
>+         extra_data=None, extra_args=None,

>+ 
>+         envJava = {}
>+         envJava['PATH'] = '/tools/jdk6/bin:%s' % envJava.get('PATH', '/opt/local/bin:/tools/python/bin:/tools/buildbot/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/cltbld/bin')
>+         self.addStep(ShellCommand(name="run_script",
>+             command=[interpreter, WithProperties(scriptName)],
>+             timeout=script_timeout, maxTime=script_maxtime,
>+             workdir=".",
>+             haltOnFailure=True,
>+             env=envJava,
>+             warnOnWarnings=True))

instead of setting envJava in here, pass it into the factory from your release_mobile_master as extra_args['env'] so that the ScriptFactory can use different envs depending on what's passed in, an empty env = {} if not.
Attachment #534023 - Flags: review?(lsblakk) → review-
(Assignee)

Comment 25

7 years ago
Created attachment 534041 [details] [diff] [review]
Android Verification Scheduler v2

Schedule android verification on releases.
Note: uses the modified script factory for 0.7 and should not be ported to 0.8, but will be easy to implement on 0.8 when necessary.

v2: moved envJava to builder creation, rather than in ScriptFactory.
Attachment #534025 - Attachment is obsolete: true
Attachment #534041 - Flags: review?(lsblakk)
Attachment #534025 - Flags: review?(lsblakk)
(Assignee)

Comment 26

7 years ago
Created attachment 534042 [details] [diff] [review]
Add a modified script factory to production-0.7 v2

This adds a modified version of the 0.8 script factory to 0.7; Note that it should not be ported to 0.8.

v2: add env variable, removing the envJava setting in ScriptFactory.
Attachment #534023 - Attachment is obsolete: true
(Assignee)

Comment 27

7 years ago
Comment on attachment 534042 [details] [diff] [review]
Add a modified script factory to production-0.7 v2

>diff --git a/process/factory.py b/process/factory.py
>--- a/process/factory.py
>+++ b/process/factory.py
>@@ -7851,8 +7851,56 @@ class AndroidReleaseBuildFactory(Android
>                                self.objdir),
>          extract_fn = parse_make_upload,
>          haltOnFailure=True,
>          description=['upload'],
>          timeout=60*60 # 60 minutes
>         )
>         if self.createSnippet and uploadSnippet:
>             self._uploadSnippet()
>+
>+class ScriptFactory(BuildFactory):
>+    def __init__(self, scriptRepo, scriptName, cwd=None, interpreter=None,
>+         env=None, extra_data=None, extra_args=None,
>+         script_timeout=1200, script_maxtime=None):
>+
>+         BuildFactory.__init__(self)
>+         self.addStep(SetBuildProperty(
>+             property_name='master',
>+             value=lambda b: b.builder.botmaster.parent.buildbotURL
>+         ))
>+         self.addStep(ShellCommand(
>+             name="clobber_scripts",
>+             command=['rm', '-rf', 'scripts'],
>+             workdir=".",
>+         ))
>+         self.addStep(ShellCommand(
>+             name="clone_scripts",
>+             command=['hg', 'clone', scriptRepo, 'scripts'],
>+             workdir=".",
>+             haltOnFailure=True))
>+         self.addStep(ShellCommand(
>+             name="update_scripts",
>+             command=['hg', 'update', '-C', '-r',
>+                      WithProperties('%(script_repo_revision:-default)s')],
>+             haltOnFailure=True,
>+             workdir='scripts'
>+         ))
>+         self.addStep(SetBuildProperty,
>+              name='set_who',
>+              property_name='who',
>+              value=lambda build:str(build.source.changes[0].who),
>+              haltOnFailure=True
>+         )
>+         self.addStep(SetBuildProperty,
>+              name='set_locale',
>+              property_name='locale',
>+              value=lambda build:str(build.source.changes[0].who.split('/')[-2]),
>+              haltOnFailure=True
>+         )
>+ 
>+         self.addStep(ShellCommand(name="run_script",
>+             command=[interpreter, WithProperties(scriptName)],
>+             timeout=script_timeout, maxTime=script_maxtime,
>+             workdir=".",
>+             haltOnFailure=True,
>+             env=env,
>+             warnOnWarnings=True))
Attachment #534042 - Flags: review?(lsblakk)
Attachment #534041 - Flags: review?(lsblakk) → review+
Comment on attachment 534042 [details] [diff] [review]
Add a modified script factory to production-0.7 v2

Awesome - great work. I'll land this today.
Attachment #534042 - Flags: review?(lsblakk) → review+
Comment on attachment 532000 [details] [diff] [review]
Add url handling to verify-android-signature tool

http://hg.mozilla.org/build/tools/rev/47ce531e5de1
Attachment #532000 - Flags: checked-in+
Comment on attachment 534041 [details] [diff] [review]
Android Verification Scheduler v2

Just need this patch but for mozilla2/release_mobile_master.py as the staging one is landed, then this bug can be closed.
(Assignee)

Comment 33

7 years ago
Created attachment 534107 [details] [diff] [review]
Scheduler for mozilla2

The same scheduler from bug 534041 but for Mozilla2.
Attachment #534107 - Flags: review?(lsblakk)
Attachment #534107 - Flags: review?(lsblakk) → review+
Comment on attachment 534107 [details] [diff] [review]
Scheduler for mozilla2

http://hg.mozilla.org/build/buildbot-configs/rev/6fa2a8b0d677

landed on default - these will get picked up in production on the next reconfig, and will get used in the next 0.7 release (Fennec 5.0b3?)
Attachment #534107 - Flags: checked-in+
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.