Update buildapi with knowledge of split APK builds

RESOLVED FIXED

Status

Release Engineering
General
P2
normal
RESOLVED FIXED
4 years ago
a year ago

People

(Reporter: coop, Assigned: coop)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 1 obsolete attachment)

(Assignee)

Description

4 years ago
buildapi doesn't know about the recent change to Android builds to create split APKs. This could affect pending counts and wait times emails.

There's also a lot of out-of-date cruft in buildapi/model/util.py that should be cleaned up
(Assignee)

Comment 1

4 years ago
Created attachment 8540922 [details] [diff] [review]
Cleanup cruft in util.py; learn about split APK Android builds

These are the same match strings we're using in slave health.
Attachment #8540922 - Flags: review?(jlund)

Comment 2

4 years ago
Comment on attachment 8540922 [details] [diff] [review]
Cleanup cruft in util.py; learn about split APK Android builds

Review of attachment 8540922 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm - r+ if you either add pre-split builder patterns to panda or tell me why it's not needed (see in-line)

I am largely looking at split apk stuff. the clean up looks good too but I did not cross reference each one against our infra. If you need a thorough sanity check, please clear this r+ and re-request this patch so it gets put back in my queue. :D

::: buildapi/model/util.py
@@ +90,1 @@
>      ],

apk-split is only on trunk. will this mean that the slave platform 'panda-android' won't pick up any of the pre-split builders? iow - should we support the old '^Android 4\.0 Panda .+' as well?
Attachment #8540922 - Flags: review?(jlund) → review+

Comment 3

4 years ago
> ::: buildapi/model/util.py
> @@ +90,1 @@
> >      ],

bah, I thought it would show more than one line. Here is more context:

>     'panda-android': [
> -        re.compile('^Android 4\.0 Panda .+'),
> +        re.compile('(Panda|panda|Android 4.0 armv7 API (10|11)\+)'),
>      ],
(Assignee)

Comment 4

4 years ago
(In reply to Jordan Lund (:jlund) from comment #3)
> > ::: buildapi/model/util.py
> > @@ +90,1 @@
> > >      ],
> 
> bah, I thought it would show more than one line. Here is more context:
> 
> >     'panda-android': [
> > -        re.compile('^Android 4\.0 Panda .+'),
> > +        re.compile('(Panda|panda|Android 4.0 armv7 API (10|11)\+)'),
> >      ],

The (Panda|panda) bit will catch the old builds.
(Assignee)

Comment 5

4 years ago
Comment on attachment 8540922 [details] [diff] [review]
Cleanup cruft in util.py; learn about split APK Android builds

Review of attachment 8540922 [details] [diff] [review]:
-----------------------------------------------------------------

https://hg.mozilla.org/build/buildapi/rev/c0d4c72885a1
Attachment #8540922 - Flags: checked-in+
(Assignee)

Comment 6

4 years ago
Deployed.

I ran the wait times emails, and we're closer now, but not perfect. We still miss:

* Android 4.0 Panda (I guess it doesn't like the ORing as JS does, mea culpa)
* Windows 8 tests
* mulet builds

Follow-up patch incoming.
(Assignee)

Comment 7

4 years ago
Created attachment 8542624 [details] [diff] [review]
Catch old-style panda tests, mulet builds, and Windows 8 tests.
Attachment #8542624 - Flags: review?(jlund)
(Assignee)

Comment 8

4 years ago
Created attachment 8542630 [details] [diff] [review]
Catch old-style panda tests, mulet builds, and Windows 8 tests.

Same patch as previous, but eliminates the win64 bucket completely. AFAICT the only builds that match these are fuzzer jobs that run on the win2k8 machines.
Attachment #8542624 - Attachment is obsolete: true
Attachment #8542624 - Flags: review?(jlund)
Attachment #8542630 - Flags: review?(jlund)

Comment 9

4 years ago
Comment on attachment 8542630 [details] [diff] [review]
Catch old-style panda tests, mulet builds, and Windows 8 tests.

Review of attachment 8542630 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm :)

do we need a version bump in setup.py?

::: buildapi/model/util.py
@@ +57,5 @@
>          re.compile('^Rev4 MacOSX Snow Leopard 10\.6.+'),
>      ],
>      'lion': [
>          re.compile('^OS X 10\.7.+'),
> +        re.compile('^OS X Mulet'),

I'm assuming the *win32* and *linux* patterns handle the Mulet builds fine and it's just osx that is the funny edge case.

@@ -87,3 @@
>      ],
>      'panda-android': [
> -        re.compile('(Panda|panda|Android 4.0 armv7 API (10|11)\+)'),

this confused me so I played a bit. IIUC, I think it is because we use pattern.match() and not pattern.search() for the regex: http://mxr.mozilla.org/build/source/buildapi/buildapi/model/util.py#304

iow - pattern.match(string), assumes the initial chars in string must be 'panda' or 'Panda'

I think we could do something like: '(.*Panda.*|.*panda.*|Android 4.0 armv7 API (10|11)\+)' though to solve that issue

::: setup.cfg
@@ +1,3 @@
>  [egg_info]
> +#tag_build = dev
> +#tag_svn_revision = true

this wasn't changed before, why are we commenting this out now?
Attachment #8542630 - Flags: review?(jlund) → review+
(Assignee)

Comment 10

4 years ago
(In reply to Jordan Lund (:jlund) from comment #9) 
> ::: buildapi/model/util.py
> @@ +57,5 @@
> >          re.compile('^Rev4 MacOSX Snow Leopard 10\.6.+'),
> >      ],
> >      'lion': [
> >          re.compile('^OS X 10\.7.+'),
> > +        re.compile('^OS X Mulet'),
> 
> I'm assuming the *win32* and *linux* patterns handle the Mulet builds fine
> and it's just osx that is the funny edge case.

Correct.

> @@ -87,3 @@
> >      ],
> >      'panda-android': [
> > -        re.compile('(Panda|panda|Android 4.0 armv7 API (10|11)\+)'),
> 
> this confused me so I played a bit. IIUC, I think it is because we use
> pattern.match() and not pattern.search() for the regex:
> http://mxr.mozilla.org/build/source/buildapi/buildapi/model/util.py#304
> 
> iow - pattern.match(string), assumes the initial chars in string must be
> 'panda' or 'Panda'
> 
> I think we could do something like: '(.*Panda.*|.*panda.*|Android 4.0 armv7
> API (10|11)\+)' though to solve that issue

I'll just land the separate line with ('.*panda.*', re.IGNORECASE) and avoid an overly-complicated single regexp.

> do we need a version bump in setup.py?
>
> ::: setup.cfg
> @@ +1,3 @@
> >  [egg_info]
> > +#tag_build = dev
> > +#tag_svn_revision = true
> 
> this wasn't changed before, why are we commenting this out now?

Oops, i excluded the .py file instead of the .cfg file. Those lines get commented out in the .cfg file to generate the dist package with no rev attached, but it's not a change that should land.
(Assignee)

Comment 11

4 years ago
Comment on attachment 8542630 [details] [diff] [review]
Catch old-style panda tests, mulet builds, and Windows 8 tests.

Review of attachment 8542630 [details] [diff] [review]:
-----------------------------------------------------------------

https://hg.mozilla.org/build/buildapi/rev/bd1a9390b47d
Attachment #8542630 - Flags: checked-in+
(Assignee)

Updated

4 years ago
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Component: Tools → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.