Closed Bug 1461303 Opened 4 years ago Closed 4 years ago

Beetmoverworkers updates for scriptworker 11.0.0

Categories

(Release Engineering :: Release Automation: Uploading, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: mtabara, Assigned: lguo)

References

Details

Attachments

(5 files)

Scriptworker 11.0.0 is shipped! This means we now support [most likely require] aiohttp>=3, and we've dropped support for py35.

Specifically, we need to update beetmoverworkers to:
- require py36; drop py35 support
- test against py36
- support aiohttp3; there are open PRs against the ones that need fixing
Similar approaches in other *scripts were done under shipitscript[1] and other scripts.

[1]: https://github.com/mozilla-releng/shipitscript/pull/6
Lisa agreed to work on this so reassigning this to her!
Assignee: nobody → lguo
Attachment #8976584 - Flags: review?(aki)
Attachment #8976584 - Flags: review?(aki) → review+
Comment on attachment 8976981 [details] [diff] [review]
update beetmoverworker puppet requirements file

Looks good! We probably want a PR in the beetmoverscript repo to bump requirements-prod.txt as well.

I can land this on Monday.
Attachment #8976981 - Flags: review?(aki) → review+
Pushed by asasaki@mozilla.com:
https://hg.mozilla.org/build/puppet/rev/ed1affbb2408
update beetmover deps. r=aki
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Hm, I see https://hg.mozilla.org/build/puppet/rev/f78fdc9188b1 landed in production. Do we need to land anything in-tree or the only leftover is the requirements-prod.txt?
Component: Release Automation: Updates → Release Automation: Uploading
Attachment #8979650 - Flags: review?(aki) → review+
Pushed by asasaki@mozilla.com:
https://hg.mozilla.org/build/puppet/rev/9a536661a1d7
bump bouncer scriptworker deps. r=aki
Comment on attachment 8979695 [details] [diff] [review]
balrog_puppet.diff

Review of attachment 8979695 [details] [diff] [review]:
-----------------------------------------------------------------

Two quick questions: 
1. Out of curiosity, how do we ensure puppet doesn't break for any possible missing package? Or have we already checked all these packages are present within our internal pypi when we rolled-out scriptworker which most likely introduced most of these?
2. Would it be a good time now to roll-out beetmoverscript bump too since we're here? In order to pick-up lisa's work from last 2 weeks - https://github.com/mozilla-releng/beetmoverscript/commits/master. hm, actually not blocking though, we can roll that out anytime.

::: modules/balrog_scriptworker/files/requirements-27.txt
@@ +24,5 @@
> +pycparser==2.18
> +python-dateutil==2.6.1
> +requests==2.18.4
> +toml==0.9.4
> +towncrier==18.5.0

Not sure why this is needed for production but it certainly can't hurt.
Attachment #8979695 - Flags: review?(mtabara) → review+
See Also: → 1463519
(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #15)
> Two quick questions: 
> 1. Out of curiosity, how do we ensure puppet doesn't break for any possible
> missing package? Or have we already checked all these packages are present
> within our internal pypi when we rolled-out scriptworker which most likely
> introduced most of these?

I generally

- create a file with all the new deps listed, e.g. `new`
mkdir download
cd download
pip download --no-deps -r ../new  # make sure to use the right pip,
                                  # needs to be py36 or py27 depending on which req file we're working against
# make sure i didn't download any mac-only wheels; if I did, I manually download the manylinux wheel instead
cd ..
scp -r download releng-puppet2.srv.releng.scl3.mozilla.com:
ssh releng-puppet2.srv.releng.scl3.mozilla.com
cd /data/python/packages-3.x
sudo mv -n ~/download/* .  # use -n to not overwrite existing
ls -lrt  # doublecheck
cd
rm -rf download  # cleanup for next time

> 2. Would it be a good time now to roll-out beetmoverscript bump too since
> we're here? In order to pick-up lisa's work from last 2 weeks -
> https://github.com/mozilla-releng/beetmoverscript/commits/master. hm,
> actually not blocking though, we can roll that out anytime.

Beetmoverscript and bouncerscript bumps are already rolled out.

> ::: modules/balrog_scriptworker/files/requirements-27.txt
> @@ +24,5 @@
> > +pycparser==2.18
> > +python-dateutil==2.6.1
> > +requests==2.18.4
> > +toml==0.9.4
> > +towncrier==18.5.0
> 
> Not sure why this is needed for production but it certainly can't hurt.

It's in requirements-dev.txt, so it shows up here. https://github.com/mozilla-releng/balrogscript/blob/master/requirements-dev.txt
Pushed by asasaki@mozilla.com:
https://hg.mozilla.org/build/puppet/rev/30ce934c839a
update balrog scriptworker deps. r=mtabara
@aki: thanks for providing all the useful detail from comment 16, that makes sense.
I think we can close this, I see the rollout is now complete - https://hg.mozilla.org/build/puppet/file/tip/modules/beetmover_scriptworker/files/requirements.txt

Feel free to reopen if I missed anything.
Thanks!
Status: REOPENED → RESOLVED
Closed: 4 years ago4 years ago
Resolution: --- → FIXED
Blocks: 1442684
You need to log in before you can comment on or make changes to this bug.