Closed Bug 894493 Opened 9 years ago Closed 9 years ago

use peep for installing dependencies

Categories

(Socorro :: General, task)

x86
macOS
task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: lonnen, Assigned: rhelmer)

References

Details

(Whiteboard: [qa-])

ErikRose's peep adds an extra layer of assurance at little cost. It's backwards compatible for devs installing from pip, but our CI can use it to verify the sha of the packages we're pulling down.

https://pypi.python.org/pypi/peep/
Spent a little time on this:
https://github.com/rhelmer/socorro/compare/bug894493-use-peep-for-deps

Since we can't trust peep to verify itself, I've simply checked in the latest version to ./tools/ and use pip to install it. From there, we use peep and SHA 256 hashes to verify packages from pypi.

Two problems right now:
 
1) peep throws an exception when it tries to print info for sqlalchemy-citext (non-standard filename, contains more than one "-")
2) configman needs to come from pypi not git

We could work around #1 (only happens if there's no hash set up yet, want it fixed since it makes upgrading a pain but could ship with it I think).

We don't have a good workaround for #2 - :peterbe can we get the right version of configman up on pypi?
Assignee: nobody → rhelmer
Status: NEW → RESOLVED
Closed: 9 years ago
Flags: needinfo?(peterbe)
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
#1 is fixed in peep 0.4.
r? anybody - https://github.com/mozilla/socorro/pull/1457

Note that we check peep in and install it from there because we the whole point here is to not trust pypi, and we can't trust a version of peep that we download from there to verify itself ;)

We *could* download peep from pypi and use a different tool, but I don't really see the point since peep is just a tiny wrapper around pypi, anyway.
(In reply to Robert Helmer [:rhelmer] from comment #3)
> r? anybody - https://github.com/mozilla/socorro/pull/1457
> 
> Note that we check peep in and install it from there because we the whole
> point here is to not trust pypi, and we can't trust a version of peep that
> we download from there to verify itself ;)
> 
> We *could* download peep from pypi and use a different tool, but I don't
> really see the point since peep is just a tiny wrapper around pypi, anyway.

Er, "peep is just a tiny wrapper around pip, anyway" imts
Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/8ba14b7776dabcad0ade1bbbf6e7f55c0722b561
bug 894493 - use peep for dependencies

https://github.com/mozilla/socorro/commit/2da829f054b85184604ea99b97dac0af34c21e31
Merge pull request #1457 from rhelmer/bug894493-use-peep-for-deps

[DO NOT MERGE] bug 894493 - use peep for dependencies
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
Flags: needinfo?(peterbe)
Target Milestone: --- → 58
Although this worked fine on leeroy, getting this test failure on socorro-release:

FAIL: test_get_count_by_day (socorro.unittest.external.postgresql.test_crashes.IntegrationTestCrashes)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/jenkins/jobs/socorro-release/workspace/socorro/unittest/external/postgresql/test_crashes.py", line 634, in test_get_count_by_day
    self.assertEquals(res, expected)
AssertionError: {'hits': {'2013-08-29': 2L}, 'total': 1} != {'hits': {'2013-08-29': 3}, 'total': 1}
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Going to try wiping the workspace before backing out
OK leeroy's failing on the backout patch! I don't even:

https://ci.mozilla.org/job/socorro-github/1514/console
I can't reproduce locally. CC'ing Shuhao as he wrote this unit test.
Found the issue. Is on https://github.com/mozilla/socorro/blob/master/socorro/unittest/external/postgresql/test_crashes.py#L615

That now is sometimes one day behind UTC now, which results in a 2 instead of 3. Fix coming soon.
Depends on: 911284
Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/03c688b767c3a25f4c300b340b6c70b1c055c9ef
fixes bug 894493 - build requirements against python 2.6

https://github.com/mozilla/socorro/commit/e3a78d1b9d4fb1ef2662e0e38db74f6c4d7c66eb
Merge pull request #1460 from rhelmer/bug894493-2.6-deps

fixes bug 894493 - build requirements against python 2.6
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
Hooray! :-D
Whiteboard: [qa-]
You need to log in before you can comment on or make changes to this bug.