823306 - update http://puppetagain.pub.build.mozilla.org/data/python/packages/ for todays talos

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Jeff Hammel]

The package index for talos + deps should be updated for modern
talos.  We need the following new versions:

* mozdevice-0.15 :
  http://pypi.python.org/packages/source/m/mozdevice/mozdevice-0.15.tar.gz#md5=70e8b39c80fad088b081b3cc50c838f8

  (We could bump talos to 0.18, the latest version, if we wanted)

* mozhttpd-0.5 :
  http://pypi.python.org/packages/source/m/mozhttpd/mozhttpd-0.5.tar.gz#md5=04c8ec430d4e0f2294179b4a28a5af14

* datazilla-1.2 :
  http://pypi.python.org/packages/source/d/datazilla/datazilla-1.2.tar.gz#md5=ae8daee92af8a66dca82693c7b987089

In addition, we need a new version of talos.  Currently, this is keyed
off of hg tag, so we could use
http://hg.mozilla.org/build/talos/archive/71f7f2ed08a7.tar.gz (albeit
we want to fix the datazilla package dep in talos, bug 823260, so the
revision will be different.  Or we could finally bump talos to 0.1.  I
have no strong opinions at this juncture, but my gut feeling is that
bumping to 0.1 is probably better.

Comment 1

[Aki Sasaki (not active)]

Jeff: Is it correct to assume that this shouldn't proceed until bug 823260 is resolved?

Comment 2

[Jeff Hammel]

(In reply to Aki Sasaki [:aki] from comment #1)
> Jeff: Is it correct to assume that this shouldn't proceed until bug 823260
> is resolved?

There's no reason not to update the dependencies (although there's no strong reason to do so either). But yes, we should wait on bug 823260 for the talos package (and also figure out if we want to name it after the version or....)

Comment 3

[Aki Sasaki (not active)]

Where are we on this?
Bug 823260 is resolved; are we going to update the version, or should I push the current revision?

If we take default, do we need to add mozcrash as well?

Comment 4

[Jeff Hammel]

The short answer is: I don't know.  Since I have no power to push talos packages, it mostly doesn't matter to me.  What I think should happen is a version bump of talos every time we deploy to m-c.  But I've suggested that before and it hasn't happened, nor are the versions up there versioned this way.

The new talos will need mozcrash when it is deployed.

I don't really have any great ideas on how to get people on the same page here. I'm open to suggestions

Comment 5

[Aki Sasaki (not active)]

Oops.

Comment 6

[Aki Sasaki (not active)]

Ok, I currently have

datazilla-1.2.tar.gz       mozhttpd-0.5.tar.gz
mozcrash-0.2.1.tar.gz      talos-766a17c47b0d.tar.gz
mozdevice-0.15.tar.gz

downloaded.

I'm going to wait til we have Cedar reset and have a green talos run before uploading, so we catch any pre-talos-bump bustage.

* IT resets cedar
* we reland talos.json for mozharness talos
* (hopefully nothing has to happen here)
* talos goes green
* upload the above 5 packages to puppetagain
* make sure talos is still green
* point talos.json to new talos tarball

Comment 7

[Aki Sasaki (not active)]

(In reply to Aki Sasaki [:aki] from comment #6)

X IT resets cedar
X we reland talos.json for mozharness talos
X (hopefully nothing has to happen here)
X talos goes green
X upload the above 5 packages to puppetagain
_ make sure talos is still green   <--- we are here
_ point talos.json to new talos tarball

https://tbpl.mozilla.org/?tree=Cedar&rev=3f4b43ae2be2

Comment 8

[Aki Sasaki (not active)]

X make sure talos is still green
X point talos.json to new talos tarball

http://hg.mozilla.org/projects/cedar/rev/01a70d37b5be
https://tbpl.mozilla.org/?tree=Cedar&rev=01a70d37b5be

Calling this fixed.

Comment 9

[Aki Sasaki (not active)]

Also had to upload mozlog 1.1

Comment 10

[Aki Sasaki (not active)]

Also had to remove sunspider from talos.json: https://hg.mozilla.org/projects/cedar/rev/993e66df0419

Backed out due to continuing bustage: https://hg.mozilla.org/projects/cedar/rev/e41c92fa3466

Comment 11

[Aki Sasaki (not active)]

Attachment: venv error -> warning

This won't fix the exception, but it'll stop highlighting the ignorable libyaml error.

Comment 12

[Aki Sasaki (not active)]

17:04:34     INFO -  tar: talos/breakpad/osx64: Cannot create symlink to `osx': No such file or directory
17:04:36    ERROR -  tar: Error exit delayed from previous errors
17:04:36    ERROR - Return code: 2
17:04:36    FATAL - Halting on failure while running c:/mozilla-build/msys/bin/bash -c "PATH=/c/mozilla-build/msys/bin:$PATH tar zx --strip-components=1 -f '/c/talos-slave/test/build/talos-766a17c47b0d.tar.gz' --wildcards '**/talos/'"
17:04:36    FATAL - Exiting 2

The osx64 -> osx softlink is responsible for at least some of the breakage.
I checked; there are 0 softlinks in the talos repo in revision 0e9224d7bc95.

Comment 13

[Aki Sasaki (not active)]

Attachment: fix talos softlink

cd talos/breakpad
hg rm osx64
hg cp osx osx64

Comment 14

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 697739 [details] [diff] [review]
fix talos softlink

Review of attachment 697739 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm

Comment 15

[Aki Sasaki (not active)]

Comment on attachment 697739 [details] [diff] [review]
fix talos softlink

http://hg.mozilla.org/build/talos/rev/4a0e7514c66c

I'm going to try using this revision on cedar now.

Comment 16

[Aki Sasaki (not active)]

We seem to be using the new version of talos successfully.
I'm going to keep this bug open 'til jhammel reviews https://bugzilla.mozilla.org/attachment.cgi?id=697733&action=edit .

Comment 17

[Jeff Hammel]

Comment on attachment 697733 [details] [diff] [review]
venv error -> warning

lgtm

Comment 18

[Aki Sasaki (not active)]

Comment on attachment 697733 [details] [diff] [review]
venv error -> warning

http://hg.mozilla.org/build/mozharness/rev/7a7dafc08703