996810 - --repo=fx-team can't be used

Status: VERIFIED FIXED

(Testing :: mozregression, defect)

Description

[William Lachance (:wlach)]

Reported from hurda on github:

C:\mozreg>mozregression --good 2014-03-01 --bits=32 --repo=fx-team
2014-03-23
Could not find build from 2014-03-23
Could not find build from 2014-03-24
Could not find build from 2014-03-25
Could not find build from 2014-03-26
Could not find build from 2014-03-27
Could not find build from 2014-03-28
Could not find build from 2014-03-29
Could not find build from 2014-03-30
Could not find build from 2014-03-31
Could not find build from 2014-04-01
Could not find build from 2014-04-02
Could not find build from 2014-04-03
Could not find build from 2014-04-04
Could not find build from 2014-04-05
Could not find build from 2014-04-06
Could not find build from 2014-04-07
Could not find build from 2014-04-08
Could not find build from 2014-04-09
Could not find build from 2014-04-10
Could not find build from 2014-04-11
Could not find build from 2014-04-12
Could not find build from 2014-04-13
Could not find build from 2014-04-14
Last good build: 2014-03-01
First bad build: 2014-04-15
Pushlog:
None/pushloghtml?startdate=2014-03-01&enddate=2014-04-15

Could not find build from 2014-04-15
Could not find build from 2014-04-16
Could not find build from 2014-04-17
Could not find build from 2014-04-18
Could not find build from 2014-04-19
Could not find build from 2014-04-20
Could not find build from 2014-04-21
Could not find build from 2014-04-22
Could not find build from 2014-04-23
Could not find build from 2014-04-24
Could not find build from 2014-04-25
Could not find build from 2014-04-26
Could not find build from 2014-04-27
Could not find build from 2014-04-28
Could not find build from 2014-04-29
Could not find build from 2014-04-30
Could not find build from 2014-05-01
Could not find build from 2014-05-02
Could not find build from 2014-05-03
Could not find build from 2014-05-04
Could not find build from 2014-05-05
Could not find build from 2014-05-06
Could not find build from 2014-05-07
Could not find build from 2014-05-08
Could not find build from 2014-05-09
Could not find build from 2014-05-10
Could not find build from 2014-05-11
Could not find build from 2014-05-12
Could not find build from 2014-05-13
Could not find build from 2014-05-14
Could not find build from 2014-05-15
Could not find build from 2014-05-16
Could not find build from 2014-05-17
Could not find build from 2014-05-18
Could not find build from 2014-05-19
Could not find build from 2014-05-20
Could not find build from 2014-05-21
Could not find build from 2014-05-22
Could not find build from 2014-05-23
Could not find build from 2014-05-24
Could not find build from 2014-05-25
Could not find build from 2014-05-26
Could not find build from 2014-05-27
Could not find build from 2014-05-28
Could not find build from 2014-05-29
Could not find build from 2014-05-30
Could not find build from 2014-05-31
Could not find build from 2014-06-01
Could not find build from 2014-06-02
Could not find build from 2014-06-03
Could not find build from 2014-06-04
Could not find build from 2014-06-05
Could not find build from 2014-06-06
Could not find build from 2014-06-07
Could not find build from 2014-06-08
Could not find build from 2014-06-09
Could not find build from 2014-06-10
Could not find build from 2014-06-11
Could not find build from 2014-06-12
Could not find build from 2014-06-13
Could not find build from 2014-06-14
Could not find build from 2014-06-15
Could not find build from 2014-06-16
Could not find build from 2014-06-17
Could not find build from 2014-06-18
Could not find build from 2014-06-19
Could not find build from 2014-06-20
Could not find build from 2014-06-21
Could not find build from 2014-06-22
Could not find build from 2014-06-23
Could not find build from 2014-06-24
Could not find build from 2014-06-25
Could not find build from 2014-06-26
Could not find build from 2014-06-27
Could not find build from 2014-06-28
Could not find build from 2014-06-29
Could not find build from 2014-06-30
Traceback (most recent call last):
  File "C:\Python27\Scripts\mozregression-script.py", line 9, in <module>
    load_entry_point('mozregression==0.14', 'console_scripts', 'mozregression')(
)
  File "build\bdist.win32\egg\mozregression\regression.py", line 276, in cli
  File "build\bdist.win32\egg\mozregression\regression.py", line 172, in bisect_
nightlies
  File "build\bdist.win32\egg\mozregression\runnightly.py", line 268, in start
  File "build\bdist.win32\egg\mozregression\runnightly.py", line 261, in install

  File "build\bdist.win32\egg\mozregression\runnightly.py", line 86, in download

  File "build\bdist.win32\egg\mozregression\runnightly.py", line 124, in getBuil
dUrl
  File "build\bdist.win32\egg\mozregression\utils.py", line 67, in urlLinks
  File "c:\python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\api.py",
 line 55, in get
    return request('get', url, **kwargs)
  File "c:\python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\api.py",
 line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "c:\python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\sessions
.py", line 383, in request
    resp = self.send(prep, **send_kwargs)
  File "c:\python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\sessions
.py", line 486, in send
    r = adapter.send(request, **kwargs)
  File "c:\python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\adapters
.py", line 330, in send
    timeout=timeout
  File "c:\python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\packages
\urllib3\connectionpool.py", line 480, in urlopen
    body=body, headers=headers)
  File "c:\python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\packages
\urllib3\connectionpool.py", line 313, in _make_request
    httplib_response = conn.getresponse(buffering=True)
  File "c:\python27\lib\httplib.py", line 1045, in getresponse
    response.begin()
  File "c:\python27\lib\httplib.py", line 409, in begin
    version, status, reason = self._read_status()
  File "c:\python27\lib\httplib.py", line 365, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "c:\python27\lib\socket.py", line 476, in readline
    data = self._sock.recv(self._rbufsize)
KeyboardInterrupt

C:\mozreg>

Comment 1

[Sam Garrett]

I don't see any builds for the fx-team repo here: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/03/. Is there a different url we should be looking at?

With the latest code this fails with:
  ...
  File "build/bdist.macosx-10.9-intel/egg/mozregression/runnightly.py", line 143, in getBuildUrl
IndexError: list index out of range

We could at least handle the failure better and suggest using a supported --repo option.

Comment 2

[(mostly gone) XtC4UaLL [:xtc4uall]]

For the Fx-team repo builds (fx-team-xxxx) can only be found in the tinderbox dir: http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/

Comment 3

[William Lachance (:wlach)]

Sam: I think this answers the question, yes?

Comment 4

[Sam Garrett]

Yes, I can probably fix this some time this week.

Comment 5

[Sam Garrett]

Hi William,

I've created a dev branch for this here: https://github.com/sinemetu1/mozregression/tree/bug996810-fx-team-repo-cant-be-used

I've tested with the following:

$ mozregression --good-rev 4560cb8554d28628aa1a7d0f415e8c9ac7de2cbf --bits=32 --repo=fx-team --inbound --bad-rev edeea8eaa741ebf05716300d385027db025168cb
Getting inbound builds between 4560cb8554d28628aa1a7d0f415e8c9ac7de2cbf and edeea8eaa741ebf05716300d385027db025168cb
Oh noes, no (more) inbound revisions :(
do you want to bisect further by fetching the repository and building? (y or n) n

$ mozregression --good 2014-03-01 --bits=32 --repo=fx-team
Running nightly for 2014-04-16
Tried to get builds from http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/04/ that match 'firefox.*mac.*\.dmg' but didn't find any.

Do you have a good set of params for the first test? I was thinking that the first test is more what hurda meant to do when he first reported the issue on github.

Comment 6

[Sam Garrett]

Attachment: Allows other repos to be used in mozregression fixing bug 996810 and bug 997322

(In reply to Sam Garrett from comment #5) 
> Do you have a good set of params for the first test?

After my latest commit these params are working nicely:

$ mozregression --good-rev bae2e26db67c18d3245a0139827b2063dd5ab795 --bits=32 --repo=fx-team --inbound --bad-rev 63c454501a6cb41b3137bd7901dbf8a6a7b3e6c7
Getting inbound builds between bae2e26db67c18d3245a0139827b2063dd5ab795 and 63c454501a6cb41b3137bd7901dbf8a6a7b3e6c7
Testing inbound build with timestamp 1401556373, revision 0712726ce2ab0f16c521221c0617ac8578620e48
Downloading build from: http://inbound-archive.pub.build.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/fx-team-macosx64/1401556373/firefox-32.0a1.en-US.mac.dmg
...

This commit should also complete this ticket: https://bugzilla.mozilla.org/show_bug.cgi?id=997322 once merged.

Does this test case take care of what hurda was looking for?

Comment 7

[William Lachance (:wlach)]

I think this looks more or less right. Things I'd like to see:

- I think the command line argument should be something like --inbound-branch. --buildPrefix is kinda confusing.
- Could you rebase the commits into a single one? It would be easier to review as one whole picture http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html

I'll assign this bug to you.

Comment 8

[William Lachance (:wlach)]

Comment on attachment 8432288 [details] [review]
Allows other repos to be used in mozregression fixing bug 996810 and bug 997322

r- mainly for the reasons above. I may have other review comments before landing but those are the big ones.

Comment 9

[Sam Garrett]

Hi William,

I've rebased these changes. I've also changed the parameter to inbound_branch instead of buildPrefix. Let me know if there are any other issues.

Also I ended up needing to follow this: https://github.com/edx/edx-platform/wiki/How-to-Rebase-a-Pull-Request to rebase and keep the same PR.

Comment 11

[William Lachance (:wlach)]

(In reply to Sam Garrett from comment #9)
> Hi William,
> 
> I've rebased these changes. I've also changed the parameter to
> inbound_branch instead of buildPrefix. Let me know if there are any other
> issues.
> 
> Also I ended up needing to follow this:
> https://github.com/edx/edx-platform/wiki/How-to-Rebase-a-Pull-Request to
> rebase and keep the same PR.

Hey Sam, sorry it took me a few days to look at this. This looks like a good start. We need to add support for actually passing --inbound-branch to mozregression itself. I also made some more minor requests for changes inside the pull request.

We also shouldn't forget to update the documentation / website to match.

Comment 12

[Aaron Train [:aaronmt]]

Sam, how's it going with this?

Comment 13

[Sam Garrett]

(In reply to Aaron Train [:aaronmt] from comment #12)
> Sam, how's it going with this?

Sorry for the delay. I will get to this today.

Comment 14

[Sam Garrett]

@William: Can you review again?

I've rebased and included the requested changes. Let me know if there is anything else that needs to be fixed.

Comment 15

[Sam Garrett]

Hi William,

Can you review? I rebased again. :)

Also, I started creating some tests for the inboundfinder but they're not in this commit.

Comment 16

[William Lachance (:wlach)]

(In reply to Sam Garrett from comment #15)
> Hi William,
> 
> Can you review? I rebased again. :)
> 
> Also, I started creating some tests for the inboundfinder but they're not in
> this commit.

Unfortunately the changes don't seem to work for me. :/ See the pull request for more information. I suspect the fix is pretty straightforward, but I don't really have time to look into it at the moment unfortunately.

Comment 17

[Sam Garrett]

Hey Will,

I've updated this PR. Can you review again?

Comment 18

[William Lachance (:wlach)]

Merged, thanks!

Comment 19

[Aaron Train [:aaronmt]]

Thanks Sam, this works for me on master!