Closed Bug 734368 Opened 11 years ago Closed 11 years ago

Firebug test-bot should download test files from GIT instead of SVN

Categories

(Testing :: General, defect)

x86
Windows Vista
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: Honza, Assigned: cmtalbert)

Details

(Whiteboard: [firebug-p1])

Attachments

(1 file, 1 obsolete file)

Firebug source base has been moved to GIT (github.com), which also means that the test-bot should download all automated test files from a new location.

Here is Firebug repo:
https://github.com/firebug/firebug

Here is the folder with all the test files:
https://github.com/firebug/firebug/tree/master/tests/content

It was previously on SVN here:
http://code.google.com/p/fbug/source/browse/#svn%2Ftests%2Fcontent%2Fbranches%2F1.10

Honza
Whiteboard: [firebug-p1]
This should do it. It will enable us to use either SVN (for old releases still on SVN) and git for new releases.

Jan, one thing you will need to do is to include a GIT_TAG in your config file in place of the SVN_REVISION that you previously had. This way we can pull the correct version of the git repo for testing.

Here is an example test_bot.config file that I used for testing:
[Firebug.10]
FIREBUG_XPI=http://getfirebug.com/releases/firebug/1.10/firebug-1.10.0a4.xpi
FBTEST_XPI=http://getfirebug.com/releases/fbtest/1.10/fbTest-1.10b3.xpi
GIT_TAG=1.10.0a5
GECKO_VERSION=beta,central
TEST_LIST=http://getfirebug.com/tests/content/testlists/firebug1.10.html
Assignee: nobody → ctalbert
Status: NEW → ASSIGNED
Attachment #606823 - Flags: review?(halbersa)
Excellent!

Couple of questions:

1) Could we also specify GIT hash for GIT_TAG? Just in case we'd like to test and let the test-bot check specific revision that is not tagged (related to a release)

2) The TEST_LIST URL should be: http://getfirebug.com/tests/content/firebug.html 
coming from: https://github.com/firebug/firebug/blob/master/tests/content/firebug.html

Which also means that we need to sync:
https://github.com/firebug/firebug/tree/master/tests/
to:
http://getfirebug.com/tests/

---

There are actually more dirs on getfirebug.com we need to keep synced with the stuff from GIT. Should I file a new bug for it?

Summary:
1) https://github.com/firebug/firebug/tree/master/tests
to:
http://getfirebug.com/tests/

2) https://github.com/firebug/manual-tests
to:
http://getfirebug.com/manual-tests/

3) https://github.com/firebug/test-result-viewer
to:
http://getfirebug.com/test-result-viewer/


Honza
Comment on attachment 606823 [details] [diff] [review]
Fix updater to work with git or svn

Review of attachment 606823 [details] [diff] [review]:
-----------------------------------------------------------------

This is close, but I think there's a problem with the workflow (addressed below)

::: runFBTests/fb_update.py
@@ +75,5 @@
> +            GIT_TAG = test_bot.get(section, "GIT_TAG")
> +            tagrepodir = os.path.join(opt.repo, GIT_TAG)
> +            if not os.path.isdir(tagrepodir):
> +                subprocess.Popen(["git","clone","https://github.com/firebug/firebug.git",
> +                        tagrepodir]).communicate()

I'm a bit confused by this part. What exactly is GIT_TAG? Is it a changeset? Or is it an actual git tag that Jan has to manually create himself?

At any rate, I think there's a problem here. We should only have to clone the repo once, if the GIT_TAG doesn't exist locally we should do a "git pull origin master" instead of cloning the entire repository again. I think the workflow should be:
0) if this is the first time running, clone firebug
--otherwise--
1) pull latest master
2) checkout the desired changeset/tag
3) copy the required files to the webserver's document root under a GIT_TAG subdirectory
Attachment #606823 - Flags: review?(halbersa) → review-
(In reply to Andrew Halberstadt [:ahal] from comment #3)
> At any rate, I think there's a problem here. We should only have to clone
> the repo once, if the GIT_TAG doesn't exist locally we should do a "git pull
> origin master" instead of cloning the entire repository again. I think the
> workflow should be:
> 0) if this is the first time running, clone firebug
> --otherwise--
> 1) pull latest master
> 2) checkout the desired changeset/tag
> 3) copy the required files to the webserver's document root under a GIT_TAG
> subdirectory

Sounds reasonable to me.

Honza
(In reply to Jan Honza Odvarko from comment #2)
> Excellent!
> 
> Couple of questions:
> 
> 1) Could we also specify GIT hash for GIT_TAG? Just in case we'd like to
> test and let the test-bot check specific revision that is not tagged
> (related to a release)
> 
> 2) The TEST_LIST URL should be:
> http://getfirebug.com/tests/content/firebug.html 
> coming from:
> https://github.com/firebug/firebug/blob/master/tests/content/firebug.html
> 
> Which also means that we need to sync:
> https://github.com/firebug/firebug/tree/master/tests/
> to:
> http://getfirebug.com/tests/
> 
> ---
> 
> There are actually more dirs on getfirebug.com we need to keep synced with
> the stuff from GIT. Should I file a new bug for it?
> 
> Summary:
> 1) https://github.com/firebug/firebug/tree/master/tests
> to:
> http://getfirebug.com/tests/
> 
> 2) https://github.com/firebug/manual-tests
> to:
> http://getfirebug.com/manual-tests/
> 
> 3) https://github.com/firebug/test-result-viewer
> to:
> http://getfirebug.com/test-result-viewer/
> 
I don't understand how this affects the automation at all.
Do we even care about svn anymore? I'm operating under the assumption we do, and that makes the code more complex than it needs to be (and this code is already a little too complex for what it's doing).
Attached patch take 2Splinter Review
ok, in keeping the svn logic, I finally came to my senses and found a way to fake it without re-writing everything.  This uses the method y'all describe for working with firebug, and can either work with svn or git.

If you want a config file for testing to point it at you can use this one:
http://people.mozilla.org/~ctalbert/testme.config

The GIT_TAG can be a git changeset UUID or a branch or a tag. So, any of those will work.
Attachment #606823 - Attachment is obsolete: true
Attachment #612757 - Flags: review?(halbersa)
(In reply to Clint Talbert ( :ctalbert ) from comment #7)
> The GIT_TAG can be a git changeset UUID or a branch or a tag. So, any of
> those will work.
Excellent!

Honza
(In reply to Clint Talbert ( :ctalbert ) from comment #5)
> > There are actually more dirs on getfirebug.com we need to keep synced with
> > the stuff from GIT. Should I file a new bug for it?
> > 
> > Summary:
> > 1) https://github.com/firebug/firebug/tree/master/tests
> > to:
> > http://getfirebug.com/tests/
> > 
> > 2) https://github.com/firebug/manual-tests
> > to:
> > http://getfirebug.com/manual-tests/
> > 
> > 3) https://github.com/firebug/test-result-viewer
> > to:
> > http://getfirebug.com/test-result-viewer/
> > 
> I don't understand how this affects the automation at all.

These are not directly related to automated testing (even if e.g. the test-result-viewer is an online app that displays test results from
CouchDB). We just need these files online (e.g. to make it easier to manually check out a test scenario).

Perhaps I should file a new bug for this?

Honza
Comment on attachment 612757 [details] [diff] [review]
take 2

Review of attachment 612757 [details] [diff] [review]:
-----------------------------------------------------------------

Looks great. r+ with a few nits.

::: runFBTests/fb_update.py
@@ +50,4 @@
>  import urlparse
>  import socket
>  import platform
> +import traceback

Nit: traceback not used

@@ +56,5 @@
>      return urlparse.urlsplit(url).path.lstrip('/')
>  
> +def recursivecopy(src, dest):
> +    if not os.path.exists(dest):
> +        os.mkdir(dest)

Nit: should probably use os.makedirs as mkdir won't create intermediate nodes.

@@ +116,5 @@
> +            # Update or create the svn test repository
> +            if not os.path.isdir(os.path.join(opt.repo, ".svn")):
> +                os.system("svn co http://fbug.googlecode.com/svn/tests/ " + os.path.join(opt.repo, SVN_REVISION, "tests") + " -r " + SVN_REVISION)
> +            else:
> +                os.system(os.path.join(opt.repo, "svn") + " update -r " + SVN_REVISION)

Nit: I realize this is my code from way back when, but we might as well update os.system to subprocess.Popen for consistency.
Attachment #612757 - Flags: review?(halbersa) → review+
(In reply to Jan Honza Odvarko from comment #9)
> (In reply to Clint Talbert ( :ctalbert ) from comment #5)
> > > There are actually more dirs on getfirebug.com we need to keep synced with
> > > the stuff from GIT. Should I file a new bug for it?
> > > 
> > > Summary:
> > > 1) https://github.com/firebug/firebug/tree/master/tests
> > > to:
> > > http://getfirebug.com/tests/
> > > 
> > > 2) https://github.com/firebug/manual-tests
> > > to:
> > > http://getfirebug.com/manual-tests/
> > > 
> > > 3) https://github.com/firebug/test-result-viewer
> > > to:
> > > http://getfirebug.com/test-result-viewer/
> > > 
> > I don't understand how this affects the automation at all.
> 
> These are not directly related to automated testing (even if e.g. the
> test-result-viewer is an online app that displays test results from
> CouchDB). We just need these files online (e.g. to make it easier to
> manually check out a test scenario).
> 
> Perhaps I should file a new bug for this?
Yeah, let's file a new bug for this.
Thanks
Landed: https://github.com/mozilla/Firebug-Test-Runner/commit/9f022329844a7d410daa90fdc45315d9be20b2a7

So, Honza, do you have a test-bot.config file that I should use to point the automation at to have it start running from git?
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
(In reply to Clint Talbert ( :ctalbert ) from comment #11)
> > Perhaps I should file a new bug for this?
> Yeah, let's file a new bug for this.
> Thanks
Bug 743930 - Properly sync getfirebug.com with source base from GIT


(In reply to Clint Talbert ( :ctalbert ) from comment #12)
> Landed:
> https://github.com/mozilla/Firebug-Test-Runner/commit/
> 9f022329844a7d410daa90fdc45315d9be20b2a7
> 
> So, Honza, do you have a test-bot.config file that I should use to point the
> automation at to have it start running from git?
Yep

http://getfirebug.com/releases/firebug/test-bot.config

Honza
So, it the test bot supposed to work now?

I am maintaining my test bot config here:
http://getfirebug.com/releases/firebug/test-bot.config

But I don't see any results related to Firebug 1.10 here:
http://getfirebug.com/testresults/?dburi=http://firebug.couchone.com/

The test results DB is here:
http://firebug.couchone.com/


(the last result is from 04/11/2012)

Honza
Looks like I screwed up. The old server we have running the update for the test code took an exception to using the https:// address for github. It was getting a 403 error for some reason. CHanging to the git:// URL seemed to fix the issue.

I still wonder if the 10.1 tests are running though, even with this change. Watching the client, I didn't see anything actually happen on that run, so I'm skeptical that something else may be wrong too.
Actually, it looks like the reason the 1.10 tests aren't running is because there is an error in firebug.

Specifically I'm getting this error in the error console (verified without using the automation): TypeError: parent.querySelector is not a function, chrome://firebug/content/firefox/browserOverlay.js Line 142.

And the Fbtest console can't run due to this exception being thrown. 

I was going to file a bug but I'm not sure where you're currently tracking bugs. Are they still tracked as issues on code.google.com?
(In reply to Clint Talbert ( :ctalbert ) from comment #16)
> Actually, it looks like the reason the 1.10 tests aren't running is because
> there is an error in firebug.
> 
> Specifically I'm getting this error in the error console (verified without
> using the automation): TypeError: parent.querySelector is not a function,
> chrome://firebug/content/firefox/browserOverlay.js Line 142.
> 
> And the Fbtest console can't run due to this exception being thrown. 
> 
> I was going to file a bug but I'm not sure where you're currently tracking
> bugs. Are they still tracked as issues on code.google.com?
Yes, the list is here:
http://code.google.com/p/fbug/issues/list

Please report the issue and I'll fix it.

Thanks!

Honza
OK, I have found the problem. It's reported and fixed.
See: http://code.google.com/p/fbug/issues/detail?id=5437
There is also Firebug test build (with the patch included)

Clint, could you please try it with the new version?
Honza
Yep I can verify this works.  Do you plan to re-release a testbot.config pointing to this changeset? If you do that, I'll restart the test slaves.

(repeating comment from google code issue tracker so we have context in both places)
Great!

> Do you plan to re-release a testbot.config pointing to this changeset?
> If you do that, I'll restart the test slaves.
Done (can take about 30 min to appear online)

The test-bot.config here:
http://getfirebug.com/releases/firebug/test-bot.config

is pointing to the changeset here:
http://www.softwareishard.com/temp/firebug/firebug-1.10.0a7.5437.xpi

(as soon as 1.10a8 is out, xpi will be again on getfirebug.com)

Let me know how it looks like, thanks!

Honza
Firebug 1.10a8 is out and I have also updated the test-bot.config to point to 
http://getfirebug.com/releases/firebug/1.10/firebug-1.10.0a8.xpi

I still don't see any new test results (since we switched to GIT). I am using this page, to see the results: 
http://getfirebug.com/testresults/?dburi=http://firebug.couchone.com/

Honza
You need to log in before you can comment on or make changes to this bug.