Self-serve should be able to request arbitrary builds on a push (not just retriggers or complete sets of dep/PGO/Nightly builds)

RESOLVED FIXED

Status

Release Engineering
General Automation
P3
enhancement
RESOLVED FIXED
5 years ago
3 years ago

People

(Reporter: gkw, Assigned: zeller)

Tracking

({buildapi, sheriffing-P1})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(4 attachments, 7 obsolete attachments)

(Reporter)

Description

5 years ago
Now we can only trigger a (re)build of Valgrind tbpl builds by clicking the + sign when the V letter appears on a changeset.

It would be nice to have a link somewhere on other changesets to manually trigger a build of Valgrind builds on tbpl instead of waiting for the next nightly cycle or ping'ing a releng dev. This will be useful esp. as we land suppression files, etc.
This already exists AFAICT. Click on the "V" cell and then click the "+" in the bottom left. I just successfully retriggered the Linux64 one from 08d435dedc7f.
Component: Release Engineering: Automation (General) → Tinderboxpushlog
Product: mozilla.org → Webtools
QA Contact: catlee

Comment 2

5 years ago
That's only a retrigger, comment 0 was asking to be able to trigger a new valgrind build on a push that hasn't yet built one; or more generically, to be able to request a new build X when one hasn't been generated before.

This would be useful in other situations too (eg "please can you trigger only a win32 nightly on push X" type requests that we see on #developers occasionally; or for when sheriffs are trying to find the cause of bustage and have to fall back to requesting a whole set of dep builds just to get the one platform they are interested in).
Component: Tinderboxpushlog → Release Engineering: Automation (General)
Product: Webtools → mozilla.org
QA Contact: catlee
Summary: Have a way to manually trigger a Valgrind build on tbpl → Self-serve should be able to request arbitrary builds on a push (not just retriggers or complete sets of dep/PGO/Nightly builds)
Whiteboard: [sheriff-want]

Updated

5 years ago
Whiteboard: [sheriff-want] → [buildapi][sheriff-want]
Is this specific to valgrind, or you want this for everything?
Severity: normal → enhancement
Priority: -- → P3
(Reporter)

Comment 4

5 years ago
I think Ed would like this for everything, see comment 2.

Comment 5

5 years ago
Everything :-)

Updated

5 years ago
Keywords: buildapi

Updated

5 years ago
Keywords: sheriffing-P1
Whiteboard: [buildapi][sheriff-want]

Updated

5 years ago
Duplicate of this bug: 649565
Blocks: 855282
Product: mozilla.org → Release Engineering

Comment 7

4 years ago
We are trying to get this moved into the queue for Q4. And to that end, I'd like to summarize the requirements given that 2013 Q4 is a really short quarter.
* Given a build (so we have symbol files, test packages, and binary builds) on any OS, we'd like to:
** Trigger any of the standard test suite jobs/additional build jobs that were not run in the original push
** Provide an API where we can automatically trigger these jobs
** Allow this on try as well to solve the case where you built on try, then want to run a test job you didn't originally request (this prevents people from running an entirely new job on try).

Note this is all in addition to the existing ability to re-trigger already run jobs - we want that functionality to remain intact.

This will have several key benefits for infrastructure and development:
1. It will enable us to automatically bisect job failures from TBPL (we can easily find which changeset caused a regression by triggering missing jobs)
2. This will allow us to minimize the set of jobs we run so that we can experiment with different ways to reduce total end to end time in the automation (which means developers get results faster, and that trees are closed less often waiting for green changesets to come back).

For Q4, if we had just the ability to manually trigger missing jobs, that would be enough for us to start experimenting. We can add the API triggering later. 

Catlee, does that help you scope this at all?
Flags: needinfo?(catlee)
Yes. My plan is to service this by implementing this method in self-serve:
http://hg.mozilla.org/build/buildapi/file/358a04471ef1/buildapi/controllers/selfserve.py#l502

We'll need to support posting the URLs to the builds and tests and symbols as well.
Flags: needinfo?(catlee)
Blocks: 941290
(Assignee)

Updated

4 years ago
Assignee: nobody → johnlzeller
(Assignee)

Comment 9

3 years ago
Created attachment 8374520 [details]
bug793989-patch.py

Here is the phase 1 first draft. I need some feedback on this and please note that this 'patch' has not been tested as I am just seeking feedback.
Attachment #8374520 - Flags: feedback?(catlee)
Comment on attachment 8374520 [details]
bug793989-patch.py

>## controllers/selfserve.py in class SlefserveController
>def new_build_for_builder(self, branch, buildername, revision):
>    who = self._require_auth()
>
>    if branch not in self._branches_cache:
>        return self._failed("Branch %s not found" % branch, 404)
>
>    # Get priority
>    try:
>        priority = validators.Int(if_empty=0).\
>            to_python(request.POST.get('priority'))
>    except formencode.Invalid:
>        return self._failed('Bad priority', 400)
>
>    # Get properties
>    try:
>        properties = json.loads(request.POST.get('properties'))
>        assert isinstance(properties, dict)
>    except Exception:
>        properties = {}
>
>    # Get list of change objects
>    try:
>        changes = json.loads(request.POST.get('changes'))
>        assert isinstance(changes, list)
>        # TODO: Set change branches to ${branch}-selfserve because if they're set to
>        # something real, then this will trigger schedulers.
>    except Exception:
>        changes = []

It looks below like you're treating this as a list of files that have changed, which is totally fine to start with. Let's just get the naming fixed up here. Call the POST parameter 'files', and variable names to match. You've addressed the TODO below, which is great

>
>    # Set branch to ${branch}-selfserve to keep schedulers from triggering
>    branch += "-selfserve"
>    access_log.info("%s new_build_for_builder of %s %s %s",
>                    who, branch, buildername, revision)
>    retval = g.mq.newBuildForBuilder(
>                    who, branch, revision, priority, buildername, properties, changes)
>    response.status = 202
>
>    return self._format(retval)
>
>## lib/mq.py in class JobRequestPublisher
>def newBuildForBuilder(self, who, branch, revision, priority, buildername,
>                       properties, changes):
>    return self.send_msg('new_build_for_builder', who=who, branch=branch,
>                         revision=revision, priority=priority,
>                         buildername=buildername, properties=properties,
>                         changes=changes)
>
>## scripts/selfserve-agent.py in class BuildAPIAgent
>def do_new_build_for_builder(self, message_data, message):
>    who = message_data['who']
>    branch = message_data['body']['branch']
>    revision = message_data['body']['revision']
>    priority = message_data['body']['priority']
>    buildername = message_data['body']['buildername']
>    changes = message_data['body']['changes']

s/changes/files/

>    log.info("New build for %s by %s of %s %s", buildername, who, branch, revision)
>
>    # Create a sourcestamp
>    repo_path = self._get_repo_path(branch.split("-selfserve")[0])
>    q = text("""INSERT INTO sourcestamps
>            (`branch`, `revision`, `patchid`, `repository`, `project`)
>            VALUES
>            (:branch, :revision, NULL, '', '')
>            """)
>    log.debug(q)
>    r = self.db.execute(q, branch=repo_path, revision=revision)
>    ssid = r.lastrowid # SourcestampID
>    log.debug("Created sourcestamp %s", ssid)
>
>    # Create changes

so this becomes "create change object"

>    when = now.now()
>    q = text("""INSERT INTO changes
>            (`author`, `comments`, `is_dir`, `branch`, 
>            `revision`, `revlink`, `when_timestamp`, `category`, 
>            `repository`, `project`)
>            VALUES
>            (:who, '', 0, :branch, :revision, NULL, :when, NULL, '', '')
>            """)
>    log.debug(q)
>    r = self.db.execute(q, who=who, branch=branch, revision=revision, when=when)   
>    cid = r.lastrowid
>    log.debug("Created change %s", cid)
>
>    # Create change-files
>    for change in changes:
>        q = text("""INSERT INTO change_files
>                (`changeid`, `filename`)
>                VALUES
>                (:cid, :change)
>                """)
>        log.debug(q)
>        r = self.db.execute(q, cid=cid, change=change) 

s/change/file/

>    log.debug("Created change_file for change %s", cid)
>
>    # Create sourcestamp_changes
>    q = text("""INSERT INTO sourcestamp_changes
>            (`sourcestampid`, `changeid`)
>            VALUES
>            (:ssid, :cid)
>            """)
>    log.debug(q)
>    r = self.db.execute(q, ssid=ssid, cid=cid) 
>    log.debug("Created sourcestamp_changes for sourcestamp %s, changes %s", ssid, cid)
>
>    # Create a new buildset
>    now = time.time()
>    buildsetid = create_buildset(
>        self.db,
>        idstring=None,
>        reason='Self-serve: Requested by %s' % who,
>        ssid=ssid,
>        submitted_at=now,
>    )
>
>    # Create buildset properties (buildid, builduid)
>    q = text("""INSERT INTO buildset_properties
>            (`buildsetid`, `property_name`, `property_value`)
>            VALUES
>            (:buildsetid, :key, :value)
>            """)
>    props = {
>        'buildid': json.dumps((genBuildID(now), "self-serve")),
>        'builduid': json.dumps((genBuildUID(), "self-serve")),
>    }
>    props.update(message_data['body']['properties'])
>    log.debug(q)
>    for key, value in props.items():
>        r = self.db.execute(q, buildsetid=buildsetid, key=key, value=value)
>        log.debug("Created buildset_property %s=%s", key, value)
>
>    # Create buildrequests
>    q = text("""INSERT INTO buildrequests
>            (`buildsetid`, `buildername`, `submitted_at`, `priority`, `claimed_at`, `claimed_by_name`, `claimed_by_incarnation`, `complete`, `results`, `complete_at`)
>            VALUES
>            (:buildsetid, :buildername, :submitted_at, :priority, 0, NULL, NULL, 0, NULL, NULL)""")
>    log.debug(q)
>    r = self.db.execute(
>        q,
>        buildsetid=buildsetid,
>        buildername=buildername,
>        submitted_at=now,
>        priority=priority)
>    log.debug("Created buildrequest %s: %i", buildername, r.lastrowid)
>    return {"errors": False, "msg": "Ok"}


Other than that, looks great!
Attachment #8374520 - Flags: feedback?(catlee) → feedback+
(Assignee)

Comment 11

3 years ago
Created attachment 8378691 [details] [diff] [review]
bug793989.patch

Okay this patch includes the feedback from catlee and the patch is tested via revision 90b8ba3a9b43. Buildbot seems to like it and it shows up as a pending build request on buildbot and in the schedulerdb.
Attachment #8374520 - Attachment is obsolete: true
Attachment #8378691 - Flags: review?
(Assignee)

Updated

3 years ago
Attachment #8378691 - Flags: review? → review?(catlee)
(Assignee)

Comment 12

3 years ago
(In reply to John Zeller [:zeller] from comment #11)
> Created attachment 8378691 [details] [diff] [review]
> bug793989.patch
> 
> Okay this patch includes the feedback from catlee and the patch is tested
> via revision 90b8ba3a9b43. Buildbot seems to like it and it shows up as a
> pending build request on buildbot and in the schedulerdb.

To test this patch you can use this short script to send a POST request to buildapi, assuming it's running locally. http://zeller.pastebin.mozilla.org/4341766

Comment 13

3 years ago
John, this is awesome work. I don't have buildapi running locally, but I can't wait to test this out in staging or production.
Comment on attachment 8378691 [details] [diff] [review]
bug793989.patch

Review of attachment 8378691 [details] [diff] [review]:
-----------------------------------------------------------------

Looking good!

Mostly style nits and cleanup left.

I do want to nail down what's going on with the encoding of the property values.

::: buildapi/config/routing.py
@@ +77,4 @@
>      map.connect('new_build_at_rev', '/self-serve/{branch}/rev/{revision}', controller='selfserve', action='new_build_at_rev', conditions=dict(method=['POST']))
>      map.connect('new_pgobuild_at_rev', '/self-serve/{branch}/rev/{revision}/pgo', controller='selfserve', action='new_pgobuild_at_rev', conditions=dict(method=['POST']))
>      map.connect('new_nightly_at_rev', '/self-serve/{branch}/rev/{revision}/nightly', controller='selfserve', action='new_nightly_at_rev', conditions=dict(method=['POST']))
> +    map.connect('test_new_build_for_builder', '/self-serve/{branch}/test_builders', controller='selfserve', action='new_build_for_builder')#, conditions=dict(method=['POST']))

can this go away?

@@ +78,5 @@
>      map.connect('new_pgobuild_at_rev', '/self-serve/{branch}/rev/{revision}/pgo', controller='selfserve', action='new_pgobuild_at_rev', conditions=dict(method=['POST']))
>      map.connect('new_nightly_at_rev', '/self-serve/{branch}/rev/{revision}/nightly', controller='selfserve', action='new_nightly_at_rev', conditions=dict(method=['POST']))
> +    map.connect('test_new_build_for_builder', '/self-serve/{branch}/test_builders', controller='selfserve', action='new_build_for_builder')#, conditions=dict(method=['POST']))
> +    map.connect('new_build_for_builder', '/self-serve/{branch}/builders/{builder_name}/{revision}', controller='selfserve', action='new_build_for_builder', conditions=dict(method=['POST']))
> +    

some extra whitespace to clean up

::: buildapi/controllers/selfserve.py
@@ +250,4 @@
>          return self._format(retval)
>  
>      @beaker_cache(query_args=True, expire=60)
> +    def builder(self, branch, buildername):

this change seems unnecessary?

@@ +518,5 @@
> +            assert isinstance(properties, dict)
> +        except Exception:
> +            properties = {}
> +
> +        # Get list of change objects

# Get the list of files

@@ +523,5 @@
> +        try:
> +            files = json.loads(request.POST.get('files'))
> +            assert isinstance(files, list)
> +            # TODO: Set change branches to ${branch}-selfserve because if they're set to
> +            # something real, then this will trigger schedulers.

I think this TODO can go away, since you're handling it below

::: buildapi/lib/mq.py
@@ +154,5 @@
>      def newNightlyAtRevision(self, who, branch, revision, priority):
>          return self.send_msg('new_nightly_at_revision', who=who, branch=branch, revision=revision, priority=priority)
>  
> +    def newBuildForBuilder(self, who, branch, revision, priority, buildername,
> +                           properties, files):

let's be consistent with naming here. you're using builder_name above, so let's use builder_name here too

::: buildapi/scripts/selfserve-agent.py
@@ +577,5 @@
>              msgs.append(result['msg'])
>          return {"errors": errors, "msg": "\n".join(msgs)}
>  
> +    def do_new_build_for_builder(self, message_data, message):
> +        print "In do_new_build_for_builder"

remove this, or change into a log.debug call

@@ +582,5 @@
> +        who = message_data['who']
> +        branch = message_data['body']['branch']
> +        revision = message_data['body']['revision']
> +        priority = message_data['body']['priority']
> +        buildername = message_data['body']['buildername']

same naming consistency here, and below

@@ +587,5 @@
> +        files = message_data['body']['files']
> +        log.info("New build for %s by %s of %s %s", buildername, who, branch, revision)
> +
> +        # Create a sourcestamp
> +        repo_path = self._get_repo_path(branch.split("-selfserve")[0])

why are you stripping off -selfserve from the branch here?

@@ +601,5 @@
> +
> +        # Create change object
> +        when = time.time()
> +        q = text("""INSERT INTO changes
> +                (`author`, `comments`, `is_dir`, `branch`, 

clean up trailing whitespace here, and below

@@ +656,5 @@
> +        }
> +        props.update(message_data['body']['properties'])
> +        log.debug(q)
> +        for key, value in props.items():
> +            r = self.db.execute(q, buildsetid=buildsetid, key=key, value=value)

I *think* the values should be json encoded. Can you check other places where we're working with buildset properties to verify? It's not clear here if message_data['body']['properties'] are json encoded values, or have been decoded.
Attachment #8378691 - Flags: review?(catlee) → review-
(Assignee)

Comment 15

3 years ago
(In reply to Clint Talbert ( :ctalbert ) from comment #13)
> John, this is awesome work. I don't have buildapi running locally, but I
> can't wait to test this out in staging or production.

Thanks Clint! Much of the thanks should go to catlee for getting the patch more than half way there before I picked it up. Nevertheless, I appreciate the kudos :)
(Assignee)

Comment 16

3 years ago
Created attachment 8379344 [details] [diff] [review]
bug793989.patch

(In reply to Chris AtLee [:catlee] from comment #14)
> Comment on attachment 8378691 [details] [diff] [review]
> bug793989.patch
> 
> Review of attachment 8378691 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Looking good!
> 
> Mostly style nits and cleanup left.
> 
> I do want to nail down what's going on with the encoding of the property
> values.
> 
> ::: buildapi/config/routing.py
> @@ +77,4 @@
> >      map.connect('new_build_at_rev', '/self-serve/{branch}/rev/{revision}', controller='selfserve', action='new_build_at_rev', conditions=dict(method=['POST']))
> >      map.connect('new_pgobuild_at_rev', '/self-serve/{branch}/rev/{revision}/pgo', controller='selfserve', action='new_pgobuild_at_rev', conditions=dict(method=['POST']))
> >      map.connect('new_nightly_at_rev', '/self-serve/{branch}/rev/{revision}/nightly', controller='selfserve', action='new_nightly_at_rev', conditions=dict(method=['POST']))
> > +    map.connect('test_new_build_for_builder', '/self-serve/{branch}/test_builders', controller='selfserve', action='new_build_for_builder')#, conditions=dict(method=['POST']))
> 
> can this go away?

Absolutely

> 
> ::: buildapi/controllers/selfserve.py
> @@ +250,4 @@
> >          return self._format(retval)
> >  
> >      @beaker_cache(query_args=True, expire=60)
> > +    def builder(self, branch, buildername):
> 
> this change seems unnecessary?

Leftovers from somewhere, I removed them

> 
> @@ +587,5 @@
> > +        files = message_data['body']['files']
> > +        log.info("New build for %s by %s of %s %s", buildername, who, branch, revision)
> > +
> > +        # Create a sourcestamp
> > +        repo_path = self._get_repo_path(branch.split("-selfserve")[0])
> 
> why are you stripping off -selfserve from the branch here?

Unclear why I did that... I must've thought it was necessary, but I looked at it again and I am not sure how I came to that conclusion. I've removed it.

> 
> @@ +656,5 @@
> > +        }
> > +        props.update(message_data['body']['properties'])
> > +        log.debug(q)
> > +        for key, value in props.items():
> > +            r = self.db.execute(q, buildsetid=buildsetid, key=key, value=value)
> 
> I *think* the values should be json encoded. Can you check other places
> where we're working with buildset properties to verify? It's not clear here
> if message_data['body']['properties'] are json encoded values, or have been
> decoded.

We aren't directly handling properties in any other part of selfserve-agent. In every other case, we are simply making a SQL query to copy properties from one row to another, and so we aren't handling any JSON details.

At this point in the flow message_data['body']['properties'] are decoded. They are JSON encoded in the POST request, decoded in selfserve, passed to mq, passed to selfserve-agent and then by this point are still decoded. Given that the rest of props, just above the props.updates line is json.dump'd, do you think we should go ahead and do that here as well?

I made the other changes you've requested and am adding the new patch here.
Attachment #8378691 - Attachment is obsolete: true
Attachment #8379344 - Flags: review?(catlee)
(Assignee)

Comment 17

3 years ago
Created attachment 8381132 [details] [diff] [review]
bug793989.patch

Okay this patch takes care of the JSON encoding of property values before entering into the database and the patch now represents the diff from the current buildapi tip.
Attachment #8379344 - Attachment is obsolete: true
Attachment #8379344 - Flags: review?(catlee)
Attachment #8381132 - Flags: review?(catlee)
Comment on attachment 8381132 [details] [diff] [review]
bug793989.patch

Review of attachment 8381132 [details] [diff] [review]:
-----------------------------------------------------------------

::: buildapi/scripts/selfserve-agent.py
@@ +658,5 @@
> +        props = {
> +            'buildid': json.dumps((genBuildID(now), "self-serve")),
> +            'builduid': json.dumps((genBuildUID(), "self-serve")),
> +        }
> +        props.update(((k, json.dumps((v, "self-serve"))) for (k,v) in message_data['body']['properties'].iteritems()))

A comment explaining this would be awesome!
Attachment #8381132 - Flags: review?(catlee) → review+
(Assignee)

Comment 19

3 years ago
Created attachment 8381680 [details] [diff] [review]
bug793989.patch

Added a comment, and based on the review+ from comment this should be good to go!
Attachment #8381132 - Attachment is obsolete: true
Attachment #8381680 - Flags: checked-in?

Updated

3 years ago
Attachment #8381680 - Flags: checked-in? → checked-in+
so are we able to trigger arbitrary builds now for a given revision?
(Assignee)

Comment 21

3 years ago
(In reply to Joel Maher (:jmaher) from comment #20)
> so are we able to trigger arbitrary builds now for a given revision?

Just about, waiting on deployment of the new buildapi tip. I will update here once it is up.
(Assignee)

Comment 22

3 years ago
Created attachment 8384980 [details] [diff] [review]
puppet_install_bug_793989.patch

Here is the patch for the puppet manifest including the new tar.gz package distro name.

However, I noticed that sdist gave me a tarball with the prefix buildapi-0.2.1dev rather than just plainly buildapi-0.2.1. This provokd me to dig further and so I ran a diff between buildapi-0.2.0.tar.gz and buildapi-0.2.1dev.tar.gz. See diff attached in next comment. Any idea if the missing schemas, RESTAPI_NOTES.txt, etc is legit?
Attachment #8384980 - Flags: review?(hwine)
(Assignee)

Comment 23

3 years ago
Created attachment 8384981 [details] [diff] [review]
0.2.0_to_0.2.1dev.patch

Here is the diff I referred to in comment 22
Comment on attachment 8384980 [details] [diff] [review]
puppet_install_bug_793989.patch

Review of attachment 8384980 [details] [diff] [review]:
-----------------------------------------------------------------

::: modules/selfserve_agent/manifests/install.pp
@@ +28,5 @@
>                  "PasteScript==1.7.3",
>                  "Pygments==1.4",
>                  "Pylons==1.0",
>                  "amqp==1.4.3",
> +                "buildapi==0.2.1dev",

see email - we want without the 'dev' suffix, but that is baked into various parts.
Attachment #8384980 - Flags: review?(hwine) → review-
(Assignee)

Comment 25

3 years ago
Created attachment 8386387 [details] [diff] [review]
puppet_install_bug_793989.patch

Ran new tarball for buildapi-0.2.1, uploaded to python packages, and ran new patch for selfserve puppet manifests.
Attachment #8384980 - Attachment is obsolete: true
Attachment #8386387 - Flags: review?(hwine)
(Assignee)

Comment 26

3 years ago
Created attachment 8386394 [details] [diff] [review]
puppet_install_bug_793989.patch

update for 0.2.2
Attachment #8386387 - Attachment is obsolete: true
Attachment #8386387 - Flags: review?(hwine)
Attachment #8386394 - Flags: review?(nthomas)
Comment on attachment 8386394 [details] [diff] [review]
puppet_install_bug_793989.patch

http://hg.mozilla.org/build/puppet/rev/a27342af3b33 and merged to production.
Attachment #8386394 - Flags: review?(nthomas)
Attachment #8386394 - Flags: review+
Attachment #8386394 - Flags: checked-in+
This is in production as of 2:40p Pacific.

Comment 29

3 years ago
Yay!
I would like to test this, how do I go about it?

I tried loading this:
https://secure.pub.build.mozilla.org/buildapi/self-serve/mozilla-central/builders/b2g_mozilla-central_win32_gecko%20build/d2dac18d0562
(In reply to Armen Zambrano [:armenzg] (Release Engineering) (EDT/UTC-4) from comment #29)
> Yay!
> I would like to test this, how do I go about it?

Clarifying the status -- the selfserve_agent with code to support the new functionality is deployed. Some additional verification and testing is needed that could not be done in the dev environment. So, this is not yet live.

> I tried loading this:
> https://secure.pub.build.mozilla.org/buildapi/self-serve/mozilla-central/
> builders/b2g_mozilla-central_win32_gecko%20build/d2dac18d0562

At this stage, you can not use a GET -- you need to use a POST with the parameters in JSON.

:zeller - can you attach a sample script to send a message?
Flags: needinfo?(johnlzeller)
(Assignee)

Comment 31

3 years ago
Created attachment 8387894 [details]
sample_post_to_bug793989_patch.py

(In reply to Hal Wine [:hwine] (use needinfo) from comment #30)
> (In reply to Armen Zambrano [:armenzg] (Release Engineering) (EDT/UTC-4)
> from comment #29)
> > Yay!
> > I would like to test this, how do I go about it?
> 
> Clarifying the status -- the selfserve_agent with code to support the new
> functionality is deployed. Some additional verification and testing is
> needed that could not be done in the dev environment. So, this is not yet
> live.
> 
> > I tried loading this:
> > https://secure.pub.build.mozilla.org/buildapi/self-serve/mozilla-central/
> > builders/b2g_mozilla-central_win32_gecko%20build/d2dac18d0562
> 
> At this stage, you can not use a GET -- you need to use a POST with the
> parameters in JSON.
> 
> :zeller - can you attach a sample script to send a message?

Surely! The attached script will rebuild a build, and doesn't specify any other files or properties to be included. Take a swing!
Flags: needinfo?(johnlzeller)

Updated

3 years ago
Depends on: 981825

Comment 32

3 years ago
So if I were to trigger a test run, I'd essentially put the revision etc into the script mentioned in comment 31 with a proper builder name? Like "mochitest-1 opt" or some such?
(Assignee)

Comment 33

3 years ago
(In reply to Clint Talbert ( :ctalbert ) from comment #32)
> So if I were to trigger a test run, I'd essentially put the revision etc
> into the script mentioned in comment 31 with a proper builder name? Like
> "mochitest-1 opt" or some such?

Close. As long as branch is a valid branch, and revision is an existing valid revision on that branch, then just add the buildername you wish to trigger, such as "Ubuntu VM 12.04 try opt test mochitest-1" or "Linux x86-64 try build". Any buildername that shows up in the builder column on buildapi should work.
(Assignee)

Comment 34

3 years ago
Okay the patch has been run through its paces and it looks good so far! You can now go ahead and start launching your arbitrary builds/tests with the new functionality. Use attachment 8387894 [details] to get yourself started!

As you can see in the simple sample script, buildapi expects a POST request that includes the paramaters properties and files, where properties is a dictionary and files is a list, to the REST URL at https://secure.pub.build.mozilla.org/buildapi/self-serve/{branch}/builders/{buildername}/{revision}, where {branch} is a valid branch, {revision} is a valid revision existing on that branch, and {buildername} is the arbitrary builder for the build/test that you would like to trigger. Examples of appropriate buildernames are "Ubuntu VM 12.04 try opt test mochitest-1" or "Linux x86-64 try build"; Any buildername that shows up in the builder column on buildapi should work. Please file a bug if you find any exceptions!

As a specific example, if you are launching the test "Ubuntu VM 12.04 x64 try opt test jetpack" after having run the build "Linux x86-64 try build", then you need to supply the files list parameter as ["http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/cviecco@mozilla.com-b5458592c1f3/try-linux64-debug/firefox-30.0a1.en-US.linux-x86_64.tar.bz2", "http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/cviecco@mozilla.com-b5458592c1f3/try-linux64-debug/firefox-30.0a1.en-US.linux-x86_64.tests.zip"].
(Assignee)

Comment 35

3 years ago
Updated the docs on the BuildAPI SelfServe Interface and on wiki.m.o., here: https://secure.pub.build.mozilla.org/buildapi/self-serve
and, here: https://wiki.mozilla.org/ReleaseEngineering/Applications/BuildAPI#Job_Requests

Please report any problems that arise in a new bug :)
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Assignee)

Updated

3 years ago
Blocks: 984593

Updated

3 years ago
Depends on: 1009565
Duplicate of this bug: 918038
Created attachment 8473675 [details]
arbitrary.py

John, I improved your triggering script a bit. It now prompts for username, password, builder, branch, and revision.
Attachment #8387894 - Attachment is obsolete: true
The script is now in a repo at:
https://hg.mozilla.org/build/braindump/file/default/buildbot-related/trigger_arbitrary_job.py

Docs at:
https://wiki.mozilla.org/ReleaseEngineering/How_To/Trigger_arbitrary_jobs
You need to log in before you can comment on or make changes to this bug.