Closed Bug 1325452 Opened 7 years ago Closed 7 years ago

Questions about labels, descriptions, SETA, etc.

Categories

(Taskcluster Graveyard :: Discussion, defect)

Product:
Taskcluster Graveyard
Component:
Discussion
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: armenzg, Unassigned)

Details

If you load this TH url [1] you will see that the CPP jobs acquire "Android armv7 unit tests" [2]

I would like to find a way to prevent this from happening.

For example the gecko decision task would not proceed if it finds more than two tasks with the same description.

I see a the job showing up on the runnable jobs api [3] as [4].

IIUC not having a meaningful ref_data_name (e.g. "desktop-test-linux64/opt-mochitest-devtools-chrome-e10s-9") prevents the action task from being able to schedule arbitrary TC jobs.



[1] https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=android%20armv7%20unit%20tests
[2] https://dxr.mozilla.org/mozilla-central/source/taskcluster/ci/android-stuff/kind.yml#71
[3] https://treeherder.mozilla.org/api/project/mozilla-inbound/runnable_jobs/?decision_task_id=KN1xmuTUR3mL_b2UvB8J_g
[4]
        {
            "ref_data_name": "android-test",
            "build_platform": "android-4-0-armv7-api15",
            "state": "runnable",
            "job_type_name": "android-test",
            "build_system_type": "taskcluster",
            "job_group_symbol": "tc",
            "result": "runnable",
            "job_type_symbol": "test",
            "platform_option": "opt",
            "job_group_name": "Executed by TaskCluster",
            "job_coalesced_to_guid": null,
            "platform": "android-4-0-armv7-api15",
            "job_type_description": "Android armv7 unit tests"
        },
The jobs linked in [1] are all Buildbot jobs, so no decision task is involved.  If something's wrong there, this is the wrong bugzilla product :)

Task descriptions are set in tests.py:
  https://dxr.mozilla.org/mozilla-central/source/taskcluster/ci/android-test/tests.yml#12
  cppunit:
      description: "CPP Unit Tests"

and indeed, that description is correct in the final task:
  https://tools.taskcluster.net/task-inspector/#K_4944SrTHaZR8JOQKlimA/

[2] is a collection of misfit tasks that someone really needs to rewrite (that's why it's called "android-stuff").  That said, the task you link to (android-test) is not "CPP" by any measure I can see.  Its symbol is tc(test).  I can't find any of those quickly in treeherder (searching for "test" is not a big help!), but I'm confident that they will have correct description "Android armv7 unit tests"

The Android CPP unit tests run via taskcluster do, indeed, have the correct description:
  https://tools.taskcluster.net/task-inspector/#EZoCcXrVQSScNtf-YLbjLw/

There are no tasks with label "android-test", so I don't think ref_data_name is correctly reflecting the task label, which is the thing you need in order to schedule arbitrary jobs.  In fact, I don't see anything in that JSON that looks like a label, so that is probably a treeherder bug of some sort.

So basically, I'm not sure what you're getting at..
Flags: needinfo?(armenzg)
Hi dustin, thank you so much for looking into it. My apologies for the confusion.
I did not realize that they were BB jobs. TH matched "Android armv7 unit tests" to those jobs and since that piece of text comes from TC I had assumed it came from there.

I can see now that there is:
* android-test
* android-build
* android-*stuff* - news to me

I can't find it on TH but that is probably because some jobs need to change first:
> files-changed:
>   - "mobile/android/base/**"
>   - "mobile/android/tests/background/junit4/**"

I wonder if there could be a way to set a value on the task definitions to tell SETA and runnable-jobs "hey, pay attention to this task"
rather than having to mess with naming of tasks.

I say not to mess with the naming because the word 'test' can be used for any task.

An easy fix might be to just tell runnable jobs to only pay attention to tasks starting with desktop-test-*/ and android-test-*/

What are your thoughts?



[1]
https://public-artifacts.taskcluster.net/PSpzHXVYTHG12Rri2-fC_A/0/public/full-task-graph.json
[2]
"android-test": {
    "attributes": {
      "build_platform": "android-test",
      "build_type": "opt",
      "kind": "android-stuff",
      "run_on_projects": [
        "all"
      ]
    },
    "dependencies": {
      "docker-image": "build-docker-image-desktop-build"
    },
    "kind_implementation": "taskgraph.task.transform:TransformTask",
    "label": "android-test",
    "task": {
      "created": {
        "relative-datestamp": "0 seconds"
      },
      "deadline": {
        "relative-datestamp": "1 day"
      },
      "expires": {
        "relative-datestamp": "1 year"
      },
      "extra": {
        "treeherder": {
          "collection": {
            "opt": true
          },
          "groupName": "Executed by TaskCluster",
          "groupSymbol": "tc",
          "jobKind": "test",
          "machine": {
            "platform": "android-4-0-armv7-api15"
          },
          "symbol": "test",
          "tier": 2
        },
        "treeherderEnv": [
          "production",
          "staging"
        ]
      },
      "metadata": {
        "description": "Android armv7 unit tests",
        "name": "android-test",
        "owner": "rbarker@mozilla.com",
        "source": "https://hg.mozilla.org/integration/mozilla-inbound//file/60ba1893ecf3690dcee0be7872d3de2ca26c557e/taskcluster/ci/android-stuff"
      },
      "payload": {
        "artifacts": {
          "public/android/unittest": {
            "expires": {
              "relative-datestamp": "1 year"
            },
            "path": "/home/worker/workspace/build/src/obj-firefox/gradle/build/mobile/android/app/reports/tests",
            "type": "directory"
          },
          "public/build": {
            "expires": {
              "relative-datestamp": "1 year"
            },
            "path": "/home/worker/artifacts/",
            "type": "directory"
          }
        },
        "cache": {
          "level-3-mozilla-inbound-build-android-test-workspace": "/home/worker/workspace",
          "level-3-mozilla-inbound-tc-vcs": "/home/worker/.tc-vcs",
          "tooltool-cache": "/home/worker/tooltool-cache"
        },
        "command": [
          "/bin/bash",
          "bin/build.sh"
        ],
        "env": {
          "GECKO_BASE_REPOSITORY": "https://hg.mozilla.org/mozilla-central",
          "GECKO_HEAD_REF": "60ba1893ecf3690dcee0be7872d3de2ca26c557e",
          "GECKO_HEAD_REPOSITORY": "https://hg.mozilla.org/integration/mozilla-inbound/",
          "GECKO_HEAD_REV": "60ba1893ecf3690dcee0be7872d3de2ca26c557e",
          "GRADLE_USER_HOME": "/home/worker/workspace/build/src/dotgradle",
          "MH_BRANCH": "mozilla-inbound",
          "MH_BUILD_POOL": "taskcluster",
          "MH_CUSTOM_BUILD_VARIANT_CFG": "android-test",
          "MOZHARNESS_ACTIONS": "get-secrets build multi-l10n update",
          "MOZHARNESS_CONFIG": "builds/releng_base_android_64_builds.py disable_signing.py platform_supports_post_upload_to_latest.py\n",
          "MOZHARNESS_SCRIPT": "mozharness/scripts/fx_desktop_build.py",
          "MOZ_BUILD_DATE": "20161223002227",
          "MOZ_SCM_LEVEL": "3"
        },
        "features": {
          "relengAPIProxy": true,
          "taskclusterProxy": true
        },
        "image": {
          "path": "public/image.tar.zst",
          "taskId": {
            "task-reference": "<docker-image>"
          },
          "type": "task-image"
        },
        "maxRunTime": 36000
      },
      "provisionerId": "aws-provisioner-v1",
      "routes": [
        "tc-treeherder.v2.mozilla-inbound.60ba1893ecf3690dcee0be7872d3de2ca26c557e.85087",
        "tc-treeherder-stage.v2.mozilla-inbound.60ba1893ecf3690dcee0be7872d3de2ca26c557e.85087"
      ],
      "scopes": [
        "docker-worker:relengapi-proxy:tooltool.download.internal",
        "docker-worker:relengapi-proxy:tooltool.download.public",
        "secrets:get:project/releng/gecko/build/level-3/*",
        "docker-worker:cache:tooltool-cache",
        "docker-worker:cache:level-3-mozilla-inbound-tc-vcs",
        "docker-worker:cache:level-3-mozilla-inbound-build-android-test-workspace"
      ],
      "tags": {
        "createdForUser": "rbarker@mozilla.com"
      },
      "workerType": "gecko-3-b-android"
    }
  },
Flags: needinfo?(armenzg)
android-stuff is a kind - https://dxr.mozilla.org/mozilla-central/source/taskcluster/ci/android-stuff  You're right that there is a test in that kind that has label `android-test`.  Add that to the list of cleanups to make in that kind!  Does that resolve the question about duplicate descriptions?  (Maybe you meant label there?  I don't think we should be doing anything with descriptions except showing them to users, and certainly not matching on them)


As for what SETA pays attention to, first, why not pay attention to everything and just consider it important unless otherwise configured?

If that doesn't work, perhaps it makes sense to add a `task.extra.seta` section for seta-eligible tasks, and have the SETA service consume that information.

In any case, I don't want to do any kind of substring matching on labels -- that's fragile and will most definitely fail.  For example, I'm working on renaming the 'desktop-test' and 'android-test' kinds right now.  Of course, that change will take months to merge to all branches, so in the interim you'll have different labels across branches.  Any external service like SETA is going to need to deal robustly with such variance.
Component: Task Configuration → Discussion
Summary: Prevent tasks from acquiring generic descriptions → Questions about labels, descriptions, SETA, etc.
the problem with SETA is that we need to look at jobs that catch failures- and if we determine that the linting jobs don't catch failures (that are marked with fixed by commit), then we won't run them all the time, or if we do find they are high value we will run them every time even though the code that triggered it won't be necessary.

If we put something in tree or in the task configuration for seta_should_skip_this, then SETA would never know about it- we would need to do some filtering logic in seta and somehow get the data from in-tree.

Basically SETA is designed to work on unittests, not linting, docs, new tool compilation tasks, builds, or even performance tasks.
The in-tree code only consults SETA for unittests, so whatever SETA decided about linting jobs, for example, wouldn't have any impact on what tasks run.

> If we put something in tree or in the task configuration for seta_should_skip_this, then SETA would never know about it

why not?
SETA would need to ignore all treeherder information from tasks that had this information prior to calculating- so this couldn't be done at optimization time.

My thought was that SETA would need to pull from the in-tree configs, write a parser, and update itself properly- maybe even store extra fields in the database.
> SETA would need to ignore all treeherder information from tasks that had this information prior to calculating- so this couldn't be done at optimization time.

I'm still confused -- which information?

SETA can pull from in-tree, but which version of in-tree?  There is a multi-month spread of different in-tree taskgraph generation versions running simultaneously.
I think seta needs in-tree to get the extra attributes to use or ignore a given task.  is there another way this could get included into SETA from the data it gets from treeherder?
We can have this convo on IRC.

Runnable jobs consumes artifacts from the gecko decision task and can we can get that information as attributes.
oh, that would work
I think we can close this bug for now.
Once we work on SETA for a manifest based Treeherder we will need to add more SETA logic in tree and less on TH.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → INVALID
Product: Taskcluster → Taskcluster Graveyard
You need to log in before you can comment on or make changes to this bug.