job details API gives "FieldError: Non-relational field given in select_related: 'name'" under Django 1.10

RESOLVED FIXED

Status

Tree Management
Treeherder
P2
normal
RESOLVED FIXED
a year ago
a year ago

People

(Reporter: emorley, Assigned: emorley)

Tracking

Details

Attachments

(1 attachment)

(Assignee)

Description

a year ago
With Django 1.10, tests/webapp/api/test_job_details_api.py fails with:

FieldError: Non-relational field given in select_related: 'name'. Choices are: repository_group

This is presumably due to:
https://docs.djangoproject.com/en/1.10/releases/1.8/#select-related-now-checks-given-fields
https://docs.djangoproject.com/en/1.10/releases/1.10/#select-related-prohibits-non-relational-fields-for-nested-relations
(Assignee)

Comment 1

a year ago
Full test failure output:
https://emorley.pastebin.mozilla.org/8958198
(Assignee)

Comment 2

a year ago
The UI makes requests like:
http://localhost:8000/api/jobdetail/?job_guid=8d58d43e5834a42100c22df70077ae45fffacb20

Which returns eg:
{
    "next": null,
    "previous": null,
    "results": [
        {
            "job_id": 9574,
            "job_guid": "8d58d43e5834a42100c22df70077ae45fffacb20",
            "title": "artifact uploaded",
            "value": "resource-usage.json",
            "url": "http://mozilla-releng-blobs.s3.amazonaws.com/blobs/mozilla-inbound/sha512/174a070a6b846acb5a3b25aeba547965e7a55c1507bdf37c78402a6cbb37f6fb0ce302d247ae46e8aee0c619260a35df2c9a25fef2c3430a437b37c8c1cfc189"
        },
...

Under Django 1.9.12 (where this query doesn't error), the generated SQL as reported by debug_toolbar is:

SELECT `job_detail`.`id`, `job_detail`.`job_id`, `job_detail`.`title`, `job_detail`.`value`, `job_detail`.`url`, `job`.`id`, `job`.`repository_id`, `job`.`guid`, `job`.`project_specific_id`, `job`.`coalesced_to_guid`, `job`.`signature_id`, `job`.`build_platform_id`, `job`.`machine_platform_id`, `job`.`machine_id`, `job`.`option_collection_hash`, `job`.`job_type_id`, `job`.`product_id`, `job`.`failure_classification_id`, `job`.`who`, `job`.`reason`, `job`.`result`, `job`.`state`, `job`.`submit_time`, `job`.`start_time`, `job`.`end_time`, `job`.`last_modified`, `job`.`running_eta`, `job`.`tier`, `job`.`push_id`, `repository`.`id`, `repository`.`repository_group_id`, `repository`.`name`, `repository`.`dvcs_type`, `repository`.`url`, `repository`.`branch`, `repository`.`codebase`, `repository`.`description`, `repository`.`active_status`, `repository`.`performance_alerts_enabled` FROM `job_detail` INNER JOIN `job` ON (`job_detail`.`job_id` = `job`.`id`) INNER JOIN `repository` ON (`job`.`repository_id` = `repository`.`id`) WHERE `job`.`guid` = '8d58d43e5834a42100c22df70077ae45fffacb20' ORDER BY `job_detail`.`id` DESC LIMIT 2001

The join on the `repository` seems redundant for this case, and is presumably only helpful when filtering on repository, which is presumably a leftover from the datasource transition?
Created attachment 8823840 [details] [review]
[treeherder] mozilla:jobdetails-select-related > mozilla:master
(Assignee)

Updated

a year ago
Attachment #8823840 - Flags: review?(wlachance)
(Assignee)

Updated

a year ago
Assignee: nobody → emorley
Comment on attachment 8823840 [details] [review]
[treeherder] mozilla:jobdetails-select-related > mozilla:master

LGTM! I did not know select_related worked that way.
Attachment #8823840 - Flags: review?(wlachance) → review+
(Assignee)

Comment 5

a year ago
Yeah me neither until now -- turns out .only() and .defer() and the ones to use when wanting to reduce the number of fields returned, but the Django docs suggest rare use of them:
https://docs.djangoproject.com/en/1.10/ref/models/querysets/#django.db.models.query.QuerySet.defer
(Assignee)

Comment 6

a year ago
s/and the ones/are the ones/

Comment 7

a year ago
Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/bf1166f25e99309313e46684621d29455574fef2
Bug 1328294 - Correct select_related() usage inJobDetailViewSet

The fields listed must be relational fields - `select_related()` does
not control which fields are returned, just which tables are joined.

Under Django <=1.9 any non-relational fields are ignored, however under
Django 1.10 these cause errors of form:
"FieldError: Non-relational field given in select_related: 'name'. Choices are: repository_group"

This change is a no-op in terms of generated SQL.
(Assignee)

Updated

a year ago
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.