Open Bug 2090689 Opened 12 days ago Updated 1 day ago

Make agents load the matching skill before they edit the files it covers

Categories

(Developer Infrastructure :: AI for Development, task)

task

Tracking

(Not tracked)

ASSIGNED

People

(Reporter: dao, Assigned: dao)

References

(Blocks 2 open bugs)

Details

(Keywords: leave-open)

Attachments

(4 external files, 1 obsolete file)

A skill's description does not reliably get it loaded before an edit it covers: an agent that judges the edit simple skips the skill. One agent deleted a line from button.tokens.json without desktop-theme-tokens, missed New Tab's _tokens.scss until the build rewrote it, and was value-neutral only by luck.

Proposal: one generic PreToolUse hook in .claude/settings.json on Edit|Write, reading a table of path patterns and skill names. On a matching edit it checks the session transcript for a load of that skill and, if there is none, denies the edit with a reason naming the skill. Later edits pass. Each skill family adds its row on its own bug; the first sends *.tokens.json, the generated tokens-*.css and New Tab's _tokens.scss to desktop-theme-tokens.

In a trial of fresh sessions on tasks that reach a token edit midway, the current description let the edit come first once in 15 runs; with the hook none did, and its one denial sent the agent to the skill before it returned to the file.

The hook reaches Claude Code only, fires at edit time and not during review.

Attachment #9677347 - Attachment description: Bug 2090689 - WIP: Make agents load the matching skill before they edit the files it covers. DONTBUILD → WIP: Bug 2090689 - Make agents load the matching skill before they edit the files it covers. DONTBUILD
Assignee: nobody → dao+bmo
Status: NEW → ASSIGNED
Attachment #9677347 - Attachment description: WIP: Bug 2090689 - Make agents load the matching skill before they edit the files it covers. DONTBUILD → Bug 2090689 - Make agents load the matching skill before they edit the files it covers. r?#ai4dev-reviewers DONTBUILD
See Also: → 2093951

In a trial of fresh Opus sessions, the parent's description loaded a theme skill on as many theme reviews, open theme bugs and screenshot requests as the four subject descriptions did, and on none of the non-theme bugs. It saves about 1,100 bytes of skill descriptions in every session. Harnesses reading .agents/skills ignore skillOverrides and still list the subject descriptions.

A Bash command counts as an edit of the files it visibly writes (a redirection, tee, sed -i, perl -i, cp, mv); a script that writes a file itself, or a cd earlier in the command, gets through unchecked.

Attachment #9677347 - Attachment is obsolete: true
Blocks: 2094544
Keywords: leave-open
Pushed by dgottwald@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/04d86d4ddc06 https://hg.mozilla.org/integration/autoland/rev/156f334ffc47 Add a desktop-theme skill that routes to the theme skills, and list those by name only. r=ai4dev-reviewers,padenot DONTBUILD https://github.com/mozilla-firefox/firefox/commit/1530f63e34c8 https://hg.mozilla.org/integration/autoland/rev/a05a4a362761 Point agents at the desktop-theme skill from browser/themes/ and toolkit/themes/. r=ai4dev-reviewers,padenot DONTBUILD
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: