Open Bug 1955351 Opened 1 year ago Updated 2 months ago

[wllama] Allow graceful generation abort without terminating engine

Categories

(Core :: Machine Learning: On Device, enhancement, P3)

Product:
Core
Component:
Machine Learning: On Device
Type:
enhancement

Tracking

()

People

(Reporter: atossou, Unassigned)

References

(Blocks 2 open bugs)

Details

(Whiteboard: [genai])

Allow graceful generation abort without terminating engine

Blocks: 1955352
Component: Machine Learning: General → Machine Learning: On Device
Assignee: atossou → nobody
Status: ASSIGNED → NEW
Blocks: llama-cpp

I guess the work here is to use some kind of AbortController to stop text generation. It's unclear if we'll keep wllama or what the current state is. We should probably do a bit more work to figure out what's needed here or mark this issue as invalid.

Priority: -- → P3
You need to log in before you can comment on or make changes to this bug.