Typing must never wait on heavy work, so heavy work runs on other threads. But in Chez two threads changing one object (a hashtable, a port) can corrupt memory and crash Tuna, and Chez cannot stop a thread from outside. Earlier designs let plugin code run on other threads and tried to check, by reading the code, what it could touch; the checks leaked in ordinary code. So the rule is the one Racket uses for its places: a worker is compiled apart from its plugin, so it cannot name anything of the plugin, another plugin or the editor. It gets plain data in and gives plain data back.
The rule to remember: start a job for each change, for: what it is about, and you never see a stale result.
(define-worker (name input) body …)core
Where. At the top level of plugin.ss, not inside another form. Also allowed in M-:.
Does. Declares heavy work. name is then a worker you pass to start-job. The body sees only input and a fixed vocabulary: R6RS Scheme without exit; a reviewed list of Chez procedures that touch nothing shared between threads (formatting, sorting, string ports, fixnum and flonum arithmetic, immutable vectors and strings, paths, directory listing); the persistent map and the plain-data library; and post, next-input and sleep. Code it needs from the plugin comes in with include.
Underneath. The build writes the body into a Chez library of its own that imports only that vocabulary. If the body names anything else, Chez’s own expander refuses it. Each job calls the worker afresh, so its internal definitions are new each time. Not reachable, by being absent: eval, environments, threads and mutexes, timers, the foreign-function interface, exit.
Errors. At build: the worker scan at …/plugin.ss:4 uses own-scan, which a worker cannot reach: it is a definition of this plugin; pass what it needs in its input. A define-worker not at top level: a worker is defined at the top level of plugin.ss.
Tuna · highlight/plugin.ss (part)
(define-worker(find-hello text)(post (let loop ([i 0][acc '()])(if(> (+ i 5)(string-length text))(reverse acc)(loop (+ i 1)(if(string=? (substring text i (+ i 5))"hello")(cons (list i (+ i 5)'highlight) acc)
acc))))))
Emacs
;; Emacs's nearest: a subprocess, which also shares;; nothing and talks through text.(make-process :name"scan":command'("my-scanner"):filter#'my-scan-filter)
Where Tuna differs. Emacs Lisp threads run one at a time and share everything; real parallel work in Emacs means a subprocess and text through a pipe. A Tuna worker runs truly in parallel, in the same process, sharing read-only data without copying it, and still cannot touch editor state.
Does. Runs worker on input on a new thread. Each value the worker posts is passed to on-result, on the editor thread, in the order posted. With for: k, a running job of this owner for the same k (compared with equal?) is cancelled and its waiting results dropped: a newer job for a buffer replaces the older one.
Underneath.
The input is walked once. Read-only strings, bytevectors, vectors and maps are shared without copying, and remembered, so a 100,000-name vector is checked once (1 ms the first time, about 20 µs after). Mutable strings and vectors are copied as read-only. Lists are always copied, since Chez has no read-only pairs. Anything else is refused.
A thread is started for the job (about 15 µs). It is not a pool, because workers that wait on input (a terminal’s reader) would hold a pool’s threads.
The job belongs to the run that started it, or to the caller under as-caller. When that run ends, its jobs are cancelled without waiting, and results that arrive after are dropped.
for: keys are compared only among the owner’s own jobs, so for: id is enough; two plugins’ jobs for the same buffer do not meet.
If the worker raises an error, it is reported with the worker’s name.
Errors.start-job: #<eqv hashtable> is not plain data; start-job: … is not a worker.
Stale results. If every change starts a job for: the same thing, only the newest job’s results are ever applied, so no check is needed. Work that is not restarted on every change must check, in on-result, that what it was for is still current: the completion popup checks its prompt is still open.
Tuna · highlight/plugin.ss (part)
(define(highlight! id)(start-job find-hello (buffers:text id)(lambda(spans)(buffers:set-faces! id spans))for: id))
Does. Hands one result to the editor thread, where the job’s on-result applies it. Every post arrives, in order.
Underneath. The value is checked and copied as plain data in the worker’s own thread, into slots the editor thread allocated, which keeps garbage collection fast. Once 1,000 of this job’s results are waiting, post waits, so a flooding worker cannot eat memory. A cancelled job’s post stops the job.
Errors.post: … is not plain data; outside a worker, post: only a worker can post.
Tuna
(post (best-matches names query 50))
Emacs
;; In the subprocess, printing is posting:;; print(json.dumps(matches), flush=True);; and Emacs's filter receives it:(defun my-filter (proc text)(my-apply (json-read-from-string text)))
(next-input) → valueworker
Does. Waits for the next value sent to this job with job-send!, and returns it. Lets one long-lived job serve many requests: a drawing worker gets each frame this way.
Underneath. Wakes and stops the job if it is cancelled while waiting.
Errors.next-input: only a worker can wait for input.
Does. Cancels the job. Its waiting results are dropped.
Underneath. Chez cannot stop another thread, but each thread has its own timer. A job’s thread sets its timer to fire every 20,000 ticks (roughly that many procedure calls); once the job is cancelled, the next tick jumps out of the worker to the job’s start, past any guard, and the thread ends. An endless loop that never checks anything stopped 0.28 ms after the cancel in the prototype. The jump never happens while the job holds the core’s internal lock.