Use Buttress for inference

BRICKS Foundation can offload LLM, MLX, speech-to-text, text-to-speech, and Vector Store embedding work to a BRICKS Buttress server on the same LAN. The brick’s generator API stays the same; the device transparently delegates the work to the server when it would run faster (or fit at all) there.

​

Requirements

• BRICKS Foundation 2.25 or later
• A Buttress server reachable on the LAN — see Buttress installation
• The server is bound to the same workspace as the device — see Workspace binding

​

How it works

1. When the device’s workspace has at least one bound Buttress server, the launcher starts a discovery manager and obtains a workspace-scoped JWT.
2. The manager listens for UDP announcements on port 8089 and pools every server whose serverId is on the workspace’s bound list.
3. Each LLM, MLX, Speech-to-Text (GGML), Text-to-Speech (GGML), or Vector Store brick reads its Buttress (Remote Inference) group and either picks a server automatically or uses a manually configured URL. The Speech-to-Text (ONNX) and Text-to-Speech (ONNX) Generators discover a capable server automatically.
4. The capability comparison decides whether to run locally, remotely, or either; the strategy you pick on the brick decides how that recommendation is interpreted.

If the launcher can’t reach a server (no LAN, server offline, workspace mismatch), it falls back to local execution — unless you explicitly turned that off.

​

Configure offloading on a brick

In BRICKS Controller > Config Editor, open any LLM, MLX, Speech-to-Text (GGML), Text-to-Speech (GGML), or Vector Store brick. The Buttress (Remote Inference) property group appears under Connection. The Speech-to-Text (ONNX) and Text-to-Speech (ONNX) Generators do not expose this group yet — they discover a capable server on the LAN automatically.

​

Strategies

prefer-buttress is the default because Foundation devices that opt into Buttress almost always do so because the local hardware is not the fastest path.

​

Fallback

Pick use-local if you want the brick to keep working when the LAN drops; pick no-op if you would rather see a clear failure than silently consume battery on a model the device can’t handle.

​

Vector Store

A Vector Store brick offloads the embedding model and the tokenizer, so a device can index and search documents without downloading either GGUF. Only the GGML source is offloaded — with the OpenAI Compatible source the brick calls that API directly and Buttress never enters the picture. The brick asks for a ggml-llm generator, the same backend type LLM bricks use, so auto-discovery picks any server that hosts one. The server then loads the brick’s own embedding model in embedding mode, downloading it if it does not have it yet. Give the Vector Store its own embedding model — pointing it at the chat model a nearby LLM brick uses does not work. Pick the embedding and tokenizer models from the Hugging Face picker rather than pasting a bare model URL. Buttress needs a repo id, and a brick that only has a URL silently stays local. What runs remotely depends on the tokenizer: Embeddings written by the remote model carry the same model description as locally produced ones, so an index built on-device stays searchable after you switch the brick to Buttress, and back, as long as the model is the same.

​

Text-to-Speech (GGML)

A Text-to-Speech (GGML) brick offloads synthesis to the server’s ggml-tts generator, which loads the same backbone and codec / vocoder pair the brick would load locally. The server returns a WAV, and the device downloads it into the brick’s own audio cache — so the local cache-hit and cleanup behavior is the same whether the audio was synthesized on-device or remotely. Point Model URL and Vocoder URL at Hugging Face resolve URLs, or pick them with the Hugging Face picker. The brick recovers the repo id from the URL to tell the server what to load; a model URL that is not a Hugging Face resolve URL keeps the brick on local inference. Before offloading a non-OuteTTS model, the brick checks that the server advertises that model’s family — OuteTTS, Soprano, NeuTTS, CSM, Qwen3-TTS, MOSS-TTSD, MOSS-TTS-Realtime, Chatterbox, or BlueMagpie. If it doesn’t, the brick stays local rather than returning audio that says the wrong thing. This check runs on every strategy, including prefer-buttress.

​

When the workspace changes

If the device’s workspace changes — for example, an admin reassigns it from BRICKS Controller — the launcher:

1. Stops the active Buttress manager and closes any open WebSocket connections.
2. Discards the cached access token.
3. Starts a new manager with the new workspace’s bound-server list and a freshly issued token.

In-flight generators that were authed against the old workspace error out cleanly rather than entering an infinite reconnect loop.

​

Audio uploads (STT)

Speech-to-text transcription needs the audio file on the server. The brick uploads to POST /buttress/upload over HTTPS, and the server stores the file in the temp directory configured by [server] temp_file_dir (default <os-tmpdir>/.buttress). After transcription, the file is auto-cleaned along with the rest of the session’s temp files. Text-to-speech runs the other direction: the server returns the synthesized audio as a WAV file, and repeated phrases are served instantly from the server’s on-disk output cache.

​

Troubleshooting

​

Related