Configuration

The Buttress server reads a single TOML file passed via --config. Every section is optional; omit it to use defaults.

Minimal example

[server]
port = 2080

[[generators]]
type = "ggml-llm"
[generators.model]
repo_id = "ggml-org/gpt-oss-20b-GGUF"
quantization = "mxfp4"
n_ctx = 12800

Top-level sections

Section	Purpose
`[env]`	Environment variables exported into the process only if not already set
`[server]`	HTTP/WebSocket listener (port, log level, body limits)
`[runtime]`	Global defaults shared by every generator
`[runtime.session_cache]`	KV-cache reuse store for `ggml-llm`
`[autodiscover]`	LAN UDP / HTTP discovery toggles
`[openai_compat]`	Enable OpenAI-compatible HTTP routes
`[anthropic_messages]`	Enable Anthropic-compatible HTTP routes
`[functions]`	Enable local functions — server-side `.ts` / `.js` files exposed as MCP tools and HTTP endpoints
`[[generators]]`	Array of generator instances — one entry per loaded model

`[server]`

Key	Type	Default	Description
`id`	string	`buttress-<machineId>`	Stable server id used for binding and discovery
`name`	string	`Buttress Server (<short id>)`	Friendly name shown in BRICKS Controller
`port`	number	`2080`	HTTP/WebSocket port (overridden by `--port`)
`log_level`	string	unset	One of `debug`, `info`, `warn`, `error`
`max_body_size`	number or string	`"50MB"`	Max upload size; accepts `"100MB"`, `"1GB"`, or raw bytes
`session_timeout`	number or string	`60000`	WebSocket idle timeout in ms; also accepts `"1m"`, `"30s"`
`temp_file_dir`	string	`$TMPDIR/.buttress`	Directory for STT audio uploads and other temp files

`[runtime]`

Global defaults shared by every generator. Per-generator values under [generators.model] win; otherwise these defaults apply.

[runtime]
cache_dir = "~/.buttress/models"
huggingface_token = "hf_..."
n_gpu_layers = "auto"

Key	Type	Description
`cache_dir`	string	Model and metadata cache root (default `~/.buttress/models`)
`huggingface_token`	string	Hugging Face auth token; falls back to `$HUGGINGFACE_TOKEN`. Applied to all backends regardless of variable name
`http_headers`	table	Extra headers attached to Hugging Face / HTTP downloads
`context_release_delay_ms`	number	Idle time before unloading a context (default `10000`; `0` = immediate)
`prefer_variants`	string[]	Override backend variant probe order (ggml backends)
`n_threads`	number	CPU thread count
`n_ctx`	number	Context window (per-model value wins; auto-capped at training context)
`n_gpu_layers`	number or `"auto"`	Layers offloaded to GPU (default `"auto"`)
`n_batch`	number	Prompt batch size. Note: the model layer defaults `n_batch` to `512`, which shadows the runtime value unless `n_batch` is set explicitly under `[generators.model]`
`n_ubatch`	number	Prompt micro-batch size
`n_parallel`	number	Parallel sequences (default `4`)
`n_cpu_moe`	number	MoE expert layers offloaded to CPU
`flash_attn_type`	string	`"on"`, `"off"`, or `"auto"`. Default is GPU-conditional: `"auto"` when a GPU backend is selected, `"off"` on CPU
`cache_type_k`, `cache_type_v`	string	KV-cache dtype (`f16`, `f32`, `q8_0`, `q4_0`, …)
`kv_unified`	boolean	Use a unified KV cache across sequences
`swa_full`	boolean	Materialize full attention even for sliding-window layers
`ctx_shift`	boolean	Allow llama.cpp’s rolling context shift
`use_mmap`, `use_mlock`	boolean	Memory-mapping / locking
`no_extra_bufts`	boolean	Disable extra compute buffer types
`cpu_mask`, `cpu_strict`	string / boolean	CPU affinity (advanced)
`devices`	string[]	Restrict to specific GGML devices
Speculative keys	various	`speculative`, `spec_type`, `spec_draft_n_max`, `spec_draft_n_min`, `spec_draft_p_min`, `spec_draft_p_split`

`[runtime.session_cache]`

For ggml-llm generators, the server can persist KV cache state between requests so that a follow-up completion sharing a prompt prefix skips prompt processing.

[runtime.session_cache]
enabled = true
max_size_bytes = "10GB"
max_entries = 1000

Key	Default	Description
`enabled`	`true`	Enable persistent KV cache
`max_size_bytes`	`"10GB"`	Total disk budget; accepts `"500MB"`, `"50GB"`, or a number
`max_entries`	`1000`	Max number of cached states (LRU eviction)

Cache files are stored under {cache_dir}/.session-state-cache/. mlx-llm keeps a separate session cache under {cache_dir}/mlx-session-cache/, configured independently per generator.

`[[generators]]`

Each [[generators]] block declares one model the server can host. Repeat the block to host multiple. Every block has a type, an optional [generators.backend] table, and a [generators.model] table. type is one of ggml-llm, ggml-stt, ggml-tts, mlx-llm, onnx-stt, or onnx-tts.

[[generators]]
type = "ggml-llm"

[generators.backend]
# backend selection and resource planning

[generators.model]
repo_id = "..."
# model identity and runtime overrides

Common `[generators.model]` keys

Shared by all generator types (ggml-llm, ggml-stt, ggml-tts, mlx-llm, onnx-stt, onnx-tts):

Key	Type	Description
`repo_id` (required)	string	Hugging Face repo (`org/repo`)
`revision`	string	Default `"main"`
`download`	boolean	Pre-download at server startup (default `false`)

Honored by ggml-llm, ggml-stt, and ggml-tts only. mlx-llm derives quantization from the repo itself, and the ONNX backends (onnx-stt, onnx-tts) select weights with dtype instead — so all three ignore these:

Key	Type	Description
`filename`	string	Pin a specific artifact in the repo
`url`	string	Direct download URL (skips manifest lookup)
`quantization`	string	Preferred quant tag — e.g. `q4_0`, `q8_0`, `mxfp4`
`preferred_quantizations`	string[]	Ordered fallback list when `quantization` doesn’t match (alias: `quantizations`)
`allow_local_file`	boolean	Required to use `local_path` or `mmproj_local_path`
`local_path`	string	Use a local file as the load path. Repo metadata is still resolved from Hugging Face, so `repo_id` is still required
`api_base`, `base_url`	string	Override Hugging Face API / blob hosts (mirrors or proxies)

`ggml-llm` (llama.cpp / GGUF)

[[generators]]
type = "ggml-llm"

[generators.backend]
variant_preference = ["cuda", "vulkan", "default"]
gpu_memory_fraction = 0.95

[generators.model]
repo_id = "ggml-org/gpt-oss-20b-GGUF"
quantization = "mxfp4"
n_ctx = 12800
download = true

[generators.backend] only controls backend selection and resource planning. Runtime overrides (n_ctx, n_gpu_layers, flash_attn_type, etc.) go under [generators.model]. [generators.backend]

Key	Type	Default	Description
`variant`	string	auto	Force `cuda`, `vulkan`, `snapdragon`, or `default`
`variant_preference`	string[]	`["cuda", "vulkan", "snapdragon", "default"]`	Probe order when `variant` is unset
`gpu_memory_fraction`	number	`0.85`	Max GPU fraction the hardware guardrails may plan against
`cpu_memory_fraction`	number	`0.5`	Max RAM fraction for CPU-side buffers

[generators.model] — in addition to the common ggml keys above, every [runtime] key can be overridden per-generator: n_ctx, n_gpu_layers, n_batch, n_ubatch, n_threads, n_parallel, n_cpu_moe, flash_attn_type, cache_type_k, cache_type_v, kv_unified, swa_full, ctx_shift, use_mmap, use_mlock, no_extra_bufts, cpu_mask, cpu_strict, devices. Embedding models

Key	Type	Default	Description
`embedding`	boolean	`false`	Load this model in llama.cpp’s embedding mode
`pooling_type`	string	—	Pooling override such as `mean`; unset leaves the model’s own default
`vocab_only`	boolean	`false`	Load the vocabulary without the weights — enough to tokenize and detokenize, and it occupies no model memory

A dedicated embedding model needs embedding = true. context.buttress.embedding in a local function rejects any generator that does not set it, so give the embedding model its own [[generators]] block rather than reusing the chat model’s. An embedding context runs a single native sequence, so parallel decoding slots do not apply to it. A vocabulary-only context has no compute backend at all, so it serves tokenizer calls only. The same repo can be hosted more than once in different modes — chat, embedding, vocabulary-only. Each mode is a separate entry in the generator registry, so one consumer’s context never inherits another’s configuration. A Vector Store brick drives both of these keys remotely, without any of them appearing in this file. Multimodal (mtmd) — auto-downloads the matching mmproj-*.gguf from the same repo:

Key	Type	Description
`enable_mtmd`	boolean	Default `false`
`mmproj_filename`	string	Pin a specific projector file
`mmproj_url`	string	Direct URL override
`mmproj_local_path`	string	Local projector file (requires `allow_local_file = true`)
`mmproj_use_gpu`	boolean	Unset = auto (true when `n_gpu_layers > 0`)
`mmproj_image_min_tokens`	number	Min visual tokens (dynamic-resolution models; `-1` = unset)
`mmproj_image_max_tokens`	number	Max visual tokens (`-1` = unset)

Speculative decoding

Key	Type	Description
`speculative`	string	Draft model identifier
`spec_type`	string	Strategy (backend-defined)
`spec_draft_n_max`	int	Max drafted tokens per step
`spec_draft_n_min`	int	Min drafted tokens
`spec_draft_p_min`	number	Min acceptance probability
`spec_draft_p_split`	number	Split threshold

`ggml-stt` (whisper.cpp)

[[generators]]
type = "ggml-stt"

[generators.backend]
variant_preference = ["cuda", "vulkan", "default"]

[generators.model]
repo_id = "BricksDisplay/whisper-ggml"
filename = "ggml-large-v3-turbo-q8_0.bin"
use_gpu = true
use_flash_attn = "on"
download = true

[generators.backend]

Key	Type	Default	Description
`variant`	string	auto	Force `cuda`, `vulkan`, or `default`
`variant_preference`	string[]	`["cuda", "vulkan", "default"]`	Probe order
`gpu_memory_fraction`	number	`0.85`
`cpu_memory_fraction`	number	`0.5`

[generators.model] — in addition to the common ggml keys above:

Key	Type	Default	Description
`repo_id`	string	`"BricksDisplay/whisper-ggml"`	Defaulted (unlike `ggml-llm`)
`preferred_quantizations`	string[]	`["q8_0", <no-quant>, "q5_1"]`	Default fallback chain
`use_gpu`	boolean	`true`	Set to `false` to force CPU even when a GPU is available
`use_flash_attn`	string or boolean	`"auto"`	`"on"`, `"off"`, or `"auto"`. `true` / `false` are accepted as shortcuts. `"auto"` enables flash-attn when a GPU is in use

Runtime extras — under [runtime] for ggml-stt only:

Key	Type	Description
`max_threads`	number	Caps the whisper.cpp thread count

`ggml-tts` (llama.cpp + codec.cpp)

[[generators]]
type = "ggml-tts"

[generators.backend]
variant_preference = ["cuda", "vulkan", "default"]

[generators.model]
repo_id = "OuteAI/OuteTTS-1.0-0.6B-GGUF"
quantization = "q4_k_m"
vocoder_repo_id = "BricksDisplay/codec.cpp-gguf"
vocoder_filename = "ibm-research--DAC.speech.gguf"
use_gpu = true
download = true

[generators.runtime]
n_ctx = 8192
vocoder_batch_size = 4096

Speech synthesis from a GGUF backbone plus its audio codec / vocoder GGUF. Both artifacts stay resident. Output is a WAV file. The model family — OuteTTS, Soprano, NeuTTS, CSM, Qwen3-TTS, MOSS-TTSD, MOSS-TTS-Realtime, Chatterbox, or BlueMagpie — is detected from the backbone, along with the synthesis flow it needs, so there is nothing to configure per family. The server advertises the list it can drive, and a device asking for a family the server does not advertise stays local instead of getting back audio that says the wrong thing.

No phonemizer is wired into this backend, so NeuTTS receives raw text rather than phonemes — the same limitation the on-device generator has.

[generators.backend]

Key	Type	Default	Description
`variant`	string	auto	Force `cuda`, `vulkan`, or `default`
`variant_preference`	string[]	`["cuda", "vulkan", "default"]`	Probe order
`gpu_memory_fraction`	number	`0.85`
`cpu_memory_fraction`	number	`0.5`

[generators.model] — in addition to the common ggml keys above:

Key	Type	Default	Description
`preferred_quantizations`	string[]	`["q4_k_m", "q8_0", "fp16"]`	Default fallback chain
`vocoder_repo_id`	string	falls back to `repo_id`	Repo holding the codec GGUF
`vocoder_filename`	string	auto	Pin a specific codec artifact
`vocoder_url`	string	—	Direct codec URL (skips manifest lookup)
`vocoder_quantization`	string	—	Preferred quant tag for the codec
`vocoder_revision`	string	falls back to `revision`	Git revision for the codec repo
`vocoder_local_path`	string	—	Local codec file (requires `allow_local_file = true`)
`use_gpu`	boolean	`true`	Set to `false` to force CPU even when a GPU is available
`n_gpu_layers`	number	`99`	Forced to `0` when the GPU is unused
`sample_rate`	number	`24000`	Fallback only — the real rate is read from the loaded codec

A repo that ships the backbone and the codec together is split by filename (codec, vocoder, wavtokenizer, dac, mimi). A repo hosting several unrelated codecs cannot be resolved by quantization preference alone, so set vocoder_filename explicitly for those. Runtime extras — under [generators.runtime] for ggml-tts:

Key	Type	Default	Description
`n_ctx`	number	`8192`	Backbone context size
`n_batch`	number	`8192`	Backbone batch size
`n_ubatch`	number	`512`	Backbone micro-batch size
`vocoder_batch_size`	number	`4096`	Batch size for the codec context
`max_threads`	number	auto	Caps the llama.cpp thread count
`context_release_delay_ms`	number	`10000`	How long an idle context stays loaded before release

Output cache — synthesized WAVs are cached on disk under {cache_dir}/.tts-cache/, keyed by text + model + options, so repeated phrases return instantly. Configure it under [generators.runtime.output_cache]:

Key	Default	Description
`enabled`	`true`	Enable the persistent output cache
`max_size_bytes`	`2147483648`	Total disk budget (2 GB)
`max_entries`	`5000`	Max number of cached clips (LRU eviction)

`mlx-llm` (Apple Silicon)

[[generators]]
type = "mlx-llm"

[generators.model]
repo_id = "mlx-community/Qwen2.5-VL-3B-Instruct-4bit"
vlm = true
download = true

There is no [generators.backend] section for mlx-llm. On first use, the backend creates a Python virtualenv at {cache_dir}/mlx-env and installs mlx_lm_package, mlx_vlm_package, plus torch and torchvision (required by some VLM processors). If an existing venv already has mlx_vlm and torch importable, the install step is skipped. [generators.model] — common repo_id / revision / download plus:

Key	Type	Default	Description
`adapter_path`	string	—	Local LoRA adapter directory
`vlm`	`"auto"` or boolean	`"auto"`	Force VLM (`true`) vs text-only (`false`); `"auto"` infers from the repo
`tokenizer_config`	table	—	Forwarded to `mlx_lm.load(..., tokenizer_config=...)`
`model_config`	table	—	Forwarded to `mlx_lm.load(..., model_config=...)`

quantization, filename, and preferred_quantizations are not used — the MLX repo itself determines the quantization. Runtime extras — under [runtime] for mlx-llm:

Key	Type	Default	Description
`mlx_env_dir`	string	`{cache_dir}/mlx-env`	Location of the auto-managed Python venv
`mlx_lm_package`	string	`"mlx-lm==0.31.1"`	pip spec used when provisioning the venv
`mlx_vlm_package`	string	`"mlx-vlm==0.4.0"`	pip spec used when provisioning the venv

`onnx-stt` (ONNX Runtime / Whisper)

[[generators]]
type = "onnx-stt"

[generators.backend]
provider = "coreml"  # or cuda / dml / webgpu / qnn / cpu; omit for auto
provider_preference = ["cuda", "coreml", "dml", "webgpu", "cpu"]

[generators.model]
repo_id = "onnx-community/whisper-base"
dtype = "auto"
download = true

Automatic speech recognition with Whisper ONNX models. Weights download from the repo’s onnx/ subfolder on Hugging Face into {cache_dir}/{owner}/{repo}/, and the backend streams partial transcription results before returning the final text. [generators.backend]

Key	Type	Default	Description
`provider`	string	auto	Execution provider: `cuda`, `coreml`, `dml`, `webgpu`, `qnn`, or `cpu`. An explicitly configured provider is honored when available; an unavailable one falls through to the next candidate with a warning
`provider_preference`	string[]	`["cuda", "coreml", "dml", "webgpu", "cpu"]`	Probe order when `provider` is unset. `qnn` is accepted when requested explicitly but is not in the default preference
`gpu_memory_fraction`	number	`0.85`	Max GPU fraction the hardware guardrails may plan against
`cpu_memory_fraction`	number	`0.5`	Max RAM fraction for CPU-side buffers

If none of the configured providers initialize, the generator fails to start. [generators.model]

Key	Type	Default	Description
`repo_id` (required)	string	—	Hugging Face repo, e.g. `onnx-community/whisper-base`
`revision`	string	`"main"`	Git revision
`task`	string	`"automatic-speech-recognition"`	ONNX pipeline task
`dtype`	string	`"auto"`	Weight variant: `auto`, `fp32`, `fp16`, `int8`, `uint8`, `q8`, `q4`, `q4f16`, or `bnb4`. `auto` reads the repo’s transformers.js config, falling back to `fp32`
`device`	string	auto	Device hint passed to ONNX Runtime
`base_url`	string	`"https://huggingface.co"`	Download host (mirrors or proxies)
`download`	boolean	`false`	Pre-download at server startup

Runtime extras — under [generators.runtime]:

Key	Type	Description
`prefer_providers`	string[]	Override the execution-provider probe order for this generator

The global [runtime] keys cache_dir, huggingface_token, and http_headers also apply.

`onnx-tts` (ONNX Runtime / Kokoro, VITS, SpeechT5)

[[generators]]
type = "onnx-tts"

[generators.backend]
provider = "coreml"  # or cuda / dml / webgpu / qnn / cpu; omit for auto
provider_preference = ["cuda", "coreml", "dml", "webgpu", "cpu"]

[generators.model]
repo_id = "..."  # a Kokoro, VITS, Bert-VITS2, or SpeechT5 ONNX repo
dtype = "auto"
download = true

Text-to-speech across the Kokoro, VITS / MMS-TTS, Bert-VITS2, and SpeechT5 model families. Output is a WAV file. Weights download from the repo’s onnx/ subfolder on Hugging Face into {cache_dir}/{owner}/{repo}/. [generators.backend] — same keys as onnx-stt above: provider, provider_preference, gpu_memory_fraction, cpu_memory_fraction. [generators.model]

Key	Type	Default	Description
`repo_id` (required)	string	—	Hugging Face repo of the TTS model
`revision`	string	`"main"`	Git revision
`task`	string	`"text-to-speech"`	ONNX pipeline task
`dtype`	string	`"auto"`	Weight variant: `auto`, `fp32`, `fp16`, `int8`, `uint8`, `q8`, `q4`, `q4f16`, or `bnb4`
`device`	string	auto	Device hint passed to ONNX Runtime
`base_url`	string	`"https://huggingface.co"`	Download host
`download`	boolean	`false`	Pre-download at server startup
`vocoder_repo_id`	string	—	HiFi-GAN vocoder repo. SpeechT5 only — the only family that uses a separate vocoder

Output cache — synthesized audio is cached on disk under {cache_dir}/.tts-cache/, keyed by text + model + options, so repeated phrases return instantly. Configure it under [generators.runtime.output_cache]:

Key	Default	Description
`enabled`	`true`	Enable the persistent output cache
`max_size_bytes`	`2147483648`	Total disk budget (2 GB)
`max_entries`	`5000`	Max number of cached clips (LRU eviction)

The [generators.runtime] prefer_providers key and the global [runtime] keys (cache_dir, huggingface_token, http_headers) apply as with onnx-stt.

`[autodiscover]`

The server announces itself on UDP 8089 so Foundation devices on the same LAN can find it. Auto-discovery is on by default.

[autodiscover]
[autodiscover.udp]
port = 8089

[autodiscover.udp.announcements]
enabled = true
interval = 5000

[autodiscover.udp.requests]
enabled = true
responseDelay = 100

[autodiscover.http]
enabled = true
path = "/buttress/info"
cors = true

Set autodiscover = false to disable discovery entirely. See the autodiscovery reference for protocol details.

`[env]`

Environment variables applied at startup, but only if they are not already set in the system environment. System variables and command-line exports take precedence.

[env]
HUGGINGFACE_TOKEN = "hf_..."
CUDA_VISIBLE_DEVICES = "0"

The ggml backends read HUGGINGFACE_TOKEN (not HF_TOKEN). For a single token that applies to every backend regardless of variable name, set [runtime] huggingface_token instead.

Compatibility endpoints

These endpoints are experimental. The schemas, error shapes, and CORS defaults may change.

The server can expose OpenAI- and Anthropic-compatible HTTP routes alongside the native WebSocket RPC. Each is opt-in.

[openai_compat]
enabled = true
# cors_allowed_origins = "*"

[anthropic_messages]
enabled = true
# cors_allowed_origins = ["http://localhost:3000"]

Endpoint	Config flag
`POST /oai-compat/v1/chat/completions`	`[openai_compat] enabled = true`
`GET /oai-compat/v1/models`	`[openai_compat] enabled = true`
`POST /anthropic-messages/v1/messages`	`[anthropic_messages] enabled = true`
`POST /anthropic-messages/v1/messages/count_tokens`	`[anthropic_messages] enabled = true`

You can also enable each endpoint via env var: ENABLE_OPENAI_COMPAT_ENDPOINT=1 or ENABLE_ANTHROPIC_MESSAGES_ENDPOINT=1.

`[functions]`

Local functions are experimental. The endpoints, the function file contract, and these config keys may change between releases.

Expose server-side .ts / .js files as MCP tools and HTTP endpoints. Off by default.

[functions]
enabled = true
dir = "./functions"
# default_timeout = "5m"
# hot_reload = false
# allow_unauthenticated = false
# cors_allowed_origins = ["http://localhost:3000"]

[functions.config]
# api_base = "https://example.internal"

Key	Type	Default	Description
`enabled`	boolean	`false`	Enable the `/functions` surface
`dir`	string	—	Functions directory. Required; relative paths resolve against the config file, and `~` expands to the home directory
`default_timeout`	number or string	`"5m"`	Per-call deadline in ms, or a duration string. A function’s `meta.timeout` wins
`hot_reload`	boolean	`false`	Watch the directory and reload on save, in addition to the per-call check
`allow_unauthenticated`	boolean	`false`	Let an unbound server accept calls. Without it, an unbound server rejects every call
`cors_allowed_origins`	string[] or `"*"`	—	Browser origins allowed to call functions. Empty means no browser may
`[functions.config]`	table	`{}`	Free-form values handed to every function as `context.config`

Env-var equivalents: ENABLE_FUNCTIONS_ENDPOINT=1, BUTTRESS_FUNCTIONS_DIR=<dir>, BUTTRESS_FUNCTIONS_HOT_RELOAD=1, and BUTTRESS_FUNCTIONS_ALLOW_UNAUTHENTICATED=1. See Local functions for the file contract, the injected context, the endpoints, and the security model.

Overview

Guides

Changelog

Minimal example

Top-level sections

`[server]`

`[runtime]`

`[runtime.session_cache]`

`[[generators]]`

Common `[generators.model]` keys

`ggml-llm` (llama.cpp / GGUF)

`ggml-stt` (whisper.cpp)

`ggml-tts` (llama.cpp + codec.cpp)

`mlx-llm` (Apple Silicon)

`onnx-stt` (ONNX Runtime / Whisper)

`onnx-tts` (ONNX Runtime / Kokoro, VITS, SpeechT5)

`[autodiscover]`

`[env]`

Compatibility endpoints

`[functions]`

Next steps

Workspace binding

LAN auto-discovery

​Minimal example

​Top-level sections

​[server]

​[runtime]

​[runtime.session_cache]

​[[generators]]

​Common [generators.model] keys

​ggml-llm (llama.cpp / GGUF)

​ggml-stt (whisper.cpp)

​ggml-tts (llama.cpp + codec.cpp)

​mlx-llm (Apple Silicon)

​onnx-stt (ONNX Runtime / Whisper)

​onnx-tts (ONNX Runtime / Kokoro, VITS, SpeechT5)

​[autodiscover]

​[env]

​Compatibility endpoints

​[functions]

​Next steps

Workspace binding

LAN auto-discovery

Minimal example

Top-level sections

`[server]`

`[runtime]`

`[runtime.session_cache]`

`[[generators]]`

Common `[generators.model]` keys

`ggml-llm` (llama.cpp / GGUF)

`ggml-stt` (whisper.cpp)

`ggml-tts` (llama.cpp + codec.cpp)

`mlx-llm` (Apple Silicon)

`onnx-stt` (ONNX Runtime / Whisper)

`onnx-tts` (ONNX Runtime / Kokoro, VITS, SpeechT5)

`[autodiscover]`

`[env]`

Compatibility endpoints

`[functions]`

Next steps