Config TOML

XERJ reads one TOML file. Every key has a production-ready default, so the smallest working config is an empty file. The table below is the full surface; the sections below it walk each group with a runnable example.

Paths for the config are, in precedence order: --config /path/to/xerj.toml on the command line, then /etc/xerj/xerj.toml, then ./xerj.toml in the working directory. If none exist, the full default is used.

Full key table

[server]

Network listeners and the data directory. Most deployments only touch data_dir and the bind address.

[server]
rest_port      = 8080            # native /v1/* API
es_compat_port = 9200            # ES wire-compatible API
grpc_port      = 8081            # reserved
bind_address   = "0.0.0.0"       # listen on all interfaces
data_dir       = "/var/lib/xerj" # absolute path is strongly recommended

[auth]

Static API-key authentication. The first-run admin key is written to <data_dir>/admin.key; subsequent starts reuse it. Clients pass Authorization: ApiKey <key> on every request.

[auth]
enabled       = true
admin_api_key = ""               # blank → auto-generated on first run

# Or provide your own:
# admin_api_key = "ak_live_c8f9a4…"

[tls]

TLS termination at the server. In Kubernetes or behind a load balancer, leave this off and terminate at the proxy instead — one place to rotate certs, one place to log handshakes.

[tls]
enabled   = true
cert_path = "/etc/xerj/certs/server.crt"
key_path  = "/etc/xerj/certs/server.key"

[storage]

The WAL and flush tuning. wal_sync is the durability knob everyone looks for — pick "sync" for financial/compliance workloads, "batched" for everything else, and "async" only in benchmarks.

[storage]
wal_sync            = "batched"
wal_batch_ms        = 100        # fsync every 100 ms
wal_max_size_mb     = 512        # roll WAL every 512 MiB
flush_size_mb       = 256        # flush memtable at 256 MiB
flush_interval_secs = 30

[merge]

Segment compaction. size_tiered is the right default: it merges same-size segments, which is cheap and write-optimal. Switch to log_structured if your reads are doing a lot of full-range scans.

[merge]
strategy           = "size_tiered"
min_segments       = 10
max_segment_mb     = 5120        # 5 GiB cap
io_rate_mb_per_sec = 100         # don't starve queries
max_concurrent     = 2           # bump on NVMe

[compression]

See Compression for the encoding catalog. This section picks the outer block codec only — the inner per-column encodings are chosen automatically at write time.

[compression]
enabled         = true
level           = "balanced"     # LZ4 / Zstd L3 / Zstd L19
block_size_docs = 128

[fts]

Default analyzer applied to untyped text fields. Override per-field in the mapping when creating an index. See Analyzers for the built-ins.

[fts]
default_analyzer = "standard"    # unicode words + lowercase

[vector]

HNSW index tuning and the default quantization scheme. These defaults are chosen so a 1 M × 768-dim index fits in a few GiB of RAM with ~99% recall. Override per-field in the mapping for exotic cases.

[vector]
default_metric         = "cosine"
hnsw_m                 = 16
hnsw_ef_construction   = 200
hnsw_ef_search         = 100
default_quantization   = "scalar8"   # 4× RAM saving
hnsw_offload_threshold = 1000000     # auto-scalar4 past 1 M vectors
max_dimensions         = 16384

[logs]

Time-series retention. Log indices are sliced into partitions of time_partition width so retention prunes are O(partitions), not O(documents).

[logs]
retention_days = 30              # keep 30 days
time_partition = "1h"            # 1-hour partitions

[embedding]

Delegates vector generation to an OpenAI-compatible endpoint. Leave default_endpoint empty if clients provide vectors themselves. Token limits are model-specific; the chunker in the ai crate splits long documents to fit the model's window.

[embedding]
# OpenAI:
default_endpoint = "https://api.openai.com/v1/embeddings"
default_model    = "text-embedding-3-small"
batch_size       = 64
timeout_ms       = 5000

# Or a local Ollama:
# default_endpoint = "http://localhost:11434/v1/embeddings"
# default_model    = "nomic-embed-text"

[limits]

Hard caps to protect the server from runaway queries and mapping explosions. Lower these on shared nodes, raise max_query_memory_mb for aggregation-heavy workloads.

[limits]
max_query_memory_mb     = 512
max_concurrent_searches = 64
max_fields_per_index    = 500

[indexing]

Turbo mode knobs. Turbo is opt-in per-request via POST /v1/indices/:name/turbo-ingest or the X-Turbo: true header on _bulk; these settings only apply when turbo is active.

[indexing]
turbo_batch_size    = 2000
turbo_parallel      = true
turbo_fast_analyzer = false      # true only if recall doesn't matter

[cluster]

Multi-node mode. Default is off — single-node doesn't need a consensus layer. When enabled, the embedded Raft implementation replicates metadata only (schemas, shard assignments, node roster). See Clustering for the full story.

[cluster]
enabled = true
port    = 9300                   # intra-cluster gRPC + Raft
peers   = [
  "a=10.0.0.11:9300",
  "b=10.0.0.12:9300",
  "c=10.0.0.13:9300",
]
tick_ms = 50

Source · engine/xerj.default.toml · engine/crates/common/src/config.rs