Playbook · Log analytics

Ingest Nginx, JSON, syslog, or OTLP at line rate. Query hot logs in milliseconds. Keep the volume bounded by dropping old indices yourself (see Retention). This is the workload XERJ was originally built for.

Schema

$ curl -sX PUT http://localhost:8080/v1/indices/logs \
    -H 'Content-Type: application/json' \
    -d '{
      "fields": {
        "@timestamp":  "date",
        "service":     "keyword",
        "level":       "keyword",
        "host":        "keyword",
        "trace_id":    "keyword",
        "span_id":     "keyword",
        "status":      "integer",
        "latency_ms":  "integer",
        "bytes_out":   "integer",
        "client_ip":   "ip",
        "message":     "text"
      }
    }'

Ingest options

• Raw NGINX/Apache → POST /v1/indices/logs/logs auto-detects the format.
• Structured JSON → POST /v1/indices/logs/turbo-ingest NDJSON body.
• Syslog → POST /v1/indices/logs/syslog.
• OTLP → POST /v1/indices/logs/otlp.

Tail recent errors

{
  "query": {
    "bool": {
      "filter": [
        { "term":  { "level": "error" } },
        { "range": { "@timestamp": { "gte": "now-15m" } } }
      ]
    }
  },
  "size": 100,
  "sort": [ { "@timestamp": "desc" } ]
}

Top slow endpoints · percentiles

{
  "query": { "term": { "service": "api-gateway" } },
  "aggs": {
    "by_route": {
      "terms": { "field": "url_template", "size": 20 },
      "aggs": {
        "p": { "percentiles": { "field": "latency_ms", "percents": [50, 95, 99] } }
      }
    }
  },
  "size": 0
}

Retention

Automatic retention is not implemented in this build. The [logs] retention_days and time_partition keys are accepted and nothing reads them (setting either logs a warning at startup), so log data is kept until you delete it. Write to one index per day or week and delete the indices that fall outside your window, or run _delete_by_query with a range on the timestamp field.

DELETE /logs-2026.09.01
POST /logs/_delete_by_query
{ "query": { "range": { "@timestamp": { "lt": "now-30d" } } } }

Dashboard

Open the playground and pick LOGS · OVERVIEW — nine panels with click-to-filter TopN, 1 ms date_histogram, heatmap, and drill-through to the SYSTEM · OVERVIEW dashboard.

Source · engine/crates/xerj-logs/src/lib.rs