Context compaction¶

As a conversation grows it eventually approaches the model's context window. Rather than let a turn fail, Wallah compacts: it replaces the oldest turns with a summary while keeping a recent tail of the conversation live and verbatim. This runs automatically between turns, and you can force it at any time with /compact.

Automatic compaction follows a sawtooth: once enough new context has accrued since the last compaction (grow_by), the oldest turns are summarized down, leaving the keep_recent tail untouched. A hard ceiling (compact_at) forces a compaction regardless of the other gates, so a single very large turn cannot overflow the window. A minimum gain and a refractory period keep it from compacting too eagerly.

The summary has two parts: a deterministic reduction of the dropped turns, plus an optional model-written Observations summary. The latter maps over the dropped conversation in chunks and can run against a different (typically cheaper) model than the session itself.

Configure both from config.lua in a reconfigure handler, which fires at startup and on /reload:

agent.on('reconfigure', function()
  agent.configure_compaction {
    mode = 'auto',        -- 'auto' (default) compacts between turns; 'off'
                          -- leaves only the manual /compact command
    grow_by = '35%',      -- sawtooth cadence: compact once this much new
                          -- context has accrued since the last compaction
    compact_at = '85%',   -- hard ceiling: always compact once live context
                          -- reaches this, overriding the other gates
    keep_recent = 20000,  -- recent-conversation budget always kept verbatim
    min_gain = 10000,     -- skip an automatic compaction that would drop less
    min_turns = 5,        -- minimum turns between automatic compactions

    -- Summarizing behaviour:
    enable_llm_summary = true,   -- false yields a deterministic-only summary
                                 -- while leaving auto-compaction on
    summarizer_model = 'ollama/llama3.1:8b',  -- [provider/]model[@effort]; any
                                 -- omitted part inherits the session's own
    summarizer_chunk = '25%',    -- per-chunk input budget for the map step
    summarizer_concurrency = 2,  -- max concurrent map calls (default 1)
    summarizer_timeout = '2m',   -- wall-clock bound on the whole pass
  }
end)

agent.configure_compaction takes every field as optional; an omitted field takes its built-in default. Token amounts accept an absolute count (20000, "20k", "1.5M") or a fraction of the model's context window (0.5, "35%"). A fraction cannot be resolved for a provider whose window is unknown (for example a local Ollama endpoint); in that case the trigger it controls is simply disabled, so prefer absolute counts for those providers. Durations like summarizer_timeout accept "500ms", "30s", "2m", or "1h" (a bare number is read as seconds).